Versions and experiments

Saved versions are named snapshots of Agent configuration. Experiments distribute new conversations across two or more saved versions so their outcomes can be compared. These APIs use the live agentConfigId, not the Agent account ID. Read the live Agent configuration before constructing a version path.

​

Save and manage versions

Version routes are rooted at: /api/v2/agent/{agentConfigId}/versions A new version snapshots staging by default. Select live only when the endpoint schema explicitly supports that source and it is what the user intends.

• Replacing a version updates the saved snapshot; it does not update staging or live.
• Renaming changes only the version title.
• Restoring targets staging by default, so test the restored configuration and publish it separately.
• Do not delete a version used by an active experiment or retained Task history.

Read the version again after create, replace, rename, or restore.

​

Create an experiment

Create candidate versions first. Then use the experiment routes with targetAccountId for account scope and agentConfigId for the live configuration being tested. Each experiment needs at least two distinct numeric versionId values with positive weights. Variant configuration can change only while the experiment is a draft. Only one experiment for an Agent configuration can be active at a time. Use this lifecycle:

1. Save and verify every candidate version.
2. Create the experiment as a draft.
3. Verify version IDs, labels, and weights.
4. Start the experiment.
5. Inspect assignments and results using the same reporting definitions across variants.
6. Pause when investigating a problem; end when no more assignments should occur.
7. Delete only after it is no longer active and its definition is no longer needed.

​

Plan the measurement first

Before activating an experiment, record:

• the hypothesis and the Agent behavior expected to change;
• one primary metric and its unit: conversation, Task attempt, or terminal Task outcome;
• the exact denominator and any reporting-field null behavior;
• guardrail metrics that would expose a worse customer or operational outcome;
• the audience, channels, and time range that belong in the comparison.

Establish a baseline and verify that each candidate version produces the same reporting fields. Where practical, change one behavior at a time between versions so the result has a useful interpretation. Use stable version titles that describe the tested difference rather than labels such as “new” and “old.”

​

Compare meaningful outcomes

Report sample size and denominator with every result. Keep reporting fields consistent across versions; otherwise apparent performance differences can be caused by incompatible measurement. Retry attempts preserve their experiment assignment. Evaluate the intended unit—conversation, attempt, or final Task outcome—rather than mixing them. Conversation and Task-attempt logs retain the experiment ID, variant label, and saved version ID. Use experiments.id and experiments.variant filters with the Logs or time-series APIs to calculate the same metric for each variant. Compare like-for-like populations, channel mix, and time ranges; do not treat raw rates with materially different event counts as equivalent evidence. Use minimumSampleSize when small groups or buckets should be withheld. Do not edit variant membership while the experiment is active. Pause it before investigating a guardrail regression or changing the draft definition, and end it when no new assignments should occur.