Skip to main content
Every result on Agent Preference and Agent Experience comes from runs. A run is one prompt or evaluation, run once by one agent with one model. Executions lists every run in the workspace, newest first, and opens each one so you can check what the agent actually did.

​
Where runs come from

A run appears in Executions as soon as it starts, from any of these:
  • Run on a preference prompt or experience evaluation.
  • Re-run on a run’s page.
  • Finishing Setup.
  • The prompt schedule, which you set in Run settings.

​
Start a run

Only an Admin can start runs.
1

Open the prompt or evaluation

On Agent Preference or Agent Experience, click the one you want to run.
2

Check what it runs with

Repository shows the code each run starts from, and Agents and models which agents run it. To change either, click Edit.
3

Run it

Click Run. Agentrail starts one run for each agent and model, and says how many it started.
Run is unavailable while a prompt or evaluation needs attention, or has no agent and model to run with. A run also cannot start while its repository is still being prepared, or if it could not be prepared; the message says which.
If Agentrail says The runs may have started, check Executions before you click Run again, or you may start the same runs twice.

​
Choose agents and models

Agents and models lists every agent and model you can run, as “Agent · Model”. Each run is one agent with one model, and Run starts one run for each one ticked. On a new prompt or evaluation the defaults are ticked already. When an agent and model a prompt used is retired, it is no longer listed; changing the list removes it.

​
Choose a repository

Repository is the codebase each run starts from. A preference prompt starts from an Empty project unless you choose a repository. An experience evaluation always starts from a repository: the default one unless you choose another. See Repositories below.

​
Run on a schedule

Prompts and evaluations also run on their own. Click Settings on the Executions page to open Run settings:
  • Samples: how many runs each prompt and evaluation gets on each of its agents and models in a period. Runs started by hand count too.
  • Cadence: Daily (each day from midnight UTC) or Weekly (each week from Monday, midnight UTC).
Until an Admin changes them, every prompt and evaluation runs once a day on each of its agents and models. A change applies from the schedule’s next check. Prompts and evaluations that need attention do not run until they are fixed. Members can see these settings but only an Admin can change them.

​
Find a run

Above the list, choose All, Experience evaluations or Preference prompts, then narrow by agent and model (All agents and models), repository (All repositories) and status (All statuses). Choosing a product under All products shows the runs that mentioned your product or a competitor, and adds filters for whether the run mentioned it and whether its code used it. The running and waiting counts at the right are runs in progress; click one to list those runs. Each row shows the agent’s model and the prompt, its Source, the Result, the Spend and when it ran under Ran at. A run that did not simply finish shows its status: A prompt’s own page lists its runs under Executions, with View in Executions to see them here.

​
Read a run

Click a run to open it. The top shows the agent’s session, its model, the prompt it ran (with Copy prompt), and Repo, Commit, Time, Duration, Tokens, Cost and Result. If the run has more than one attempt, choose one from Attempt; each keeps its own time, spend and record. While a run is going, Progress shows where it is, and the log so far refreshes every 10 seconds. Once it ends:
  • Logs: what the agent did, step by step. focus shows messages, reasoning and tool use; all shows every event. Turn on net to see each lookup and connection among the agent’s steps.
  • Files: every file the agent added, changed, deleted or renamed, with the change itself.
  • Network: every lookup and connection the agent made.
  • Judgment: what Agentrail found in the run. For a preference prompt, the final choice and what the agent said about each product; for an evaluation, what it claimed, how far it got with your product, and what failed. Each answer links to the step that shows it. Judgment arrives a few minutes after the run ends.
  • Coding agent web activity: the searches it ran and the pages it read.
Once the agent has run, Download code saves the project as it left it. View prompt or View evaluation opens what it ran. An Admin can click Re-run to start a new run with the same agent and model, from the prompt’s current text and repository.

​
Repositories

Repositories, under Product in the sidebar, lists the starter repositories a run can start from. Each is prepared ahead of time, with its setup already done, so a run starts from its code at once. The list is the same for every workspace.

​
Who can do what

See People and roles.