> ## Documentation Index
> Fetch the complete documentation index at: https://agentrail.co/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent Preference

> Find out which products coding agents discover, recommend and choose when a developer asks for something your product does.

Developers increasingly ask a coding agent which product to use. Agent Preference runs the requests your buyers would type, and shows how often agents reach for your product instead of a competitor's, and why.

## How it works

You write **preference prompts**: requests a developer would type to a coding agent. Each time a prompt runs, one agent with one model answers it on its own, and Agentrail records which products it found, named, recommended and finally chose. The **Agent Preference** page adds those runs up.

A prompt has one of two types:

| Type | What it does |
| - | - |
| **Open-ended** | Describes a problem and lets the agent choose. |
| **Head-to-head** | Names two products and asks the agent to pick one. You enter the other product as its **Alternative**. |

In the results, prompts fall into three prompt types:

* **Organic**: open-ended prompts that do not name your product.
* **Branded**: open-ended prompts whose text names your product, or one of its **Aliases** from the [Brand Vault](/docs/guides/brand-vault).
* **Head-to-head**: head-to-head prompts.

Results cover your own product and the products on your [Competitors](/docs/guides/competitors) page. A product an agent picks that is on neither list shows as **Not listed**.

<Tip>
  Agentrail recognises products by the names you give them. Add the other names people use: **Aliases** in the Brand Vault, **Other names** on a competitor.
</Tip>

## Add a prompt

Only an Admin can add, edit, run or delete prompts.

<Steps>
  <Step title="Open the prompt list">
    Open **Agent Preference** in the sidebar and scroll to **Prompts**.
  </Step>

  <Step title="Start a new prompt">
    Click **Add prompt**. The **Add preference prompt** dialog opens.
  </Step>

  <Step title="Fill in the prompt">
    Choose a **Type**, then write the **Prompt** the way a developer would ask a coding agent. For a head-to-head prompt, enter the **Alternative**: the other product the prompt names.
  </Step>

  <Step title="Group it">
    Choose a **Topic**, and any **Tags**. See [Topics and tags](/docs/guides/topics-and-tags).
  </Step>

  <Step title="Choose how it runs">
    Pick a **Repository** for runs to start from, or leave it on **Empty project**. Tick the **Agents and models** to run it with. See [Executions](/docs/guides/executions).
  </Step>

  <Step title="Save">
    Click **Add prompt**.
  </Step>
</Steps>

A prompt without a topic is flagged as needing attention and cannot run until you give it one. To list those prompts, turn on **Needs attention** above the list. A head-to-head prompt with no alternative shows **No alternative**; edit it to add one.

To change or remove a prompt, open the actions menu on its row and choose **Edit** or **Delete**. Deleting removes the prompt from every list, along with its tag assignments. Editing changes future runs only: each run keeps the text it ran with.

## Run a prompt

Open a prompt's actions menu and choose **Run**, or open the prompt and click **Run**. Agentrail starts one run for each agent and model the prompt has, and says how many it started. Prompts also run on their own, on the schedule set in **Run settings**. See [Executions](/docs/guides/executions) for both.

## Read the results

The top of the **Agent Preference** page sums up runs in the chosen view:

* Prompt type: **All prompt types**, **Organic**, **Branded** or **Head-to-head**.
* Period: **Last 7 days**, **Last 30 days** (the default), **Last 90 days**, **Last year** or **All time**, by when runs ended.
* Agent and model (**All agents**) and tag (**All tags**).
* A tab for each topic, after **All topics**.

Each figure is a percentage with the runs behind it, such as "10 of 16 runs". When nothing can be counted it says **No measurable results**, never 0%.

<AccordionGroup>
  <Accordion title="Ranking">
    Every product by how often agents **Recommended** it, with how often it was **Chosen**. Click a product to open its page.
  </Accordion>

  <Accordion title="Head-to-head">
    Your product against each alternative your head-to-head prompts name: wins and losses, and whether you are **Winning**, **Tied** or **Losing**. The **neither** count is runs where the agent picked neither product, or a third one. **Show head-to-head prompts** lists the prompts behind it.
  </Accordion>

  <Accordion title="Your product over time">
    How often agents mentioned, recommended and chose your product, per day, week or month depending on the period.
  </Accordion>

  <Accordion title="How far each product got">
    Four steps for each product: **Discovered** (it turned up in a web search or a page the agent read), **Mentioned** (the agent named it), **Recommended**, and **Chosen** (it was the agent's final choice). Each step counts out of every run, with the share of the step before that reached it.
  </Accordion>

  <Accordion title="Who wins">
    Every final choice, including products nobody listed and runs with no choice: **None: asked a question**, **None: listed options without choosing** and **None: no answer**.
  </Accordion>

  <Accordion title="Reasons against">
    Runs where an agent gave a reason not to use a product, in its own words. **Step** links open the run's log where it said so.
  </Accordion>

  <Accordion title="From memory or after research">
    Per agent and model, how often it **Answered from memory**: without searching or reading a page first.
  </Accordion>

  <Accordion title="Choice implemented">
    Of the runs whose final choice was your product or a competitor, how often the code uses that product. Filter to prompts that ask for code to make this meaningful.
  </Accordion>
</AccordionGroup>

### A prompt's page

Click a prompt in the list to open it. It shows the prompt's settings, **Results for this prompt** over the last 30 days across every agent, and every run of it under **Executions**.

### A product's page

Click a product name to see its results on its own, the reasons agents gave against it, and **Runs that mentioned it**, each with the step it was first named.

## Runs that are not counted

Results count runs that **Finished**, **Failed** or **Timed out**. When a run is left out, the figure says why under **Not counted**:

* **not judged yet**: the run has just ended. Results arrive a few minutes after a run ends.
* **judging failed**: Agentrail could not work out this run's result.
* **judged with an earlier question**: the run will count once it is read again.
* **interrupted**: the run stopped for a reason on Agentrail's side.

A figure may also say how many runs were left out because the agent cannot show that detail, or how many came from an incomplete record.

## Limits

| What | Limit |
| - | - |
| **Prompt** | 4,000 characters |
| **Alternative** | 200 characters |
| **Tags** on one prompt | 20 |

## Who can do what

| Action | Admin | Member |
| - | - | - |
| See results, prompts and runs | Yes | Yes |
| Add, edit and delete prompts | Yes | No |
| Run a prompt | Yes | No |

See [People and roles](/docs/guides/people-and-roles).

<CardGroup cols={2}>
  <Card title="Agent Experience" icon="list-checks" href="/docs/guides/agent-experience">
    Give agents a task with your product and see how far they get.
  </Card>

  <Card title="Executions" icon="history" href="/docs/guides/executions">
    Start runs, set the schedule and read each run.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.