---
title: "A Recommendation and a Finished Task Are Different Measurements"
description: "A readiness grade, a preference result, and a task result answer find, choose, and finish. Use each for the question it owns."
canonical: "https://www.agentrail.co/blog/measure-ai-agent-recommendations"
author: "Harsh Raj"
published: "2026-11-03T00:00:00Z"
last_updated: "2026-11-03T11:27:13.672Z"
---

# A Recommendation and a Finished Task Are Different Measurements

A readiness grade, a preference result, and a task result answer find, choose, and finish. Use each for the question it owns.

![Three metric cards on a dark rose gradient: a rising green chart, a flat gray line, and a falling red chart](https://images.ctfassets.net/1htfyfnn1o5y/17Hh1KlTdpUDrEFKr9xZ18/494a4debeb85213d65ca3f86c818e5e1/f8429b67-59e3-47a1-9351-ede67971abe5.png?w=1600&fm=webp&q=72)

Teams often ask how to measure AI agent recommendations and then put three different instruments on one chart. A public readiness grade, an unaided preference result, and a finished task look related. They answer different questions. One asks whether an agent can find and read what you published. One asks whether an unnamed task names you. One asks whether the environment reached the goal.

Mixing them invents a score nobody can cite. A high grade does not prove agents recommend you. A preference win does not prove the install finished. A finished task does not prove agents would have chosen you without your name in the prompt.

This page owns measurement selection for those three instruments. The wider metric set lives in [how to measure agent experience](https://www.agentrail.co/blog/how-to-measure-agent-experience). The rate formula lives in [agent task completion rate](https://www.agentrail.co/blog/agent-task-completion-rate). Readiness as a property lives in [what is agent readiness](https://www.agentrail.co/blog/what-is-agent-readiness). How to design the preference prompt lives in the [AI agent recommendation test](https://www.agentrail.co/blog/ai-agent-recommendation-test).

> **Key Takeaways**
>
> • Find, choose, and finish each need their own instrument.
>
> • A readiness grade checks declared public surfaces. It does not prove a recommendation or a finished task.
>
> • A preference result checks whether an unnamed task names you. It does not prove the environment reached the goal.
>
> • A task result checks environment state. It does not prove unaided choice.
>
> • Do not average the three into one recommendation score.

## Three questions need three instruments

[Agent experience](https://www.agentrail.co/blog/agent-experience) is the encounter an AI agent has while discovering, choosing, and using your product. Those three verbs are the selection key. Find asks whether the public contract is reachable. Choose asks whether the agent names you when the task does not. Finish asks whether the required state exists when the run ends.

[Microsoft’s 6 October 2026 essay](https://developer.microsoft.com/blog/what-is-agent-experience-ax/) splits choose from correct use. Propensity asks whether the agent finds your technology and selects it on an unnamed task. Efficacy asks whether, once the product is named, the agent uses the current interface. Keep that split. A preference result is the propensity reading. A task result covers whether use actually landed in the world.

The rest of the measurement set still matters. Retries, dead steps, repeatability, and path cost sit beside these three. This page only decides which instrument matches which question. Definitions stay on the measurement guide linked above.

## A readiness grade answers whether the public contract is declared

Use a readiness grade when the decision is about the public product an agent can reach without a login. The grade asks whether surfaces for find, read, use, and pay are on the record. That property is defined in [what is agent readiness](https://www.agentrail.co/blog/what-is-agent-readiness). The [agent readiness leaderboard](https://www.agentrail.co/agent-readiness) is one public scan of it.

A measurement essay updated 8 October 2026 states the limit plainly. A declared-surface score is a precondition. It does not predict that an agent will succeed ([How to measure Agent Experience](https://agentexperience.tech/insights/measure-agent-experience/)).

Cloudflare’s 17 April 2026 crawl shows why presence alone misleads. In a filtered set of high-traffic domains, `robots.txt` appeared on 78% of sites. Content Signals appeared on 4%. Markdown content negotiation passed on 3.9%. MCP Server Cards and API Catalogs together showed up on fewer than 15 sites ([Introducing the Agent Readiness score](https://blog.cloudflare.com/agent-readiness/)). A crawler file can exist while agent-usable surfaces stay rare.

A win on this instrument means the scan found the declared surfaces it checks. It does not mean agents recommend you. It does not mean a production task finished.

## A preference result answers whether an unnamed task names you

Use a preference result when the decision is about choice. The prompt describes the job and leaves your brand out. You record what the agent names, installs, or reaches for across repeated runs on a recorded agent profile.

Microsoft’s propensity example is the shape. Ask the agent to add email sending. Do not name a vendor. See what it reaches for. That is a choose measurement. It is not a finish measurement.

You will not always see that choice in production analytics. The pick often completes inside a coding agent before your funnel records a session worth attributing. That blind spot is the topic of [coding agent recommendations](https://www.agentrail.co/blog/coding-agent-recommendations). A scripted preference test closes the gap that silence leaves.

How you write the unaided and head-to-head prompts belongs in the [AI agent recommendation test](https://www.agentrail.co/blog/ai-agent-recommendation-test). This page only places the result. A win means agents named you under the profile you recorded. It does not mean the key was created, the payment cleared, or the MCP call returned a usable payload.

## A task result answers whether the environment reached the goal

Use a task result when the decision is about finished work. The numerator is tasks that reached the required outcome. The denominator is tasks that started. The full definition is [agent task completion rate](https://www.agentrail.co/blog/agent-task-completion-rate).

[Anthropic’s 9 January 2026 eval note](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) separates the transcript from the outcome. The agent can say the flight is booked. The outcome is whether a reservation exists in the database. Grade the state you can check. Do not grade the closing sentence.

Self-report can diverge sharply from that state. WebPageBench, a 28 September 2026 preprint, grades web-agent tasks from each mock site’s event log and stores the agent’s completion claim separately. On the public 152-task leaderboard, the gap between declared finished and log-confirmed reached 41 points. One configuration declared every task finished and met the conditions on 59% ([arXiv:2609.35026](https://arxiv.org/abs/2609.35026)). The suite uses instrumented mock sites. Treat the gap as evidence that claims are not outcomes, not as your production rate.

A win on this instrument means the required state is true. It does not mean agents would have chosen you without your name in the prompt. Partial outcomes stay beside the rate. They are not averaged into it with invented weights.

## The three instruments disagree on purpose

Expect the three readings to disagree. That disagreement is the point. A product can publish strong discovery files and still lose unaided choice. A product can win the name and still die at auth. A product can finish scripted tasks for agents that already pointed at it and still vanish from unnamed prompts.

| Question | Instrument | What a win means | What it cannot prove |
| --- | --- | --- | --- |
| Can an agent find and read what we published? | Readiness grade | Declared public surfaces pass the scan | Recommendation or finished task |
| Does an unnamed task name us? | Preference result | Agents select you on a recorded profile | Environment reached the goal |
| Did the work finish? | Task result | Required state is true in the system of record | Unaided choice without your brand |

Efficacy, the named-product correct-use question, belongs with the task and path family in the measurement set. Do not bury it inside the preference column. A head-to-head that names two vendors still measures choice. A run that names you and checks whether the current API was called measures use.

The anti-pattern is a single “recommendation score” that weights grade, name rate, and completion. The weights are fiction. The blend cannot be cited. Keep three columns.

## Pick the instrument from the decision you need to make

Start from the decision, then pick the chart.

If you are deciding whether to ship missing public surfaces, use the readiness grade. Fix the declared gaps the scan names. Remeasure the grade. Do not treat the lift as proof that agents now recommend you.

If you are deciding whether agents choose you in an unnamed category task, use the preference result. Script the task. Record the profile. Repeat the runs. Change the docs or discovery surfaces that shape choice, then repeat. The tutorial for that prompt design is linked above.

If you are deciding whether integrations finish, use the task result. Bound the task. Check environment state. Name the dead step when it fails. Hand the named step to optimization work. Do not relabel a preference win as completion.

Most platform teams eventually need all three readings. They still should not pool them. Production journeys show tasks agents already attempted and where those attempts stopped. Scripted simulations cover preference and named-product use on a held profile. We keep those as separate products for that reason. Agent readiness is included. Public pricing for journeys and simulations is on the [pricing page](https://www.agentrail.co/pricing).

## Frequently asked questions

### Can one number cover recommendations and completions?

No. A recommendation is a choose result. A completion is a finish result. Averaging them hides which decision is still open. Report each instrument beside the question it answers.

### Is a readiness grade a preference result?

No. A readiness grade inspects declared public surfaces. A preference result records what an agent names on an unnamed task. A strong grade can sit next to a lost unaided pick.

### Does a preference win mean the task finished?

No. Naming you is not the same as creating the key, calling the current route, or leaving a required row in your system. Check environment state for finish.

### Where are the full metric definitions?

The set is in [how to measure agent experience](https://www.agentrail.co/blog/how-to-measure-agent-experience). The completion fraction is in [agent task completion rate](https://www.agentrail.co/blog/agent-task-completion-rate). Readiness as a property is in [what is agent readiness](https://www.agentrail.co/blog/what-is-agent-readiness).

## Conclusion

A readiness grade, a preference result, and a task result are different measurements. Use the grade for find. Use preference for choose. Use the task result for finish. Keep them side by side when the story needs all three.

If you need the definitions behind those instruments, start with [how to measure agent experience](https://www.agentrail.co/blog/how-to-measure-agent-experience). If you need the preference prompt itself, use the [AI agent recommendation test](https://www.agentrail.co/blog/ai-agent-recommendation-test). If the open question is whether work finished, use [agent task completion rate](https://www.agentrail.co/blog/agent-task-completion-rate). Read the attempt through [agent analytics](https://www.agentrail.co/blog/agent-analytics).