---
title: "Agent task completion rate"
description: "Agent task completion rate is completed agent tasks divided by started agent tasks. The unit is the task. A benchmark score uses a different denominator."
canonical: "https://www.agentrail.co/blog/agent-task-completion-rate"
author: "Harsh Raj"
published: "2026-10-17T00:00:00Z"
last_updated: "2026-10-22T09:23:44.345Z"
---

# Agent task completion rate

Agent task completion rate is completed agent tasks divided by started agent tasks. The unit is the task. A benchmark score uses a different denominator.

![Improve Task Success title with a rising arrow on a purple gradient](https://images.ctfassets.net/1htfyfnn1o5y/KEA63Es1A8r9XifrABZ0d/57a1b3ed6bacdd6fab23e3b2f304e89e/hero_image.png?w=1600&fm=webp&q=72)

Teams already say “task success” and mean three different fractions. One is a usability lab. One is a scripted benchmark. One is whatever an agent claims in its last message. Those fractions do not describe the same work, and none of them is automatically the rate on your product.

Agent task completion rate needs a definition tight enough to cite. This page is that definition. It is not a league table, and Agentrail does not publish a house number for it.

> **Key Takeaways**
>
> – Agent task completion rate is the number of agent tasks that reach the required outcome, divided by the number of agent tasks that started.
>
> – The unit is the task. Pageviews, sessions, and tool calls are the wrong denominators.
>
> – Completion is a state in the environment. The agent’s closing sentence is not proof.
>
> – Partial success is reported next to the rate. It is not averaged in with made-up weights.
>
> – WebArena’s 14.41% and a SWE-bench range are benchmark scores. They are not this rate, and they are not an Agentrail result.

## Agent task completion rate is completed tasks divided by started tasks

Agent task completion rate is the number of agent tasks that reach the required outcome, divided by the number of agent tasks that started.

In symbols, that is completed tasks / started tasks. Multiply by 100 if you want a percentage. The definition does not change.

A required outcome is a state you can check. A key exists. A pull request was opened. A record matches the schema the task asked for. “The agent said it was done” is not an outcome.

There is no universal good rate in this definition. A payment task and a docs lookup do not share a sensible target. Publish the task’s outcome rule beside the percentage. A bare number cannot be compared, and it should not be cited as one.

Time, retries, cost, and abandonment belong to a wider set. That set is specified in [how to measure agent experience](https://www.agentrail.co/blog/how-to-measure-agent-experience). This page owns only the rate.

## The unit is the task, not the pageview or the tool call

The denominator is tasks that started. It is not requests.

One task can issue dozens of HTTP calls and several MCP tool calls. If you divide successes by calls, a careful agent that checks its work looks worse than a lucky agent that guesses once. If you divide by pageviews, a docs-heavy task looks less complete than a task that never opened docs and still failed.

The numerator has the same unit. A task is complete or it is not, under the outcome rule you wrote. Extra calls do not create extra successes. One task, one mark.

You cannot count tasks you have not bounded. Bounding is the job of [agent journey analytics](https://www.agentrail.co/blog/agent-journey-analytics): one task stitched across website, docs, API, SDK, and MCP. Until that stitch exists, a gateway error rate is a different statistic. Do not relabel it.

Nielsen Norman Group’s usability definition is the human ancestor, not the same formula. [Jakob Nielsen and Raluca Budiu](https://www.nngroup.com/articles/success-rate-the-simplest-usability-metric/) published it on 17 February 2001. The page was last reviewed on 20 July 2021. They define the success rate as the percentage of users who complete a task in a study. Their unit is the user in a lab. The unit here is an agent task in production traffic. Both are binary outcomes. The populations are not interchangeable.

## A task has started only when work toward a goal has begun

A started task is an agent task that has begun work toward a goal involving your product.

A training crawl has not started a task. A prefetch of `robots.txt` has not started a task. A single asset fetch with no goal you can name has not started a task. If you include them, the denominator inflates and the rate falls for a reason that has nothing to do with the product.

A start does not require a perfect statement of intent. Agents do not file a ticket titled with your success metric. A start requires evidence of pursuit: a docs read tied to a later call, an API attempt, an MCP tool invocation, an auth step aimed at a named action. Isolated noise stays out.

The start is also not the human’s prompt, which you often never see. You see the agent’s first relevant action on your surfaces. Count from there. If two agents pursue two goals, they are two starts, even when they share a vendor library and a minute on the clock.

Write the start rule down before you publish the rate. Otherwise the next reader will not know what you divided by, and the figure cannot be cited honestly.

## Completion is an outcome in the environment, not the agent’s last message

[Anthropic’s 9 January 2026 eval note](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) separates the transcript from the outcome. The transcript is the full record of the trial: messages, tool calls, reasoning. The outcome is the final state of the environment. Their example is a flight booking. The agent can say the flight is booked. The outcome is whether a reservation exists in the database.

Use that split in production. If the task was “create an API key,” completion is the key existing and working against the scope the task needed. A 200 from a different endpoint is not completion. A paragraph that describes a key is not completion.

Graders in that same note exist because the split is easy to get wrong. Code-based checks verify a state. They are brittle when several valid states exist. Model-based graders handle nuance and can be wrong. Human review is slow. For this rate, prefer a check you can re-run against state. Use a judge only when the outcome cannot be read from the system of record, and say that you did.

False completion is the failure mode to name beside the rate. The transcript ends in success language. The state does not match. If you count those tasks as complete, the rate measures eloquence. Exclude them from the numerator. You may count them as a separate quality problem. Do not bury them inside this fraction.

Evals and this rate still differ, even when both check state. Anthropic’s post is about tests you run on agents you configure, without live users. Agentrail’s distinction is the same one we use on the product: evals and simulations run tasks you script; production traffic is agents already arriving. Most teams want both. A sandbox score does not fill in this denominator.

## Partial success is reported beside the rate, not averaged into it

Strict completion will feel harsh, and it should stay available as its own number. Many tasks are partly done. NN/g’s flower example is the clear case. Ordering roses for a birthday can fail by the wrong flower, the wrong date, or a missing gift note. Some of those misses matter more than others.

NN/g’s rule for reporting is to give each level its own percentage. Their warning is sharper. Do not assign 1, 0.66, 0.33, and 0 to those levels and average them. The numbers are labels on an ordinal scale. You do not know that the gaps are equal. Their worked average, 53% in the article, is an illustration of the mistake. It is not a measured success rate. Do not cite it as one.

For agent tasks, publish at least two figures when partial work matters. The strict rate uses only full outcomes in the numerator. A second breakdown names the partial classes you actually care about: authenticated but not created, created but wrong scope, stopped after docs with no call. Do not collapse those classes into one blended percentage.

If you cannot name the classes, you are not ready to report partial credit. Report the strict rate and a count of failures. That pair is citable. A single blended score is not.

## A benchmark score is a different denominator

Published benchmarks are useful because they show how hard scripted tasks are. They are the wrong number to paste onto your dashboard.

[WebArena](https://arxiv.org/abs/2307.13854v4), Zhou and colleagues, evaluates functional correctness of web tasks in a built environment. The abstract states that the best GPT-4-based agent reached an end-to-end task success rate of 14.41%, against human performance of 78.24%. Cite that pair as a 2023–2024 baseline on those tasks. Do not cite it as the 2026 rate of agents on your product. The humans in that study were doing the benchmark, not using your signup flow.

SWE-bench Verified is a different script. Anthropic’s January 2026 post says a solution passes only if the tests that were failing now pass, and existing tests stay green. The same paragraph says LLMs moved from 40% to greater than 80% on this eval in one year. That sentence is Anthropic’s. The post does not include the leaderboard table, so this article does not invent a model-by-model row. The point of citing the range is the contrast. A coding-agent benchmark can move a long way while saying nothing about whether agents finish tasks on your API.

Both benchmarks grade an outcome, which is why they belong in a definition page. Both use tasks the benchmark author wrote. Your denominator is tasks that showed up. Mixing the two makes a chart that cannot be reproduced.

|  | Agent task completion rate | A public benchmark score |
| --- | --- | --- |
| Numerator | Tasks that reached your outcome rule | Tasks the harness marked correct |
| Denominator | Tasks that started on your product | The benchmark’s task list |
| Who wrote the task | The arriving agent’s goal | The benchmark authors |
| What it is not | A model leaderboard | Your production rate |

Neither figure is a target for your product. A floor, if you set one, belongs to a named workflow whose failure is expensive. It is not imported from a paper.

## Consistency is not the same as one lucky trial

A single success does not mean the task is reliable. Anthropic separates pass@k from pass^k. pass@k asks whether at least one trial succeeded. pass^k asks whether all k trials succeeded.

Their worked example is arithmetic, not a field study. If the per-trial success rate is 75% and you require three trials, pass^k is 0.75³, about 42%. At k=1 the two metrics match. As k grows, pass@k climbs toward certainty that some trial worked, and pass^k falls because every trial must work.

Use the example to keep this rate honest. Agent task completion rate, as defined here, counts tasks in traffic, each once. It is closer to a per-task result than to pass^k. If the same goal is attempted many times and only some finish, say so beside the rate. Do not quote a best-of-n run as if it were the traffic rate.

That is as far as this page goes into the neighboring metrics. Variance, cost per success, and recovery belong with the measurement set, not in a second copy of it.

[Agent analytics](https://www.agentrail.co/blog/agent-analytics) is the category for reading production tasks this way. Journeys are how we hold the unit: 100 included per organization each month, then $0.10 a journey, on the [pricing page](https://www.agentrail.co/pricing). The price is not a benchmark. The rate is still yours to define with the rules above. We will not fill it in for you with a number we did not measure in public.

## FAQs

### What is agent task completion rate?

Agent task completion rate is completed agent tasks divided by started agent tasks. A task is complete only when the required outcome is true in the environment. The agent’s last message is not the outcome.

### What should the denominator include?

Tasks that began work toward a goal on your product. Training crawls, bare asset fetches, and requests with no pursuable goal stay out. The boundary of one task is a journey, defined separately.

### Why not use WebArena or SWE-bench as the rate?

Those scores use a fixed task list and a harness. WebArena’s abstract reports 14.41% for the best GPT-4 agent in that study, versus 78.24% for humans. SWE-bench Verified grades software patches. Neither denominator is the set of tasks that arrived on your product.

### Should partial success count as a fraction of a task?

No. Report full completion as its own rate. Report partial classes as their own percentages. Nielsen Norman Group warns that averaging ordinal labels such as 1, 0.66, and 0.33 is not a valid success rate.

## Conclusion

Agent task completion rate is a fraction with a task in both places and an outcome you can check. Benchmarks can inform how you write the check. They do not replace the traffic.

State the start rule, the outcome rule, and the partial classes you refused to average. Then cite the rate. The wider set of measures sits in [how to measure agent experience](https://www.agentrail.co/blog/how-to-measure-agent-experience). The place those tasks are read together is [agent analytics](https://www.agentrail.co/blog/agent-analytics).