---
title: "First-try success is not eventual success"
description: "First-try success means the first attempt reached the outcome. Eventual success means a later attempt did. Compare both before you publish one success number."
canonical: "https://www.agentrail.co/blog/agent-first-try-success-rate"
author: "Harsh Raj"
published: "2026-11-12T12:12:00Z"
last_updated: "2026-11-16T06:16:56.950Z"
---

# First-try success is not eventual success

First-try success means the first attempt reached the outcome. Eventual success means a later attempt did. Compare both before you publish one success number.

![A white checklist icon on a dark blurred background](https://images.ctfassets.net/1htfyfnn1o5y/1KDfp24S6dOrTK899WM8wI/889c5d165ad14346c2c62e75ef1af789/b1121044-71c3-4b0f-b4ab-740fae80642e.png?w=1600&fm=webp&q=72)

The agent first try success rate and eventual success are two readings of the same tasks. First-try success means the outcome was reached on the first attempt. Eventual success means a later attempt reached the outcome, and it includes tasks that were already done. How a finished task enters the completion fraction stays in [agent task completion rate](https://www.agentrail.co/blog/agent-task-completion-rate).

> **Key Takeaways**
>
> First-try success marks the first attempt. Eventual success marks whether any attempt reached the outcome.
>
> Immediate successes sit in both readings. Rescues sit in eventual success only. Tasks that never finish sit in neither.
>
> Publish the gap, the tasks that needed a later attempt, beside the two readings.
>
> A public Pass@1 is a first-attempt score on benchmark tasks. It is not either reading on your product.

## Mark the first attempt, then any later one

Keep the task set fixed. On each task, mark two facts. The first attempt reached the outcome, or it did not. A later attempt reached the outcome, or it did not.

First-try success uses only the first mark. Eventual success uses the later mark and keeps every task that was already done. If the agent never makes a second attempt, the two readings match for that task. A handback or a hard error can end the run there. An unobserved second try is missing data, so leave the rescue cell empty.

Publish the gap as its own line. The gap is the set of tasks that needed a later attempt. Averaging the readings invents a third number with no question attached. Adding them double-counts every immediate success.

Write down what an attempt is before you mark anything. A refetch of the same doc after an empty page is another attempt. The same method, path, and body, sent again after an error, is another attempt. The next endpoint the task required is a new step. A pagination call that advances a cursor the task needed is progress. A call that repeats because the response never named a cursor is a retry.

If the record has no order, publish a note that attempt order is missing. Don't guess which call was first. Ordering those attempts on one task is [agent journey analytics](https://www.agentrail.co/blog/agent-journey-analytics).

## A lab trial starts clean

[Anthropic's eval note](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) of 9 January 2026 treats each attempt as a trial. Outputs vary, so they run more than one. pass@k credits the task if any attempt in the set succeeds. pass^k credits the task only if every attempt succeeds. With one attempt, the two summaries match. With more, they diverge. In coding, the note treats the first try as the reading that usually matters.

Those summaries belong to a harness. You pick the tasks, start each trial from a clean state, and grade them yourself. A production journey is one task. Inside it, a failed call and the retry that follows are two attempts on that same task. The second call still has the state the first call left.

Pass@1 asks whether the attempt in front of the grader succeeded. [SWE-Bench Pro](https://arxiv.org/abs/2509.16941) reports that shape under a unified scaffold. Widely used coding models stayed below 25% Pass@1. GPT-5 is the highest score the abstract cites, at 23.3%. That pair is a first-attempt result on those coding tasks. Leave it on the paper.

pass^k is the stricter lab reading, for a job that has to succeed on every run. Keep that name on the harness.

## The gap holds the calls the headline drops

Eventual success counts the task when any attempt reached the outcome. The failed calls stay in the transcript and drop out of the headline. Latency, rate limits, and tickets live in that dropped stretch. A real rescue still belongs in the eventual reading. The agent read an error and then sent a valid request. The outcome exists. Report the failed attempts beside the finish, as the gap.

Eventual success can rise while the first path stays clumsy. The finish rate looks fine, and the first call is still wasted.

[Anthropic's tool essay](https://www.anthropic.com/engineering/writing-tools-for-agents) of 11 September 2025 treats that repeat as a design signal. Invalid-parameter errors usually mean the description or the examples are thin. Repeated calls usually mean pagination, filtering, or defaults are wrong. When validation fails, the error should name a correction. An opaque code gives the model nothing to change, so the next attempt sends the same request.

A short step budget can remove the rescue. Pan and colleagues surveyed 306 practitioners from April through November 2025 and reported on 86 systems in production or pilot. Among those 86 systems, 68% execute at most 10 steps before a human intervenes ([Measuring Agents in Production](https://arxiv.org/abs/2512.04123), revised 3 February 2026). Many of those systems stop, or ask a person, before a long rescue can run. On that traffic the first attempt is the whole story. The 68% is a share of those 86 systems. It is not your rate.

## Read the pattern, then change the first response

High eventual success and low first-try success means the outcome is reachable and the first path is unreliable. Look at the first response. Name the surface, the action, and what came back. A thin error, a doc that lacked the fact, or a sample the API rejected is the edit.

Low on both readings means the outcome is not showing up. Another automatic retry copies the failed call. Classify the stop, then change the surface that blocked the task. Compare the readings again after that change.

| Reading | Question | What it hides |
| --- | --- | --- |
| First-try | Did the first attempt reach the outcome? | Later attempts that did finish |
| Eventual | Did any attempt reach the outcome? | The failed attempts before the finish |
| Gap | Which tasks needed a later attempt? | Why the first attempt failed |

Fill the table from your tasks. Leave a cell blank until your own marks exist. The change that removes a wasted first attempt sits in [agent experience optimization](https://www.agentrail.co/blog/agent-experience-optimization). Run the comparison again on the same tasks, one surface at a time.
