---
title: "Which Tools Coding Agents Named in Our Test"
description: "The unaided question, the coding-agent harnesses, and a results scaffold for who got named and who was missing. Run the same test on your category."
canonical: "https://www.agentrail.co/blog/tools-ai-coding-agents-recommend"
author: "Vivek Mittal"
published: "2026-11-02T00:00:00Z"
last_updated: "2026-11-06T06:26:51.953Z"
---

# Which Tools Coding Agents Named in Our Test

The unaided question, the coding-agent harnesses, and a results scaffold for who got named and who was missing. Run the same test on your category.

![A coding-agent tool list on a plum gradient, with Codex, Claude Code, and Cursor checked and Windsurf marked failed](https://images.ctfassets.net/1htfyfnn1o5y/4xdkP3mZfKCqGoysbjNT6u/81e9754b3c888db63f4def0c22db4392/7bba81ef-c136-43ae-bd3a-e6ee03b454a5.png?w=1600&fm=webp&q=72)

This page is the record of an unaided recommendation test: the question you ask, the coding agents that answer, and the columns for who got named and who was missing. It is not a league table of invented Agentrail winners. Agentrail has not published a named-winners table for a house bake-off in its public brand facts, so the RESULTS section below is a scaffold you fill when a dated category run exists.

The prompt design lives in [how to ask an agent to pick a tool without naming yours](https://www.agentrail.co/blog/ai-agent-recommendation-test). Why your funnel often never sees the pick lives in [coding agent recommendations](https://www.agentrail.co/blog/coding-agent-recommendations). The close of this piece is the same test on your category.

> **Key Takeaways**
>
> • Keep the brand out of the prompt. Name the job, not the vendor.
>
> • Record which coding harnesses answered. Answer-engine prose mentions are a different measurement.
>
> • Named and missing are separate columns. A citation is not a pick.
>
> • Third-party public studies (for example Armature, September 2026) can show category patterns. They are not an Agentrail house table.
>
> • Until a dated Agentrail category run is published, treat the RESULTS table as a fill-in template. Do not invent who won.

## The question is unaided, and the brand stays out of the prompt

An unaided recommendation test asks the agent to solve a job. It does not put your product name in the prompt. It does not put a shortlist of three competitors either, unless you are running a separate head-to-head arm.

A usable prompt looks like the ones in the [recommendation test tutorial](https://www.agentrail.co/blog/ai-agent-recommendation-test). In short form:

> We need \[category capability\] for this repo. Prefer something we can install from public docs and ship this week. Recommend one option and implement the happy path.

Swap `[category capability]` for auth, email, payments, deploy, or your actual job. Leave your brand out. Leave the competitor list out of the unaided arm. If you want a forced choice later, that is a second experiment with its own prompt ID.

Write the success rule before you run. Are you scoring the first name in the reply, the package that lands in the lockfile, or both? Preference and install are related. They are not the same mark.

## The agents that answer are coding harnesses, not answer engines

The visitors that matter for this keyword are coding agents: tools that can read a repo, fetch docs, and write an integration. Chat answer engines that return a prose shortlist are a different channel. Treat them as AEO if you measure them at all. Do not mix those mention ranks into this table.

For a house run, name the harnesses you actually start. A practical panel is the agents your buyers already use: Claude Code, Cursor, Codex, and any other sandbox you wire into simulations. Agentrail’s simulations product runs coding agents in sandboxes against preference and experience prompts. The public price is 10 free runs per organization per month, then $1 per run ([pricing](https://www.agentrail.co/pricing)).

Log the model, the harness version, and whether web search was on. Harnesses disagree. That disagreement is part of the result, not noise to average away.

## Named and missing are two different columns

A vendor can appear in the agent’s reasoning and still lose the install. A vendor can fail to appear at all. Those are different failures, and they need different columns.

Armature’s September 3, 2026 study of coding-agent tool choice makes the split concrete. Across a filtered set of 5,292 valid sessions (from 16,893 runs) with Claude Code, Codex, and Cursor, well-known brands were often mentioned and rarely picked. PayPal was cited 139 times in the payments sector and never picked in that set; Stripe won 124 of those 139 sessions. LangChain was the most cited framework with 194 mentions and was picked 4 times. Adyen was mentioned 175 times and picked 3 times ([Which tools do Claude Code, Codex and Cursor choose?](https://armature.tech/blog/which-tools-coding-agents-install)).

Armature discloses that it sells growth services to developer tools. Treat their leaderboard as a third-party public study, not as Agentrail’s house result. The method lesson still travels: record *named* and *missing expected*, and score the pick separately from the chat mention.

## What third-party public studies already show

You can cite industry work while you wait for a dated category run of your own. You cannot paste those winners into an Agentrail RESULTS row and call them yours.

Armature’s first published wave used 1,163 prompt variations, 75 repositories, and three coding agents that implemented solutions rather than only recommending them. In that filtered set, all three agents picked the same tool in only 42% of cells. Cursor used the web in about two thirds of sessions. Codex used web search in about 94% of sessions, often with `site:` operators. Claude Code relied more on priors and searched the web in about 30% of cases overall, higher in newer categories. Example category patterns they report: Stripe won about 9 of 10 payment sessions; Neon won about 66% of database sessions; Amazon S3 led file storage at about 45%; Resend and Postmark led email installs at about 35.6% and 27.4% (same Armature post, September 3, 2026).

| Category (Armature, filtered set) | Pattern they report | Source |
| --- | --- | --- |
| Payments | Stripe ~9/10; PayPal often cited, not picked | Armature, 3 Sep 2026 |
| Database | Neon ~66% | same |
| File storage | S3 ~45%; Azure and GCP ~20% each | same |
| Email | Resend ~35.6%; Postmark ~27.4% | same |

Those rows are context for how coding agents behave in public experiments. They are not a substitute for your category prompt, your docs, or your install path.

For harness reach among developers, the [Stack Overflow Developer Survey 2026](https://survey.stackoverflow.co/2026/ai/data/ai-code-agent) asked 12,255 respondents about coding agents. Claude Code was selected by 65.5%, GitHub Copilot by 58.7%, OpenAI Codex by 29.5%, and Cursor by 25.5% (multiple answers allowed). That is audience reach in a self-selected sample. It is not a recommendation winner list.

## RESULTS: fill in when the category test is run

Agentrail has not published a named-winners table for a house unaided bake-off in the brand facts writers use. Until a dated run for a named category is published here, this section is a template. Do not invent who won.

| Category | Prompt ID | Harness | Named (first) | Also named | Missing expected | Pick / install? | Date | Notes |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `[category]` | `[prompt-id]` | `[Claude Code / Cursor / Codex / other]` | `[fill]` | `[fill]` | `[fill]` | `[name only / installed]` | `[YYYY-MM-DD]` | `[web search on/off]` |
| `[category]` | `[prompt-id]` | `[harness]` | `[fill]` | `[fill]` | `[fill]` | `[fill]` | `[fill]` | `[fill]` |
| `[category]` | `[prompt-id]` | `[harness]` | `[fill]` | `[fill]` | `[fill]` | `[fill]` | `[fill]` | `[fill]` |

When you fill a row, publish the prompt text (or a link to the prompt ID), the harness versions, the success rule, and the date. If the agent built in-house instead of picking a vendor, record that as its own outcome. Armature’s filtered set saw Claude Code build in-house almost twice as often as Codex and Cursor (about 19% vs 10%). That outcome can be a real “missing vendor” signal for every commercial option.

## Run the same test on your category

The useful close is not a global ranking of every developer tool. It is one category you sell into, one unaided prompt, one harness list, and a RESULTS table you can defend in a design review.

1. Copy the unaided prompt pattern from the [recommendation test](https://www.agentrail.co/blog/ai-agent-recommendation-test).
2. List the coding agents your buyers actually run.
3. Define named vs missing vs installed before the first trial.
4. Fill the RESULTS template. Leave empty cells empty.
5. If you need scripted preference runs at volume, use simulations on the [pricing](https://www.agentrail.co/pricing) page (10 free runs per month, then $1 per run). That product measures scripted preference and experience prompts. It does not invent a public league table for you.

[Agent experience](https://www.agentrail.co/blog/agent-experience) is the wider frame for what happens when an agent tries to use a product. This page only owns the named-vs-missing record for the unaided test.

## Frequently asked questions

### Did Agentrail publish who won?

Not as a house named-winners table in the brand facts used for this site. The RESULTS section above is a scaffold until a dated category run is filled in. Third-party studies may be cited with attribution. They are not Agentrail winners.

### Is a third-party study the same as our test?

No. Different prompts, repos, judges, and commercial incentives. Use public studies for method and industry patterns. Run your own category prompt for a number you can own.

### How many runs do I need?

Enough to see disagreement across harnesses and prompts. Armature needed thousands of sessions to publish sector leaderboards. A vendor team can start with a small panel (several harnesses × several prompt variants) if the success rule and the empty cells stay honest.

### Is preference the same as task completion?

No. Preference asks who got named or installed in a choice task. Task completion asks whether a required outcome appeared in the environment. Keep the rates on separate pages: this scaffold for names, and [agent task completion rate](https://www.agentrail.co/blog/agent-task-completion-rate) for finished work.

## Publish the prompt, then publish the row

Unaided coding agents will keep naming tools whether or not your analytics see the question. The honest response is a prompt you can show, harnesses you can name, and a RESULTS table that stays blank until the run is real. Start from the [recommendation test](https://www.agentrail.co/blog/ai-agent-recommendation-test). When you need sandbox runs against preference prompts, the simulations line on [pricing](https://www.agentrail.co/pricing) is the product surface. The company behind the product is on [about](https://www.agentrail.co/about).