Abstract editorial illustration on a deep navy field: a wide constellation of small pale-blue agent nodes is arranged in loose concentric bands around a large, dark central node. Thin faint arcs connect bands of nodes to the centre, and a single narrow ringed aperture in a translucent boundary sits at the lower right. Fine dotted grids and unlabelled tick marks fill the background. No text, no letters, no numbers, no people, no organisation marks and no logos appear in the artwork; the OrcaRouter logo is composited in the bottom-right corner.
AI Safety Incidents

Anthropic: Claude "leads" 26% of its AI R&D with 30,000 agents running in parallel

Author

Elias Hawthorne

Date Published

Latest models · 20View all models →
Benchmarks: Artificial Analysis · updated daily
Back to all posts

Anthropic published an R&D Automation Index on 17 September 2026 — a prototype measure of how much of the company's own AI research and development Claude performs — and reported that by August the model "led" 26% of that work, up from under 1% in February. More than 90% of the measured work sat at "AI collaborates" or above, and roughly 30,000 agents were running research and engineering tasks at any one time on Anthropic's most-used internal platform. Nothing was breached and no third party was harmed: this is a disclosure about a capability trend inside one laboratory, and Anthropic states that Claude operates fully autonomously — its top level, AL5 — for no measured subset of the work. The archive's full record of the disclosure, with its grading and cited sources, sits in the AI incident archive alongside the other governance entries in this series.

What the index actually counts

The index borrows Epoch AI's six-level scale. AL0 is "no AI involvement", AL1 "minimal AI involvement" and AL2 "AI assists". AL3 is the level Anthropic calls "collaborates": the model "can do large chunks of work under close human direction". AL4 is "leads": it "can complete most of the task end-to-end from a high-level prompt, while the human supervises". AL5 — "AI operates fully autonomously, with no human in the loop" — is what the recursive self-improvement discussion is ultimately about, and Anthropic's page calls it "a level we have not yet reached."

The measurement was built in July 2026. Anthropic sampled 20% of staff in the relevant departments each week, had a Claude research agent review their work records, and produced a list of about 15,000 tasks, which the company organised into a frozen 542-node tree with 378 leaf tasks. Categories were weighted by how much human person-time they absorb. InfoQ's account of the publication adds a detail that matters for anyone whose work was sampled: the Claude agent was reading Slack messages and internal documents to reconstruct what people did.

Two clarifications follow from that method, and both are easy to lose under the headline. The 26% is not 26% of every task Anthropic recorded; as heise reported on 18 September, it is the share of the weighted work basket that Anthropic assesses as reaching AL4. And it does not mean that a quarter of the company's R&D runs unmanned — InfoQ is explicit that the figure does not mean 26% of Anthropic's R&D is fully automated, nor that 26% of the company's researchers have been replaced; it means roughly a quarter of the work now sits at the stage where a human sets the goal and supervises while Claude does most of it.

Self-authored HTML card headed 'R&D automation index, as of August 2026' with the title '26% by August, from under 1% in February' and the subtitle 'Anthropic's person-time-weighted task basket, scored on Epoch AI's six-level automation scale.' A vertical ladder lists six levels: AL0 no AI involvement; AL1 minimal AI involvement; AL2 AI assists; AL3 collaborates, large chunks of work under close human direction; AL4 leads, most of a task end-to-end from a high-level prompt while a human supervises, marked at 26%; and AL5 fully autonomous, with no human in the loop, marked at 0%, described as a level not yet reached. Four stat tiles read: 26% at AL4, August 2026; under 1% at AL4, February 2026; above 90% at AL3 or higher; about 30,000 concurrent agents on the most-used internal platform. A blue-bordered quotation band carries the line that Claude is not operating fully autonomously for any measured subset of AI R&D work. A footnote states that the 26% is a share of the weighted basket rather than a raw task count, that it is not 26% of Anthropic's R&D running without people, and names Anthropic's index page, heise of 18 September 2026 and InfoQ of the same date as sources. The OrcaRouter logo sits in the bottom-right corner.

The February comparison comes from the chart caption on Anthropic's page: "Claude now leads 26% of Anthropic's model R&D tasks, up from under 1% in February 2026." Nothing on the page gives a month-by-month series, so the trend as disclosed is two points, not a curve.

The judge is the model being judged

The ratings were not produced by an independent panel. Anthropic's write-up concedes the central problem: the judge model "could make the same kinds of errors as the model it is checking". On exact agreement, the model and a human matched 59% of the time; two humans matched each other 35% of the time; and model and human ratings fell within one level of each other 97% of the time. The 35% human-to-human figure is the one Anthropic uses to argue that the grading task is genuinely hard; it is not offered as a validation of the judge.

Self-authored HTML card headed 'Method and reliability' with the title 'Claude graded the work, and Claude is also the caveat' and the subtitle 'How Anthropic built the R&D Automation Index, and what its own agreement checks found.' A left column headed 'The method' lists six numbered steps: sample 20% of staff in the relevant departments each week in July 2026; have a Claude research agent review work records including Slack messages and internal documents; produce about 15,000 tasks; freeze them into a 542-node tree with 378 leaf tasks; weight categories by human person-time; rate each category from AL0 to AL5 with a Claude judge. A right column headed 'The check' shows three large figures: 59%, exact agreement between the Claude judge and a human rater; 35%, exact agreement between two human raters on the same task; 97%, model and human ratings within one level of each other. A quotation box carries Anthropic's line that the judge model could make the same kinds of errors as the model it is checking. Three amber chips below list the further stated limits: borderline cases between AL3 and AL4 remain disputed, a frozen basket can miss work that did not exist when it was fixed, and no external audit exists yet because the company plans to embed third-party evaluators. The OrcaRouter logo sits in the bottom-right corner.

Two further limits are stated on the page rather than discovered by critics. Borderline cases between "collaborates" and "leads" remain disputed, which is precisely where the 26% lives. And a frozen task basket can miss work that did not exist when the basket was frozen, though Anthropic says a January 2026 comparison found no rise in novel tasks.

On outside scrutiny, the page describes a plan rather than a completed audit. "We plan to embed independent third-party evaluators from multiple organizations at Anthropic," the page says — an arrangement in the future tense, not a current audit. The one completed external check it cites is that METR has independently red-teamed its offline monitoring platform. In heise's 18 September report, the caveats are Anthropic's own; the German outlet quotes no outside expert challenging the index. Independent outlets have reproduced the numbers, not verified them, and that distinction is the most important thing to carry away from the coverage.

Thirty thousand agents, and the ledger that follows them

Alongside the automation index, Anthropic disclosed the scale of the agent population behind it: "approximately 30,000 agents doing research and engineering work at Anthropic at any one time" on its most-used internal platform. Read plainly, that is a concurrency count rather than a headcount of distinct agents over a period, and heise renders it more cautiously as agents "active simultaneously at times in August".

The controls Anthropic describes are designed to make that population auditable rather than anonymous. Each agent has "an individual identity", and all the data it creates is tied to that identity; the identity "is not tied to a model, it persists through model upgrades", so that "every action can be audited per agent across time" and "every data record can be linked back to a specific agent". Agents "communicate through a shared, open messaging system, rather than privately"; every message is tied to the identity of the agent that posted it, can link to other content so that original references are preserved, and is cross-referenced with agent transcripts so that monitoring can "traverse communication between agents".

The oversight numbers are the part that reads like an incident record, although the record classifies this disclosure as informational. Every action passes through the online monitor before execution, usually within seconds; every action is ingested offline after it is taken; blocked actions are reviewed by a human within a week. Of more than a billion decisions in August, 0.002% — "about 1 in 47,000" — were blocked. Offline flagging runs at roughly one to two transcripts per thousand, about 100,000 transcripts a week, with roughly 50 high-priority cases escalated to humans.

A block rate is not a safety rate. It measures how often the monitor stops an action, not how many actions were wrong, and Anthropic reports it next to the coverage figure — 100% of online actions checked before execution — which describes the monitor's reach. The same asymmetry applies to compute allocation: for the week of 13–20 July, "about 6% of compute that went to AI R&D was allocated toward safety", rising to "about 12% of compute that went to AI-driven AI R&D". Anthropic calls those "deliberately conservative estimates", drawn from a sample of about 14% of nearly 10,000 research training and evaluation runs weighted toward higher-compute runs, and notes that a compute share measures spending rather than effort.

What "recursive self-improvement" means here

Anthropic ties the index to recursive self-improvement, which its companion essay defines as "a model fully autonomously building its successor". The essay's position is unambiguous: "We are not there yet, and recursive self-improvement is not inevitable." It places a fully closed loop in a hypothetical stage labelled "Closing the loop" and dated "20XX?", where agents build and train models and future Claude systems improve Claude itself. Today, it says, humans still supply the goal; what has changed is that "they no longer need to supply the method".

The essay also explains why the index is meant to be read as a distance measure rather than a milestone. AL4 work — Claude completing most of a task from a high-level prompt while a human supervises — is the level the 26% refers to. AL5, full autonomy with no human in the loop, is the level Anthropic says it has not reached. The gap between the two is the subject.

What is established, and what is inferred

Established. Anthropic published these measurements on 17 September 2026, the date the record carries at day precision. The live index page shows no visible publication date of its own, and the companion essay carries an "Update 9/18/2026" stamp. The figures — 26%, under 1% in February, above 90% at AL3 or higher, about 30,000 agents, 0.002% of decisions blocked, 6% and 12% of compute — are Anthropic's own, stated on its pages and reproduced by heise and InfoQ within a day.

Established. No measured subset of AI R&D work reached AL5, no harm to any third party is reported, and the record's real-harm field is empty. The record is graded confidence A for documentation, is not flagged as disputed, and carries severity "info", which reflects that no harm was recorded rather than anything about how well the document is sourced.

Not established. That Claude "leads" 26% of Anthropic's R&D in any sense independent of Anthropic's own grading. The measurement is company-run, the judge was a Claude model, and the external evaluators the page describes are planned rather than in place. The 59% exact agreement between judge and human is the number to hold next to the headline.

Not established. That the trend continues, or that it points at recursive self-improvement. Anthropic declines to claim either, and a two-point comparison between February and August is not a forecast. Nor does the block rate establish that the agent population is under control; it establishes that a monitor saw every online action and stopped about one in 47,000.

Judgment, not measurement. That a rise from under 1% to 26% in six months is a fast pace. Nothing in the disclosure fixes a threshold at which the pace becomes a problem, and Anthropic's framing leaves that judgment to the third parties it says it will embed.

Self-authored HTML card headed 'Evidence check' with the title 'A company measuring itself, precisely' and the subtitle 'What the record supports, and what it leaves to Anthropic's own judgment.' A left panel headed 'Established' lists four points: published 17 September 2026, the date the record carries at day precision, with the live page showing no date of its own and the companion essay stamped 18 September; the figures 26%, under 1% in February, above 90% at AL3 or higher, about 30,000 agents, 0.002% of decisions blocked and 6% and 12% of compute are Anthropic's own and were reproduced by heise and InfoQ within a day; no measured subset reached AL5; no third-party harm reported, with the record's real-harm field empty and severity 'info'. A right panel headed 'Not established' lists four points: that Claude leads 26% in any sense independent of Anthropic's grading, since the measurement is company-run and the judge is a Claude model; that the trend continues or points to recursive self-improvement, which Anthropic explicitly declines to claim; that a 0.002% block rate means the agent population is under control; and any effect on staffing, research quality or safety outcomes, none of which the index measures. A blue-bordered band reads that confidence A grades how well the disclosure is documented and does not mean the measurement has been independently confirmed. A footer cites the Anthropic Institute pages, heise of 18 September 2026, InfoQ of 18 September 2026 and the archive record id. The OrcaRouter logo sits in the bottom-right corner.

Anthropic argues for regular publication of comparable figures and for independent verification, heise reports — which is also the clearest statement of what the index is not yet. Until outside evaluators are inside the loop, the 26% is a laboratory measuring itself with its own instrument: more specific than a press release, and still not an audit.