Anthropic Measured Agent Autonomy — and Argued Against Approval Mandates

On 18 February 2026, Anthropic's Societal Impacts team published "Measuring AI agent autonomy in practice," an analysis of 998,481 tool calls made through its public API and 500,000 interactive Claude Code sessions, and ended it with a recommendation aimed at legislators: oversight rules should not prescribe particular interaction patterns, including requiring a human to approve every action an agent takes. The post is a research report, not a filing, but it is logged in the incident archive as a policy record — severity info, confidence A, no recorded harm, not disputed — because a model vendor used its own first-party telemetry to argue that one common regulatory design "will create friction without necessarily producing safety benefits." The full record, with its primary source and classification, sits in the Orca AI Incident Archive, alongside the other governance entries in the series.

The report carries twenty named contributors, among them Miles McCain, Thomas Millar, Saffron Huang, Alex Tamkin, Jack Clark and Deep Ganguli. It draws on Clio, Anthropic's privacy-preserving analysis tool, which the post credits with letting the company study real customer traffic without reading the underlying conversations. That methodology is the substance of the piece: every number below is a measurement of what people did with agents, not of what a benchmark says agents can do.

What Anthropic measured, and how

Anthropic defines an agent as "an AI system equipped with tools that allow it to take actions" — running code, calling external APIs, messaging other agents. The definition sets the unit of analysis. Rather than trying to reconstruct each customer's agent architecture, which the company says it cannot see, it counted individual tool calls and the context around them.

The two data sources trade breadth for depth. Public API traffic gives Anthropic visibility across thousands of customers, but only at the level of isolated tool calls; the company says it has "no reliable way to associate independent requests to our API into 'sessions' of agentic activity." Claude Code gives complete sessions — what triggered a stop, how long Claude worked, when a human cut in — but it is a single product used overwhelmingly for software engineering. Autonomy findings come mostly from Claude Code; risk and domain findings come from the API sample.

Turn duration, the elapsed time between Claude starting work and stopping, is the paper's proxy for autonomy. Anthropic is explicit that the proxy is imperfect: faster models and parallel subagents shorten turns, more ambitious tasks lengthen them, and the user base roughly doubled between January and mid-February 2026, changing who is in the sample.

The numbers the report puts on the record

These are the figures Anthropic publishes, stated as the report states them.

• The 99.9th-percentile Claude Code turn nearly doubled, from under 25 minutes in late September 2025 to over 45 minutes in early January 2026.

• The median turn stayed near 45 seconds, fluctuating between 40 and 55 seconds, and nearly every percentile below the 99th was stable.

• Inside Anthropic, Claude Code's success rate on internal users' hardest tasks doubled between August and December 2025 while average human interventions per session fell from 5.4 to 3.3.

• New users with fewer than 50 sessions used full auto-approve in roughly 20% of sessions; by 750 sessions that rose to over 40%.

• Interruptions moved the other way: users with around 10 sessions interrupted Claude in 5% of turns, experienced users in around 9%.

• Claude Code asked for clarification more than twice as often on the most complex tasks as on minimal-complexity ones, and on those hardest tasks it stopped to ask more than twice as often as humans interrupted it.

• On the API sample, 80% of tool calls came from agents that appeared to have at least one safeguard, 73% appeared to have a human in the loop, and 0.8% appeared irreversible, such as sending a customer an email.

• Software engineering accounted for nearly 50% of API tool calls; no other domain exceeded a few percentage points.

The finding Anthropic calls a deployment overhang

The rise in the tail is smooth across model releases. "If autonomy were purely a function of model capability, we would expect sharp jumps with each new launch," the post argues, so the steadiness points to trust accumulating with experience, harder tasks being attempted and the product improving. Its conclusion is that existing models "are capable of more autonomy than they exercise in practice" — the gap the report calls a deployment overhang.

Anthropic sets that against external capability testing, citing METR's estimate that Claude Opus 4.5 can complete, at a 50% success rate, tasks that would take a human nearly five hours, while its own 99.9th-percentile Claude Code turn is about 42 minutes. It then withdraws the comparison: METR measures task difficulty in an idealised setting with no human in the loop, while turn duration measures elapsed time in a deployment where Claude pauses and users interrupt. A footnote is blunt — "we do not attempt to reason across these metrics." The juxtaposition is illustrative, not a measured multiplier, and should not be quoted as one.

The oversight paradox at the centre of the argument

Experienced users approve less and interrupt more. Anthropic reads that pair as a change in oversight strategy rather than an abdication of it: newcomers approve each step and therefore rarely need to stop Claude mid-run, while experienced users let Claude work and step in when something goes wrong. The API data points the same way — 87% of tool calls on minimal-complexity tasks had some form of human involvement, against 67% on high-complexity tasks.

The report is candid that other explanations survive. Its own footnote notes that experienced users may simply be attempting harder tasks, that Claude Code's default settings push new users towards approval-based oversight, and that software is unusually amenable to supervision because output can be tested and reviewed; in domains where checking an agent's work requires the same expertise as producing it, the shift "may be slower or take a different form."

The policy argument, and what it asks for

Anthropic's recommendations fall into three groups. Model and product developers should invest in post-deployment monitoring, because pre-deployment evaluations cannot see how people actually interact with agents. Model developers should train models to recognise their own uncertainty and surface it, a property the report says "complements external safeguards like human approval flows and access restrictions." Product developers should design for oversight — visibility into what the agent is doing plus simple ways to redirect it.

The section addressed to policymakers is the one that makes this a governance record. Under the heading "It's too early to mandate specific interaction patterns," Anthropic writes that "oversight requirements that prescribe specific interaction patterns, such as requiring humans to approve every action, will create friction without necessarily producing safety benefits," and that the focus should be "on whether humans are in a position to effectively monitor and intervene, rather than on requiring particular forms of involvement."

Trade coverage picked up exactly that framing. Inside AI Policy's report on 20 February 2026 was headlined around Anthropic urging caution on oversight mandates; Inside Cybersecurity followed on 24 February with the same angle. Neither could be read in full without a subscription, and neither examined the underlying data.

What is corroborated, and what is not

The publication and its headline figures are corroborated. GIGAZINE reported the study on 19 February 2026, the day after publication, repeating the 25-to-45-minute tail, the 20%-to-40% auto-approve shift, the 5.4-to-3.3 intervention decline and the 0.8% irreversible share. Digital Watch Observatory summarised it on 20 February 2026, as did Inside AI Policy. The corroboration is of what Anthropic said, not of whether it is true: no outside party has audited the 998,481 classified tool calls, and Anthropic does not release the dataset.

That asymmetry is worth naming without overstating it. A vendor measuring its own product, and concluding that the compliance burden should fall on monitoring rather than on per-action approval, has reached a conclusion convenient to the product it sells. Anthropic states most of the structural limits itself, in a limitations section that is unusually direct. But reproducibility is not among the things it can fix while the raw data stays private, and no independent body has checked the classifications.

The limits Anthropic states itself

• The analysis covers one model provider. Agents built on other models may show different adoption patterns and risk profiles.

• The risk, autonomy and human-involvement classifications were generated by Claude. Anthropic says privacy constraints prevented manual inspection of the underlying data, and that its human-involvement estimate should be read as an upper bound.

• Sampling at the level of individual tool calls overrepresents deployments with many sequential calls, such as software engineering workflows with repeated file edits.

• The classifications cannot distinguish production actions from evaluations or red-teaming. Several of the highest-risk clusters, the report says, "appear to be security evaluations," and it believes many of the apparent data exfiltrations are tests.

• Human review downstream of the API is invisible, so an agent that looks autonomous at the tool-call level may have a person checking its output.

• The window is late 2025 through early 2026, and the strongest findings come from Claude Code, which is overwhelmingly a software engineering product.

What changed afterwards

The archive records no harm, no fix and no dispute, and that is the accurate description: this is a measurement and an argument, not an incident with a remediation. Anthropic says it will extend the analysis over time and calls for cross-industry work on linking API requests into sessions in a privacy-preserving way. The post also notes that products like Cowork are making agents more accessible, which it expects to push the frontier of risk and autonomy outward. In the coverage reviewed for this article, no regulator had adopted, rejected or formally responded to the recommendation against per-action approval.

The durable point is narrower than the headline numbers. Anthropic's case against mandating step-by-step approval rests on a measurement that, by its own admission, only a model provider with first-party session data can currently make — and even Anthropic says it cannot link third-party API traffic into sessions. Until that infrastructure exists, the policy advice rests on one company's instrument, applied to one company's customers, over one winter. That does not make the finding wrong. It makes it a vendor's measurement, and it should be cited as one.