Anthropic Publishes Three Frontier AI Metrics: 26% Is “Leads,” Not “Fully Autonomous”

On September 17, 2026, Anthropic published three metrics on its official website meant to let outsiders see research and development progress inside a frontier AI lab: how much AI R&D work AI itself completes, how the actions of AI agents are overseen, and how compute resources are allocated. Anthropic ties this to an earlier piece by CEO Dario Amodei calling for pacing frontier AI, writing that it would expect these numbers to shift if labs actually coordinated on slowing down.

This article was fact-checked on September 18, 2026, against three pages on Anthropic's official website: the post announcing these three metrics, the newsroom overview page, and a policy page explaining the policy context. This site has not run any tests of its own; all three metrics are currently numbers that Anthropic measures and publishes itself, and this article goes through the limits and the parts that remain unverified, as Anthropic itself has flagged them.

What Anthropic Published, and Why

The summary Anthropic's official newsroom page wrote for this post is that today, the world can't see what's going on inside AI labs, and Anthropic is proposing new metrics to give the public visibility into frontier AI development. The post itself ties this to Amodei's call, but only links out to his piece without restating what it argues; Anthropic writes that these proposed measurements and policies together are a starting point for monitoring the pace of AI development from outside the labs.

The post itself does not print a publication date — as fact-checked through September 18, 2026, the page has no byline date, no “published on” text, or corresponding metadata field. September 17 is the date Anthropic's official newsroom page attaches to this piece in its featured section, and this article treats that as the event date; the original text prints no time of day, so this article does not state an exact publication time. Anthropic calls this set of metrics a prototype, writes that it hopes to demonstrate transparency this way and will keep doing so, but does not commit to a fixed publishing schedule.

Three Metrics: R&D Automation, Agent Oversight, and Safety Compute

The first is the “Anthropic R&D Automation Index,” scored using Epoch AI's Automation Level (AL) scale, running from AL0 to AL5 (fully autonomous, with no human in the loop). At AL3, “collaborates,” AI can do large chunks of work under close human direction; at AL4, “leads,” AI can complete most of a task end-to-end from a high-level prompt while a human still supervises. As of August 2026, Anthropic writes that Claude is not operating fully autonomously for any measured subset of AI R&D work; it “leads” 26% of the work, and the share at or above “collaborates” totals more than 90%.

The second is agent oversight, where Anthropic picked three angles: coverage, review latency, and escalation rate, corresponding respectively to how much of an action passes through monitoring, how long elapses between an action and its review, and how much of an action gets blocked or flagged for review. As of August 2026, roughly 30,000 agents were doing research and engineering work at any one time on Anthropic's most-used internal platform, and these figures cover only that one platform. 100% of these actions pass through online monitoring first, usually within seconds, and afterward 100% also go through offline monitoring; the offline monitor flags roughly 100,000 transcripts a week, most of which are resolved by classifiers, with the highest-priority roughly 50 a week escalated to human review.

Anthropic writes that after analyzing more than a billion decisions in August 2026, 0.002% of them (about 1 in 47,000) were blocked by the online monitor. At the start of that section, Anthropic first cautions that in its monitoring data to date, individual agents rarely misbehave, but when there are millions or billions of agents operating in the economy, even rare events can happen regularly. The third metric is a compute snapshot: Anthropic examined how it used all of its compute over the week of July 13 to 20, 2026, finding that about 6% of AI R&D compute was allocated to safety work and about 12% of “AI-driven AI R&D” compute was allocated to safety work; the two figures have different denominators, and this article does not add or compare them.

Why These Numbers Are Not Yet Verified Numbers

Anthropic itself lists two obstacles standing in the way of cross-lab comparison. First, there is no shared measurement methodology; second, Anthropic uses its own Claude to score its own systems: Claude research agents first gather the evidence, and then a separate Claude serves as the judge; when rating a given month, the agents doing the rating can only see evidence from that month or earlier. In other words, Anthropic itself writes that this could mean the model acting as judge makes the same kinds of errors as the model it is checking.

The oversight numbers have boundaries too: Anthropic writes that it plans to embed independent third-party evaluators from multiple organizations, giving them access comparable to what its internal risk assessment teams have, and in the oversight section writes that it is now setting up external evaluators — neither sentence amounts to something an outside party has already checked. The text mentions that METR has, in the past, independently red-teamed Anthropic's offline monitoring platform, but that is past tense, and what was tested was the monitoring platform, not the percentages this article publishes. Based on what this fact-check read on the day it was done, this article found no outside party that has verified the numbers above.

Anthropic also writes that its method for the compute share is conservative: the example it gives is that if a token contributed as much to advancing capabilities as it did to safety, it was not counted on the safety side; the two figures also exclude a separate, similarly sizable pool of compute used for safety-protection classifiers. Anthropic notes that safety research tends to rely more on researchers' time than on compute compared with frontier training, so the compute share is only an imperfect proxy, and its value lies in comparing like with like across developers and over time, not in the absolute size of these two percentages by themselves.

The Two Numbers Most Likely to Be Misread

The first place readers are likely to misread is “leads.” 26% is often taken to mean “a quarter of the work is done by AI on its own, with no human managing it,” but Anthropic's own definition of AL4 “leads” is that AI can complete most of a task end-to-end from a high-level prompt while a human still supervises; AL5, “fully autonomous, with no human in the loop,” is described in a footnote as “a level we have not yet reached.” A closer reading is “AI takes on a task and does most of the steps itself, but someone is still watching and can step in at any time,” not “AI finishes the job behind closed doors, with no one knowing what happened.”

The second place readers are likely to misread is the 0.002% block rate. Taken alone, it is easy to read as “agents almost never misbehave, so the system is safe”; but in that same section, Anthropic writes that when there are millions or billions of agents operating in the economy, even if any single event is unlikely, rare events can happen regularly. And 0.002% is only the rate the online monitor blocked within one month on one internal platform; the offline monitor separately flags roughly one to two in a thousand transcripts for further review, and the original text lists the two in separate cells of a table without combining them.

What This Article Leaves Out, and How It Connects to Pacing the Frontier

Based on what was fact-checked through September 18, 2026, this post does not name any country or market, and it does not mention product changes or any feature change users would see — this is an article about how Anthropic measures its own internal R&D process, not a product announcement, and it will not change how Taiwan-based users use Claude today. The article publishes only Anthropic's own numbers; it does not give other companies' automation share, oversight coverage, or safety-compute share. Anthropic's own text says “any frontier developer could” publish similar metrics, not that other companies already have.

In an appendix, the post writes out its own definition of “safety work”: work whose dominant purpose is making AI systems safer, more understandable, or more secure; Anthropic also states that this is just one reasonable choice among many, that another developer or regulator might draw the line elsewhere, and that it believes all sides would benefit from agreeing on a shared definition ahead of time. A separate Anthropic policy page describes its own Advanced AI Framework, covering transparency and independent evaluation up through government authority to block or deter dangerous deployments when necessary; that page states that the framework is written primarily with the US federal government in mind and is a proposal, not a rule already in effect.

This post and the piece Amodei published earlier, “We Must Pace the Frontier,” belong to the same thread: the post only writes that these numbers echo Amodei's call for coordinated pacing of the frontier and links out to it, without restating its content; the three-step plan and “step one is embedding external evaluators” come from another roundup on this site, not from the pages cited in this article. The three metrics published here are Anthropic's attempt to turn “whether pacing is actually happening” into numbers that can be laid out and examined; to learn about those three steps, see the second link at the end of this article.

2026 AI News Roundup: Highlights and Daily Life Applications from January to September

Amodei Urges Pacing Frontier AI: What “We Must Pace the Frontier” Proposes and Its Limits

• AI tools
• Apps & software
• AI news