Anthropic is enhancing its AI fashions about twice as quick because it did a yr earlier, in accordance with an impartial assessment. The catch: the corporate redacted a lot of the proof the reviewers mentioned mattered most, so the general public has to take their phrase for a discovering that would reshape how dangerous the tempo of AI progress seems.
The evaluation got here from Model Evaluation and Threat Research (METR), revealed on Thursday. METR spent 11 weeks inside Anthropic from mid-August to late October, with entry to inside benchmark scores, coaching metrics, employees interviews and surveys. It took no fee from the lab.
The quantity
On Anthropic’s inside functionality index (AECI), mannequin scores have been rising at an estimated 1.9 occasions the pre-October 2025 price, with a 95% confidence interval of 1.5–2.8x. In plain phrases, METR says Anthropic has made roughly two years of historic progress prior to now yr. Most METR employees on the challenge now imagine the lab is “likely” enhancing on well-specified duties at round twice the March 2024–October 2025 tempo, although they’d not be shocked if the true determine sat nearer to 1.3x or 3x.
METR says it might probably “confidently” rule out sooner progress in coaching compute because the trigger. Interviews and surveys level as an alternative to heavy use of AI instruments to assist researchers write code, run experiments and check out extra concepts. Employees Slack polls of self-reported speedup reached about 12x for the newest inside mannequin in October — free proof, METR stresses, however per a big elevate. AI is just not but making extremely vital, totally autonomous contributions to the toughest coaching steps, the reviewers have been advised, although “signs of life” for which can be showing.
What you can’t see
Anthropic had unilateral rights to redact private info from the general public report. METR says the corporate eliminated a majority of the proof the group discovered most informative for its threat conclusions, together with detailed acceleration estimates by pipeline stage, bottleneck evaluation and forecasts for future fashions. Anthropic mentioned the cuts have been meant to strip confidential info that would assist rival builders.
METR is blunt in regards to the end result: the general public can not decide the proof itself or reconcile it with different sources. “This report does not achieve ‘transparency about evidence’,” it wrote. Readers should lean on METR’s judgement greater than the group thinks is good for questions of this weight. AP News famous the identical pressure: a warning about sooner progress, tied to materials most individuals can not examine.
Why the tempo declare is contested
The reviewers listing methods they may very well be incorrect in both route. Acceleration is perhaps restricted to scrub, auto-gradable duties and overstate messy real-world influence. Anthropic’s earlier AECI progress was slower than the broader trade frontier, which might inflate the “2x” comparability. On the opposite facet, redacted inside metrics hinted that acceleration may very well be above 2x and nonetheless rising, and METR noticed no clear bottlenecks that may block additional speed-ups over the following yr.
That issues as a result of recursive self-improvement — AI programs that meaningfully assist construct the following, sooner era — is the state of affairs security researchers fear might compress timelines. METR’s guess is {that a} majority (maybe round 80%) of the present acceleration is because of AI help, with extra uncertainty about late-2025 drivers. Co-founder Sam McCandlish mentioned Anthropic plans its personal public evaluation of how briskly progress is shifting inside the corporate, The Verge reported.
Why it issues
If frontier labs are already getting a big productiveness elevate from their very own fashions, exterior watchdogs want proof the general public can verify — not solely headline multiples. A doubling of the progress price modifications how a lot time societies have to check, regulate and adapt. Redacting the underlying measurements turns an impartial evaluation into one thing nearer to a trusted middleman’s opinion, which is helpful, however not the identical as open verification. For employees already seeing AI reshape day-to-day output — including longer days rather than shorter ones — the query is now not whether or not the instruments assist. It’s how briskly that assist compounds contained in the labs that construct them, and who will get to see the numbers.

