Anthropic’s AI models improved faster on internal tests, METR finds
A third-party assessment estimated nearly twice the earlier pace of benchmark progress at Anthropic. How much AI assistance contributed remains uncertain.
Anthropic’s AI models have been improving on a set of internal benchmarks at nearly twice their previous pace, according to a November 5 assessment by METR, a nonprofit evaluator. The researchers found evidence that AI tools were helping Anthropic staff work faster, but they were less certain how much of the acceleration the tools explained.
What nearly twice as fast measures
METR analyzed results on Anthropic’s version of the Epoch Capabilities Index, a collection of benchmarks that tracks how well models do on defined tasks. The evaluator estimated that scores had been improving at 1.9 times their earlier rate since October 2025, with a 95% confidence interval of 1.5 to 2.8 times in its main analysis. Different choices about which benchmarks and models to include produced a wider range of estimates, the assessment says.
The figure measures progress speed on the selected tests. Questions about research turnaround, staff productivity and performance at other companies would need different measurements. METR also noted that the benchmark data are sparse and may lag changes in the research process.
How much credit goes to AI tools
Staff interviews and surveys pointed to AI assistance as a contributor. METR’s team judged that AI likely accounted for much of the recent acceleration, with substantial uncertainty about the size of its contribution. Staff told the evaluators that AI was mainly helping people carry out the work. According to the published findings, fully autonomous systems made limited contributions to the most important parts of model training at the time of the review.
Anthropic gave METR access to internal data and staff, as the company described in an October update on outside evaluations. Anthropic also exercised its right to redact parts of the report. That leaves readers unable to examine some of the evidence that informed METR’s conclusions.
A follow-up could test whether the speedup persists with new models and whether other labs see it under comparable measurements. Both would help clarify how much the result tells us about AI research beyond Anthropic.
Trust & transparency
How we handled this story
The benchmark estimate, its uncertainty, and the disclosure limits were checked against METR’s original assessment and Anthropic’s access statement.
Grounded in this reporting
Ask Newsmerica
Ask about this article. Answers quote the closest published passages and link back to their sources.
This tool retrieves article passages; it does not add outside facts.Keep exploring






