All news

METR estimates Anthropic's model development has roughly doubled in pace with help from SI

Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .

A rare internal assessment points to a feedback loop in which SI already helps build subsequent models faster, although the size of the effect remains highly uncertain.

From August through October, METR examined Anthropic's internal metrics, interviews and data on pretraining, reinforcement learning and employees' use of SI. On the AECI capability index, model scores have grown 1.9 times faster since October 2025 than at the previous rate; the estimate's 95% confidence interval ranges from 1.5 to 2.8. METR considers it likely that most of the current acceleration comes from using SI in research and development, but emphasizes substantial uncertainty: a plausible range for overall acceleration extends from roughly 1.3 to 3 times. The researchers found no convincing bottlenecks in data, algorithms or compute that would necessarily halt further acceleration over the next year, but do not consider that conclusion definitive. Anthropic had the right to redact confidential information and removed most of the evidence METR described as most informative. Outside readers must therefore rely more heavily on the authors' judgment rather than independently examining the full body of data.

Sources 1

History

External auditors at SI labs

9 events
  1. METR estimates Anthropic's model development has roughly doubled in pace with help from SIOpen now
  2. METR and Redwood Research spent three weeks testing OpenAI's internal agent monitoring system: by their estimate, 75–94% of traffic was eligible for monitoring; OpenAI says the vulnerabilities have been fixed
  3. METR reviews the section of Anthropic's report on misalignment risk: the risk is "low", but by its assessment because of the models' limited capabilities, not safeguards
  4. Anthropic gives six METR staff access to its internal systems close to that of its own employees; they will be able to publish findings without approval
  5. METR publishes five pieces on independently assessing labs' risk of losing control of SI
  6. Independent SI model evaluators METR, Apollo Research and Transluce are at the center of the safety debate, but how to fund them is unresolved
  7. OpenAI will let outside reviewers access models during training, but without the "desks and badges" Altman promised
  8. More than 100 researchers, including Hinton, name five conditions without which "external reviewers" inside SI labs are useless
  9. Anthropic lets outside auditors from Accenture inside the company, with the same access as employees