All news

Independent SI model evaluators METR, Apollo Research and Transluce are at the center of the safety debate, but how to fund them is unresolved

Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .

Powerful models are being checked by small independent groups, but who will pay them and what they will be able to see is still unclear.

According to CNBC, Anthropic and OpenAI are turning to small third-party groups — METR, Apollo Research and Transluce — after Dario Amodei pledged last month to embed independent evaluators in his company. It is unresolved who will fund them, what access they will have and how reporting will be structured. METR, by its August announcement, raised commitments of around $71 million over six months, against $13.6 million in all of 2024; it has fewer than 50 staff. Vals AI CEO Rayan Krishnan told CNBC his company has grown from eight to about 30 employees in a year. OpenAI said it is "finalizing contracts" with third-party evaluators and will announce details in the coming weeks; Brown University professor Suresh Venkatasubramanian told CNBC: "the problem, as always, is money."

Sources 1

cnbc.com
History

External auditors at SI labs

8 events
  1. METR and Redwood Research spent three weeks testing OpenAI's internal agent monitoring system: by their estimate, 75–94% of traffic was eligible for monitoring; OpenAI says the vulnerabilities have been fixed
  2. METR reviews the section of Anthropic's report on misalignment risk: the risk is "low", but by its assessment because of the models' limited capabilities, not safeguards
  3. Anthropic gives six METR staff access to its internal systems close to that of its own employees; they will be able to publish findings without approval
  4. METR publishes five pieces on independently assessing labs' risk of losing control of SI
  5. Independent SI model evaluators METR, Apollo Research and Transluce are at the center of the safety debate, but how to fund them is unresolvedOpen now
  6. OpenAI will let outside reviewers access models during training, but without the "desks and badges" Altman promised
  7. More than 100 researchers, including Hinton, name five conditions without which "external reviewers" inside SI labs are useless
  8. Anthropic lets outside auditors from Accenture inside the company, with the same access as employees
Follow on Telegram