All news

METR publishes five pieces on independently assessing labs' risk of losing control of SI

Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .

Independent evaluators propose how the public could learn about the risk of losing control of SI, but companies have not adopted anything yet.

METR, which collects evidence on whether people could lose control of SI agents at large scale, described how it works with SI companies: the goal is to give the public accurate, timely and independent data; the main method is to collect and publish evidence, as well as to comment on unpublished sensitive information and review risk reports. At the same time METR posted proposals for assessments: how to examine control and monitoring of internal SI agents, the alignment of unreleased models, and signs of acceleration from automated SI research, as well as how to inform the public about loss-of-control risks. These are the organization's proposals, not decisions adopted by companies.

History

External auditors at SI labs

8 events
  1. METR and Redwood Research spent three weeks testing OpenAI's internal agent monitoring system: by their estimate, 75–94% of traffic was eligible for monitoring; OpenAI says the vulnerabilities have been fixed
  2. METR reviews the section of Anthropic's report on misalignment risk: the risk is "low", but by its assessment because of the models' limited capabilities, not safeguards
  3. Anthropic gives six METR staff access to its internal systems close to that of its own employees; they will be able to publish findings without approval
  4. METR publishes five pieces on independently assessing labs' risk of losing control of SIOpen now
  5. Independent SI model evaluators METR, Apollo Research and Transluce are at the center of the safety debate, but how to fund them is unresolved
  6. OpenAI will let outside reviewers access models during training, but without the "desks and badges" Altman promised
  7. More than 100 researchers, including Hinton, name five conditions without which "external reviewers" inside SI labs are useless
  8. Anthropic lets outside auditors from Accenture inside the company, with the same access as employees
Follow on Telegram