All news

More than 100 researchers, including Hinton, name five conditions without which "external reviewers" inside SI labs are useless

Prepared by Sintelligent with the help of SI. Publication time restored from the build log. . Updated: .

External oversight of SI is turning from a promise into a profession with its own rules, and the labs will most likely get not bans but permanent reviewers inside.

The letter from the AI Evaluator Forum consortium is a response to Amodei's and Altman's promises to let external evaluators in with access "like an employee's." The conditions: reviewers do not belong to the labs and are not financially dependent on them, and decide for themselves what to publish; each lab is assessed by several teams across different risks; methods and findings are open; there is legal protection against retaliation for inconvenient results; and real access to computers, data, unreleased models and conversations with engineers. Signatories include Geoffrey Hinton, Arvind Narayanan of Princeton, and researchers from Johns Hopkins, Stanford and METR; the authors acknowledge that no current arrangement meets these conditions.

Sources 3

cnbc.comaievaluatorforum.orgtechtimes.com
History

External auditors at SI labs

8 events
  1. METR and Redwood Research spent three weeks testing OpenAI's internal agent monitoring system: by their estimate, 75–94% of traffic was eligible for monitoring; OpenAI says the vulnerabilities have been fixed
  2. METR reviews the section of Anthropic's report on misalignment risk: the risk is "low", but by its assessment because of the models' limited capabilities, not safeguards
  3. Anthropic gives six METR staff access to its internal systems close to that of its own employees; they will be able to publish findings without approval
  4. METR publishes five pieces on independently assessing labs' risk of losing control of SI
  5. Independent SI model evaluators METR, Apollo Research and Transluce are at the center of the safety debate, but how to fund them is unresolved
  6. OpenAI will let outside reviewers access models during training, but without the "desks and badges" Altman promised
  7. More than 100 researchers, including Hinton, name five conditions without which "external reviewers" inside SI labs are uselessOpen now
  8. Anthropic lets outside auditors from Accenture inside the company, with the same access as employees
Follow on Telegram