All news

OpenAI evaluators got access to only some models; critics call it weaker than promised

Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .

External review of OpenAI is limited: evaluators see only some models, talk to pre-approved employees and publish results after redaction.

Staff of four SI safety organisations became OpenAI's "embedded evaluators" and gained access to some internal models, training processes and employees. The announcement was made at the Semafor conference, while NPR published the details and the criticism on 18 November. Evaluators will be able to publish results after confidential data is redacted. Apollo Research describes access to staff as narrow — only to pre-approved contacts. Nightingale researcher Sydney Von Arx and former OpenAI employee Tomek Korbak say this is weaker than the "employee-level access" promised in September. They also point to the lack of access to internal Slack. OpenAI's Gaby Raila responds that there are no standards for embedded evaluation yet.

Sources 1

History

External auditors at SI labs

10 events
  1. OpenAI evaluators got access to only some models; critics call it weaker than promisedOpen now
  2. METR estimates Anthropic's model development has roughly doubled in pace with help from SI
  3. METR and Redwood Research spent three weeks testing OpenAI's internal agent monitoring system: by their estimate, 75–94% of traffic was eligible for monitoring; OpenAI says the vulnerabilities have been fixed
  4. METR reviews the section of Anthropic's report on misalignment risk: the risk is "low", but by its assessment because of the models' limited capabilities, not safeguards
  5. Anthropic gives six METR staff access to its internal systems close to that of its own employees; they will be able to publish findings without approval
  6. METR publishes five pieces on independently assessing labs' risk of losing control of SI
  7. Independent SI model evaluators METR, Apollo Research and Transluce are at the center of the safety debate, but how to fund them is unresolved
  8. OpenAI will let outside reviewers access models during training, but without the "desks and badges" Altman promised
  9. More than 100 researchers, including Hinton, name five conditions without which "external reviewers" inside SI labs are useless
  10. Anthropic lets outside auditors from Accenture inside the company, with the same access as employees