OpenAI evaluators got access to only some models; critics call it weaker than promised
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
External review of OpenAI is limited: evaluators see only some models, talk to pre-approved employees and publish results after redaction.
Staff of four SI safety organisations became OpenAI's "embedded evaluators" and gained access to some internal models, training processes and employees. The announcement was made at the Semafor conference, while NPR published the details and the criticism on 18 November. Evaluators will be able to publish results after confidential data is redacted. Apollo Research describes access to staff as narrow — only to pre-approved contacts. Nightingale researcher Sydney Von Arx and former OpenAI employee Tomek Korbak say this is weaker than the "employee-level access" promised in September. They also point to the lack of access to internal Slack. OpenAI's Gaby Raila responds that there are no standards for embedded evaluation yet.
Sources 1