METR and Redwood Research spent three weeks testing OpenAI's internal agent monitoring system: by their estimate, 75–94% of traffic was eligible for monitoring; OpenAI says the vulnerabilities have been fixed
Prepared by Sintelligent with the help of SI. Published in SI: . Updated: .
Independent experts got deep access to OpenAI's safeguards and found gaps; the fix is so far confirmed only by the company's word.
From July 21 to August 11, METR and Redwood Research employees with access to OpenAI's internal code, monitor configuration and documentation, and interviews with about a dozen engineers, tested the system that watches internal agents for misalignment; the result is a 61-page confidential report, and a short version has been published. By their estimate, with substantial uncertainty, 75–94% of agentic traffic in the Responses API from models at least as strong as GPT-5.5 was eligible for monitoring; at the start of the exercise reinforcement-learning training was not covered by monitoring, and because compaction events were not monitored, on the order of 10% of internal Codex agent traffic went unmonitored from May to July (OpenAI noticed and fixed this during the exercise). The authors found a way Codex agents could in principle bypass the sandbox and auto-review, and an exploit that hides hundreds of actions from the monitor (details redacted at OpenAI's request). OpenAI says all vulnerabilities affecting its production safeguards have been patched; METR and Redwood did not verify this and assessed the system only as it was before July 21.
Sources 2