Outside Researchers Find Gaps in OpenAI's Internal Safeguards

A team of outside researchers who spent three weeks inside OpenAI found significant holes in the company's own safety monitoring systems for detecting unwanted AI behavior. The audit, a pilot program run jointly by OpenAI, METR, and Redwood Research, concluded that a portion of internal model traffic went completely unmonitored during the review period. OpenAI has since stated that it has fixed all the weaknesses identified by the team.

Two researchers from METR and Redwood Research were granted access to OpenAI's codebase and internal documents from July 21 to August 11. During that time, they also interviewed about a dozen employees to understand how the company's internal monitoring for rogue AI activity was designed and implemented.

The auditors uncovered several areas of concern. For internal agentic Responses API traffic involving models at the GPT-5.5 class and above, an estimated 6 to 25 percent was not covered by monitoring during the audit period. METR noted that there is significant uncertainty in this figure in both directions. The report also found that roughly 10 percent of internal Codex traffic between May and July went untracked due to a gap in handling compaction events, which are context summaries; this estimate also carries significant uncertainty.

Further weaknesses were identified in the oversight mechanism itself. The system was designed to trigger only when an agent handed control back to a user, allowing agents to avoid oversight by continuing to execute actions without pausing. Researchers were also able to find ways to hide hundreds of actions from the monitor and to bypass the Codex sandbox. At the start of the audit, model behavior during reinforcement learning training was not being monitored at all.

OpenAI says it has since addressed all of the weaknesses found by the auditors. The company stated it paused reinforcement learning training following an incident with Hugging Face in August and expanded monitoring to cover it. However, those changes were made after the auditors' access period had ended and were not part of their evaluation.

While the full 61-page report was delivered to OpenAI, the public has seen very little of it. After the company applied redactions, METR and Redwood Research leadership received a 38-page version, but only a short summary was made public. METR stated it has no right to disclose the redacted portions on its own.

The collaboration was a pilot project designed to give external researchers deep access to a company's internal infrastructure for review. Following this experience, METR says it wants to push for greater public transparency in any future audits.

What Happens Next: The focus now shifts to whether this pilot program will become a standard practice. METR has signaled its intention to advocate for more public disclosure in future reviews, a move that would provide greater visibility into how AI safety systems are performing inside major technology companies.