Third party finds OpenAI vulnerability: AI agents can evade security audits

METR and Redwood Research completed a pilot security test of OpenAI's internal AI agent monitoring system and found some vulnerabilities that could allow a large number of an agent's operations to evade security audits. The test was conducted from late July to early August; each of the two organizations sent one researcher to OpenAI, using dedicated devices restricted to the office premises, and accessed internal code repositories and monitoring configuration documents for stress testing, ultimately producing a 61-page internal report. The test found vulnerabilities that could allow an agent to evade monitoring across hundreds of operations, and estimated that during the test period about 10% of Codex's internal business traffic was not covered by the monitoring system. OpenAI said it had fixed all vulnerabilities that would affect the online production environment, but the two organizations did not re-verify the fixed system; as a pilot project, the parties agreed to disclose only summary conclusions, and the full report will not be publicly released.