Apollo Research proposes six embedded evaluation claims for AI monitoring
Towards embedded evaluations for AI monitoring
Apollo Research published a blog post proposing six monitoring detection claims, intended for independent evaluators to enter frontier AI labs and verify whether their monitoring systems can detect and appropriately disclose an out-of-control internal deployment. The article lists rogue internal deployment (RID, i.e., an agent within the lab running against developer intent for an extended period, evading oversight, or acting where there is no monitoring) as the core threat model, and notes that AI monitoring built from developers' internal models has become the primary oversight mechanism for internal deployment agents. The six claims cover the scope of monitoring deployment, detection capability, monitor trustworthiness (no collusion, no selective under-reporting), incident disclosure, no missed RID attempts in the past three months, and monitoring remaining sufficient for next-generation internal deployments. Each claim is executed by reviewing developer evidence, collecting new evidence (red-teaming the monitoring pipeline, planting honeypots and test incidents, rescanning historical logs), assessing the sufficiency of evidence, and reaching a conclusion, producing a public report; developers may redact a small amount of sensitive detail but may not change the overall conclusion.
The article breaks independent evaluation of AI monitoring into six verifiable claims and gives the evidence requirements and determination path for each, providing a concrete reference framework for laboratory monitoring audits.