Skip to content
IAPS· Theo Bearman·Indexed by this site · Originally publishedSelectedAttention39

IAPS report: transparency disclosures for chain-of-thought monitorability

Transparency Disclosures for Chain-of-Thought Monitorability

AI summary

IAPS released a report arguing that frontier AI developers should regularly disclose a set of chain-of-thought (CoT) monitorability metrics so that independent evaluators can track their trajectories of change. The report selected five metrics from 21 candidates: third-party chain-of-thought access and infrastructure specifications, naturalness, faithfulness, detection (the proportion of harmful behavior that escapes monitoring systems), and Uplift (the performance gain that chain-of-thought access provides to monitoring). An audit of OpenAI, Anthropic, and Google DeepMind showed that none reported all the metrics, but each metric had been reported by at least one, so complete cross-institution reporting is feasible. The report cites recent disclosures from the labs saying that monitorability is declining, for example the GPT-6 Astra system card released in September 2026 shows that its chain-of-thought monitorability dropped substantially compared with the previous generation. The report recommends incorporating these metrics into existing System Card and safety evaluation practices, and conducting deeper audits by embedded evaluators or independent verification organizations; the first version of the report was a preprint from July 2026 and has been accepted by the ICML 2026 Technical AI Governance Workshop.

Why it was selected

The report proposes five chain-of-thought monitorability disclosure metrics and audits the current state of three frontier labs, providing a verifiable framework for independently tracking the weakening of monitorability.

Related papers
Read originaliaps.ai