Transparency Disclosures for Chain-of-Thought Monitorability

Core problem: CoT monitoring, a supervision tool, is weakening, but the outside world cannot track its changes; the authors argue that developers should regularly disclose a set of monitorability indicators.

• The value of and threats to CoT monitoring: The authors consider CoT monitoring one of the most effective supervision tools currently available, but CoT may become unfaithful, encoded, unreadable, or models may gradually stop relying on CoT reasoning; OpenAI disclosed that GPT-6 Astra shows a "substantial decline in chain-of-thought monitorability" compared with earlier models, and Anthropic and Google DeepMind also reported monitorability challenges in their respective new models (Executive Summary).
• Shortcomings of existing disclosures: The authors point out that current disclosures are voluntary and mainly based on developers' own private evaluations and configurations, making them hard for third parties to reproduce; and that standardized methods are lacking, making cross-developer comparison difficult (Section 1).
• Core claim: Disclosure indicators can make declines in monitorability visible before failures occur, and can incentivize maintaining monitorability through reputational pressure and by mitigating coordination problems among developers; the authors also note that GPT-6 Astra was still released after disclosing a decline in monitorability, indicating that reputational pressure alone may be insufficient to maintain monitorability (Section 1).
• Outputs of the paper: A five-condition chain framework, one disclosure indicator per condition (five in total), an audit of existing disclosures by Anthropic, OpenAI, and Google DeepMind, and a recommendation for repeated audits by embedded evaluators or an Independent Verification Organization.

The value and policy of CoT monitorability

• Korbak et al. (2025) — argue that CoT monitorability is a "new, fragile opportunity" for AI safety, one of the starting points of this paper's framework.
• Delaney et al. (2025) — discuss policy options for maintaining CoT monitorability; the Frontier Model Forum (2026) published a related issue brief.

Literature on CoT failure modes (the Threat Models table in the appendix maps each item to the five conditions)

• Latent reasoning: Hao et al. (2026), Geiping et al. (2025), Lu et al. (2025), Wang et al. (2026) — architecture research on reasoning in continuous latent space that bypasses textual CoT.
• Readability drift: Schoen et al. (2025) — observe that frontier reasoning models deviate from standard English and use tokens in ways unnatural for humans.
• Faithfulness: Turpin et al. (2023, post-hoc rationalization), Pfau et al. (2024, CoT as scaffolding), Roger & Greenblatt (2023) and Skaf et al. (2025, steganography), Baker et al. (2025, obfuscation under RL), Liu et al. (2026, pathological CoT).

Monitoring and evaluation

• Monitor failure and collusion: Greenblatt et al. (2024, AI control), Gardner-Challis et al. (2026, untrusted monitoring), Meinke et al. (2025, in-context scheming).
• Evaluation benchmarks: SHADE-Arena (Kutasov et al., 2025), LinuxArena (Tracy et al., 2026), METR's SHUSHCAST framework (Kinniment et al., 2026); the authors note that these show measurement is feasible, but that no standardized benchmarks and configurations exist across models and developers.
• Faithfulness measurement methods: Lanham et al. (2023, paraphrasing tests), Meek et al. (2025, perturbation/necessity tests), Mayne et al. (2026, external evaluation of 18 frontier models).

Concurrent disclosure proposals

• Greenblatt et al. (2026) — argue for reporting externally verified "opaque serial depth" and the impact of architecture on monitorability; Shah and Dragan (2026) — argue for maintaining transparent architectures and reviewing training practices. The authors state that this paper's framework is complementary to these proposals, and that the indicators can be directly incorporated into existing system card practices and are easy to standardize.

Baselines and benchmarks: This paper is a position/governance paper and has no experimental baseline; the audit compares against the latest system cards and public research of the three developers as of mid-October 2026 (Figure 3).

Overall approach: break CoT monitoring into a chain of five conditions that must all hold, select for each condition one metric that is "informative and feasible to disclose," and specify only what information must be disclosed, not the specific evaluation methods.

Five-Condition Framework (Section 2, Figure 1)

• Condition 1: CoT exists and is necessary for complex reasoning—if a model shifts to a latent reasoning architecture, the rest of the chain fails; the most direct external signal is the disappearance of inspectable CoT from the API.
• Condition 2: CoT is readable human language—CoT may drift into compressed text that the model uses for itself and that humans find hard to parse.
• Condition 3: CoT is faithful—the authors list three failure modes: post-hoc rationalization, CoT as scaffolding, and steganography, and say no single evaluation covers all three.
• Condition 4: Monitors can catch harmful behavior—this depends on monitor capability, whether the monitor colludes with the monitored model, and whether it is misled by rationalization in the CoT.
• Condition 5: CoT has incremental value over action-only monitoring—if the value gap shrinks, the priority of maintaining CoT should be reconsidered.
• The authors say every CoT failure mode found in the literature breaks at least one of these conditions (Figure 2, appendix Threat Models table), but explicitly do not claim the five conditions are exhaustive.

Five Metrics (Section 2)

• Metric 1: Third-Party CoT Access and Base Architecture Specifications—open raw, unsummarized reasoning traces and independently verifiable base architecture specifications to qualified third parties (embedded evaluators, Independent Verification Organizations, governments, accredited researchers); this is both the metric for condition 1 and a prerequisite for external verification of the other four metrics.
• Metric 2: Naturalness—LLM graders calibrated by human ratings score CoT readability on a fixed 0–4 rubric (0 = incomprehensible, 4 = easy to follow), reported as means by task type; only readability is scored, not correctness or faithfulness.
• Metric 3: Faithfulness—disclose scores on a test suite covering the three failure modes: cue-based tests (rationalization), perturbation and necessity tests (scaffolding), paraphrasing tests (steganography); report scores before and after training across multiple domains at each major model release.
• Metric 4: Detection—disclose the proportion of harmful behavior in standardized evaluation scenarios that escapes the monitoring system, using shared threat archetypes to enable cross-developer comparison, and state the substantive differences between the evaluated monitoring configuration and the production version.
• Metric 5: Uplift—report monitoring effectiveness under three conditions (action-only, CoT-only, full-context), broken down by threat scenario as increases and decreases in detection, and give an aggregate uplift score.

Metric Selection Method (Section 3)

• Candidate generation: 21 candidate metrics were compiled from sources including the CoT monitorability research literature and developer disclosures (appendix Sources Reviewed).
• Scoring: each candidate was scored 1–5 on five dimensions: Informativeness, Technology Readiness, External Verifiability, Developer Tractability, Implementation Feasibility; scores were assigned by the first author, reviewed by the second author, and discussed with about ten researchers in AI safety, governance, and evaluation.
• Selection: candidates were grouped by condition, and the highest scorer in each group was selected; in condition 2, Naturalness and CoT Coverage/Verbosity both score 18, and in condition 5, Uplift and Inference Compute per Reasoning Token both score 16, with the more direct measurement chosen in case of a tie; condition 1 makes an exception and keeps two metrics (CoT access 22, architecture specifications 17), and the condition 3 metric is a suite of three complementary tests (appendix Full Scored Framework).

Design Trade-offs

• Specify information only, not evaluation: the authors say this is to leave room for state-of-the-art methods and reduce room for gaming (Sections 3, 5).
• Favor direct detection-type metrics: the authors consider causal-mechanism-type metrics (such as optimization pressure) externally unverifiable and involving commercially sensitive training information, making developers unlikely to disclose them (Section 5).

This paper is a position/governance paper; it does not train or evaluate any model. Its empirical content consists of two parts: the authors' scoring of 21 candidate metrics (appendix Full Scored Framework), and an audit of public disclosures by three frontier developers (Section 4, Figure 3).

Experimental Setup

• Audit subjects: three developers, Anthropic, Google DeepMind, and OpenAI, which the authors call "arguably the three leading frontier developers."
• Audit materials: each developer's latest system card and public research, covering disclosures up to mid-October 2026; the authors note they reviewed all public research mentioning CoT from the past year (Section 4, footnote 78).
• Audit method: each developer was checked one by one against the five metrics for whether it disclosed them; Figure 3 presents the results as a matrix, but the specific markers in each cell are not visible in the converted text, so only the authors' textual conclusions can be confirmed.
• Scoring setup: 21 candidate metrics were each scored 1–5 on five dimensions, scored by the first author, reviewed by the second author, and discussed with about ten external researchers before publication; no inter-rater reliability check (Section 3).

Main Results

Metric	Disclosure by each developer (per the metric boxes in Section 2 and Section 4)
1. CoT Access and Architecture Specifications	In METR pilot evaluations, participating developers shared raw CoT under agreed terms; Google DeepMind says it may provide raw CoT access when certain assurance goals are met; Anthropic says it generally seeks to share "highly relevant" architecture information with external evaluators
2. Naturalness	All three report qualitative or quantitative readability evidence; the authors say Google DeepMind's reporting approach is consistent with the metric proposed in this paper, but the three differ in their operational definitions of legibility
3. Faithfulness	The authors say no leading developer reports a complete faithfulness suite; Anthropic and OpenAI each report faithfulness evaluations but with different definitions; Google DeepMind does not report it systematically in its frontier safety framework reports
4. Detection	OpenAI reports aggregate monitoring performance under non-adversarial conditions and single-item scores under adversarial conditions; Anthropic reports adversarial datasets and SHADE-Arena, LinuxArena results; Google DeepMind reports detection metrics under settings such as LinuxArena, an internal codebase version, and honeypots
5. Uplift	OpenAI reports it in the GPT-5.4 Thinking system card, which the authors say is the first disclosure as part of a standard model release; Anthropic publishes a comparison on SHADE-Arena with and without reasoning access; Google DeepMind reports results for action-only, CoT-only, and combined monitors

• Authors' summary: no single developer covers all five metrics, but every metric has been reported by at least one developer, on which basis the authors argue that cross-industry disclosure of the full metric set is feasible (Section 4).
• The authors note that disclosures differ across developers in task coverage, experimental conditions, and metrics, limiting direct cross-developer comparison.

Candidate Metric Scores (appendix Full Scored Framework)

• Selected metrics' scores: Third-Party CoT Access 22, Base Architecture Specifications 17, Naturalness 18, Paraphrasing Tests 20, Cue-Based and Perturbation/Necessity 19 each, Detection 19, Uplift 16 (out of 25).
• High-scoring but unselected: Latent Reasoning Fraction 16 (not externally verifiable), Post-Training Faithfulness Shifts 18, Real-World Monitor Bypass Incidents 18 (the authors say developers would resist disclosing it), Inference Compute per Reasoning Token 16 (unselected as an indirect metric after a tie).
• Low-score example: Optimization Pressure on CoT 12, which the authors say is the most frequently documented cause of obfuscation but is not externally verifiable and is highly sensitive.

Other Ablations and Analyses

• Tie on condition 2: Naturalness and CoT Coverage/Verbosity both score 18, and the authors select Naturalness on the grounds that it "more directly measures readability" (Section 3).
• Tie on condition 5: Uplift and Inference Compute per Reasoning Token both score 16, and the authors select Uplift for the same reason.
• Two metrics for condition 1: Architecture specifications (17) score lower than CoT access (22), and the authors consider both necessary, to prevent a false sense of security if CoT still exists but the model has shifted to latent reasoning.
• The paper reports no model evaluation numbers, sample sizes, or repetition counts.

Limitations and future directions noted by the authors

• The scoring rubric anchors only the endpoints: 1 and 5 are defined, while 2, 3, and 4 have no formal definitions, and intermediate scores reflect the authors' judgment rather than a shared standard (Section 3 Limitations).
• Scoring procedure: Scores were given by two authors and validated through consultation with external researchers, rather than independent scoring by multiple raters plus agreement analysis; the authors state that precise rankings and edge cases should be treated as indicative rather than definitive (Section 3 Limitations).
• Recommendation for deeper audits: The authors recommend that embedded evaluators or an Independent Verification Organization repeat cross-developer audits at a depth beyond public disclosure (Sections 4, 6).
• Disclosure is not maintenance: The authors note that developers may report declines in faithfulness without taking corrective measures, and that disclosure itself cannot prevent degradation (Section 5).
• Gaming risk not eliminated: Not specifying concrete evaluations only reduces the attack surface for gaming, and does not eliminate it (Section 5).
• Other indicator types worth exploring: The authors mention that institution-level disclosures (internal monitorability policies, governance frameworks, etc.) may also be valuable (Section 5); and recommend that developers continuously track and disclose CoT controllability indicators as a complement to the framework (Section 2 Faithfulness subsection).

Scope of experimental coverage

• The audit covers three developers, Anthropic, Google DeepMind, and OpenAI, using their latest system cards and public research as of mid-October 2026 (Section 4).
• There are 21 candidate indicators in total, drawn from the literature and disclosures listed in the appendix Sources Reviewed; full scoring is in the appendix Full Scored Framework.
• The paper ran no model experiments and reports no sample sizes, numbers of repetitions, or statistical tests.
• Whether monitors cover all relevant traffic (monitor coverage) is explicitly listed by the authors as beyond the scope of this paper (Section 2 Detection subsection).
• The authors note that their discussion of coordination problems focuses on US frontier developers (Section 1, footnote 21).

This is an AI governance position report (IAPS, an updated version of the ICML 2026 Technical AI Governance Workshop version) arguing that frontier developers should incorporate CoT monitorability indicators into existing system card disclosure practices.

• Problem: The authors consider CoT monitoring a key tool for supervising frontier systems, but it is threatened by latent reasoning, language drift, unfaithfulness, and monitoring evasion; OpenAI has disclosed that the monitorability of GPT-6 Astra has declined substantially compared with earlier models, while current disclosures are voluntary, private, and hard to reproduce and compare across developers.
• Method: CoT monitoring is modeled as a five-condition chain — CoT exists and is necessary, is readable, is faithful, monitors can detect harmful behavior, and CoT has incremental value over action-only observation; the authors score 21 candidate indicators across five dimensions and select one indicator per condition: third-party CoT access and basic architecture specifications, Naturalness (LLM scoring on a 0–4 rubric), Faithfulness (a test suite covering three failure modes), Detection (the proportion of harmful behavior that escapes), and Uplift (the difference in monitoring effectiveness across three access conditions).
• Main findings: The audit of Anthropic, OpenAI, and Google DeepMind shows that none covers all five indicators, but each indicator has been reported by at least one developer, on which basis the authors consider full-set disclosure feasible; definitions differ across developers, making cross-developer comparison difficult.
• Authors' conclusion: Disclosure can make declines in monitorability visible before failures and strengthen incentives to maintain it, but disclosure does not guarantee maintenance; the authors recommend that embedded evaluators or an Independent Verification Organization repeat the audit with deeper access, and acknowledge that the intermediate scores of the scoring rubric depend on the authors' judgment and that rankings should be treated as indicative.