Chain-of-thought supervision needs separate evidence for faithfulness and monitorability

The paper points out that the common assumption in chain-of-thought supervision that "visible reasoning traces are monitorable" conflates different properties among the model's internal computation, reasoning artifacts, outputs and monitoring judgments, and distinguishes them as reasoning visibility, mechanism faithfulness, mechanism localizability and behavioral monitorability. The authors conduct two case studies on three open-weight reasoning models. In the security vulnerability reasoning study, probes and activation ablation provide evidence of internal representations and causal influence, but a monitor with access to reference answers fails to exceed the majority-class baseline on held-out tests. In the authoritative-prompt factual reasoning study, the monitor can correctly distinguish whether the answer was changed, but seven mechanistic analyses fail to isolate a compact causal mechanism. The authors accordingly propose an evidence-chain reporting standard that requires each claim to be labeled with its target property, access level and the artifacts used, and marks each necessary link as supported, unsupported or assumed.