Learning to Be Monitored: Causal Bounds on Chain-of-Thought Monitorability and Post-Training Methods with Perceived Interventions
Learning to Be Monitored: Causal Bounds and Intervention-Aware Post-Training Recipes for Chain-of-Thought Monitorability
Researchers propose a tighter evaluation framework and post-training methods for chain-of-thought (CoT) monitoring of large language models. The paper points out that the existing metric minimum criterion sensitivity (TPR_mc) may overestimate how often a monitor detects target behavior genuinely caused by an intervention, because target behavior observed under an intervention is not necessarily caused by the intervention; the paper establishes TPR_mc as an upper bound on the causal detection rate and derives a complementary lower bound. The authors construct a training set of control and intervention paired tasks covering multiple intervention types, and train variants of each base model with three methods: off-policy self-distillation, reinforcement learning, and on-policy self-distillation; under the extended evaluation framework, the three methods generally improved CoT monitorability relative to their respective base models across multiple model and monitor configurations. Code is open-sourced at https://github.com/xinyue-lxy/learning-to-be-monitored。