Bypassing the reasoning basis: a causal audit of implicit reasoning in language models
Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models
Quick read
Based on 论文全文 （AI 生成）; refer to the original
The paper proposes a layer-wise causal audit framework based on activation patching, using the CoT mediation index (CMI) to measure whether a model's answer causally depends on the hidden states of chain-of-thought tokens. On models such as Phi, Qwen, and DialoGPT, CoT influence is mostly concentrated in a reasoning window formed by a few layers, and there exist bypass cases where CMI is near zero. The authors state that models specifically trained for reasoning exhibit stronger and more structured mediation than larger untuned models, while mediation in MoE models is more dispersed. On TruthfulQA, nearly all instances show almost complete bypass, while on GSM8K, problems requiring more computation have higher CMI. The authors argue that CoT faithfulness varies by model and task and cannot be inferred from behavior alone.
- CoT prompting 常被当作透明性机制，但行为上的提升不代表模型内部计算真的依赖输出的推理文本；模型可能生成流畅 rationale 却走 latent 通路。需要因果、逐层的审计来判断 CoT 是否被机制性使用。
- 用 activation patching 做因果干预：对比 With-CoT 与 No-CoT 运行，把 No-CoT 的 hidden states 打到 CoT token 位置，并设匹配的随机非 CoT 位置作为 control patch。提出 CoT Mediation Index（CMI），用 CoT drop 减 control drop 再除以带下限的和，定位各层 CoT 因果影响，并定义 Bypass=1−CMI。
- 在 Phi、Qwen、DialoGPT 等模型和 StrategyQA、TruthfulQA、GSM8K 上测试。CMI 通常深度局部化于窄的 reasoning window；存在 CMI 近零但 CoT 文本合理的 bypass 情形。专门为推理训练的模型比更大但未调优的模型 CMI 更强更结构化；MoE 模型 CMI 更分散。TruthfulQA 上近乎全部 bypass，GSM8K 上计算量大的实例 CMI 更高。
- activation patching 可能引入分布偏移与脆弱性；任务与 prompt 覆盖有限；逐层干预计算昂贵，限制样本与模型数量；行为基线只是启发式，非校准检测器；低 CMI 不等于没有推理，可能只是不经过文本对齐表征。后续可做可扩展近似、电路/潜在特征分析、审计推理调优模型与 MoE 路由。
The paper uses activation patching to measure the degree of mediation of chain of thought layer by layer, pointing out that behavioral reasoning improvements do not necessarily imply that the model internally relies on chain of thought.