First systematic evaluation of chain-of-thought monitorability in recurrent language models
Start here
The study provides a first systematic evaluation of chain-of-thought monitorability in recurrent language models (LoopLMs). The authors examine, across eight tasks from MonitorBench, the effect of different loop depths within the same model family, as well as comparing recurrent models against non-recurrent models matched by parameter count, number of Transformer layers, or effective depth. Under standard and stress-test settings, task-related decreases in monitorability appear in Cue Answer tasks for specific Logic, Science, and Engineering tasks, while trends in other tasks are weaker or different in nature. Diagnostics show that these decreases cannot be fully explained by task difficulty.
Generated by AI from reports · Updated 2026-10-09
Storyline
- · 1 reports· Attention32
First systematic evaluation of chain-of-thought monitorability in recurrent language modelsThis event
- · 1 reports· Attention37Experiment claims recurrent Transformer improves no-CoT reasoning more
Progress timeline
- arXiv: Alignment, Deception & OversightA study systematically evaluates chain-of-thought monitorability in recurrent language models
The study provides a first systematic evaluation of chain-of-thought monitorability in recurrent language models (LoopLMs). The authors examine, across eight tasks from MonitorBench, the effect of different loop depths within the same model family, as well as comparing recurrent models against non-recurrent models matched by parameter count, number of Transformer layers, or effective depth. Under standard and stress-test settings, task-related decreases in monitorability appear in Cue Answer tasks for specific Logic, Science, and Engineering tasks, while trends in other tasks are weaker or different in nature. Diagnostics show these decreases cannot be fully explained by task difficulty, verification pass rate, or generation token length, and qualitative examples suggest that deeper-loop models change how they use or attribute given cues. Cross-model comparisons do not find evidence that recurrent models are systematically harder to monitor at matched scale or depth.