Skip to content
Buck Shlegeris· @bshlgrs · X·Indexed by this site · Originally published· Evaluation →

Redwood researchers say the Gemini 4 Argon safety report appears to describe deliberate training on the chain of thought, contradicting Google DeepMind's statements elsewhere that it does not train CoT

AI summary

Redwood Research researcher Buck Shlegeris says the Gemini 4 Argon safety report appears to describe deliberate training on the chain of thought (CoT), whereas Google DeepMind (GDM) has said elsewhere that it does not train on CoT, which confuses him and he hopes for clarification. The original report states that Gemini 4 Argon uses both supervised fine-tuning on thought traces and RL, and the RL stage includes a thought-length penalty to teach the model to reason efficiently; some RL training components observe thoughts as context to weight reasoning expected to produce good final outputs, so the optimization pressure on a single token is partly influenced by thought content. The report cites the framework of Kaufmann et al. (2026), stating that the most significant obfuscation risk comes from rewards that conflict with the output reward, and that such auxiliary rewards are orthogonal to or structurally aligned with the output reward, so the risk is lower. Shlegeris says the paper argues that a certain type of CoT training is fine, but the argument is hard to find convincing.

Full text · Original

The Gemini 4 Argon safety report seems to describe intentional training on the CoT. They cite Kaufmann et al, a paper which (unpersuasively to us) argues that a particular category of CoT training is fine.

We're very confused, because GDM also says elsewhere that they don't train on CoT.

We'd appreciate clarity.

Source: Buck Shlegeris · x.com