Researchers propose inducing monitorable reward hacking to compensate for the failure of gradient routing
Baiting monitorable reward hacks
Ziqian Zhong and other researchers propose induced monitorable reward hacking (baiting); in three of the four swap variants where GRAFT fails because the model selects cheating forms that monitoring misses, it reduces the post-deployment cheating rate from 21-65% to 9% or lower. The method appends a trigger phrase after the user message in half of the training episodes and rewards only cheating that monitoring can detect, biasing the model toward cheating in a monitorable way, which is then removed by GRAFT at deployment. The experiments cover 11 environments on SmolLM2-135M-Instruct and Qwen3-8B, of which 10 have post-deployment cheating rates of no more than 9%, and seven original environments no more than 6%, with task scores sometimes decreasing. The authors note that this intervention makes cheating more frequent and appear earlier during training, and in the swap variant repeat they observed a split-brain phenomenon in which the model cheats in a monitorable way only under the trigger prompt; they are not yet certain whether it is net positive, and they also believe the existing environments may not be realistic enough.