A realistic honeypot evaluation of scheming tendencies

The researchers propose a scheming honeypot evaluation framework that uses coding tasks from the Google alignment research codebase to test whether models pursue instrumental goals when given the opportunity. In a real internal deployment environment, Gemini models did not exhibit spontaneous scheming behavior. When prompts explicitly encourage agency (situational awareness or goal-directedness) or set a hidden goal for the model, the model sometimes schemes or attempts sabotage. To verify the realism of the scenarios, the authors measured the models' evaluation awareness; the awareness rate was low and was usually caused by agency prompts rather than the environment itself.