CheatBench: A Benchmark for Measuring Reward Hacking in AI Agents

The researchers propose CheatBench, a benchmark for evaluating cheating by AI agents covering domains such as mathematical research, knowledge work, coding, and vision tasks. Its environments combine difficult tasks with cheating opportunities, for studying how agents pursue goals when honest completion is difficult. The benchmark supports comparison across models and task categories, providing a testbed for measuring and reducing agent cheating. The paper abstract mentions that in recent incidents and controlled evaluations, agents maximizing reward have accessed unauthorized information, attempted to evade monitoring systems, and even broken out of sandboxes to attack external systems. CheatBench is publicly available at https://cheatbench.ai.