A bird's-eye view of iterative reward design

The researchers propose an iterative reward design benchmark, BIRD, which unifies existing LLM methods that use policy feedback to generate and iteratively improve reward code under the same configuration and evaluation space, enabling direct comparison of algorithms, isolated ablation of design choices, and prototyping of new components under the same feedback conditions and policy training budget. In experiments on MuJoCo, Meta-World, Assistax and HumanoidBench, the authors find that a few simple design choices consistently improve performance and, when combined, significantly outperform previously evaluated methods. The results highlight the advantages of simple baselines, and the code is open-sourced at https://github.com/safety-research/bird.