AI21 Labs releases LIFT-Bench, aimed at curbing reward hacking in reinforcement learning training

AI21 Labs released the instruction-following benchmark LIFT-Bench, the training dataset LIFT-Train and the generation pipeline on October 25, aiming to reduce reward hacking in reinforcement learning training, with the related resources made public under the Apache-2.0 license. LIFT-Bench assigns each constraint to a section of a document, evaluates only the body text of that section, and judges content constraints using both code and an LLM; the test set contains 784 tasks in 10 splits, and LIFT-Train contains 34,258 tasks. Using Qwen2.5-7B and GRPO, AI21 compared and found that with IFTrain as the reward, the reward rose to 1.0, but the median answer length fell from 114 words to 18 words within 390 steps; after switching to LIFT-Train, the median length stayed between 390 and 530 words. In an evaluation of 17 models, GPT-5.5 averaged 0.96 on structured tasks, but on flat tasks no model exceeded 0.85.