Eat Your Precision: Precision Objectives Mitigate Reward Hacking in Rubric-Based Reinforcement Learning

The paper proposes the FORK reward, which jointly optimizes recall and precision in rubric-based reinforcement learning to mitigate reward hacking. The authors state that existing rubric rewards are usually defined as weighted by rubric coverage, and the model is in practice mainly maximizing the recall objective, leading to more complete but more verbose answers that contain more unnecessary or factually incorrect content, with preference actually lower than the base model. FORK requires each unit of an answer to be both necessary and factually correct, measured respectively by a rubric-driven necessity score and a factual correctness scorer. Experiments show that FORK improves precision and shortens answers while maintaining similar rubric coverage, raising the win rate against the base policy from 10.6% to 68.0% on RubricHub-Medical and from 22.1% to 74.1% on HealthBench, achieving Pareto optimality in coverage and overall answer quality.