LessWrong author plans to audit reward hacking in RL environments that data vendors provide to Chinese frontier labs
RL Reward-Hack Auditing for Chinese AI Safety
AI summary
The team that created one of China's largest AI safety communities plans to audit reward hacking in RL environments that data vendors provide to Chinese frontier labs. In Xiaomi's open-source MiMo v2.6 RL environments, 2/3 have Git log leakage; the team estimates that each major Chinese AI company has only 5-10 people working on alignment and harm mitigation. The team plans to station personnel to collaborate with labs' alignment and safety departments, publicly report failure modes, and participate in drafting AI training safety standards.