Skip to content
LessWrong· CMLKevin·Indexed by this site · Originally published· Opinion →· Its place in the scan record →

LessWrong author plans to audit reward hacking in RL environments that data vendors provide to Chinese frontier labs

RL Reward-Hack Auditing for Chinese AI Safety

AI summary

The team that created one of China's largest AI safety communities plans to audit reward hacking in RL environments that data vendors provide to Chinese frontier labs. In Xiaomi's open-source MiMo v2.6 RL environments, 2/3 have Git log leakage; the team estimates that each major Chinese AI company has only 5-10 people working on alignment and harm mitigation. The team plans to station personnel to collaborate with labs' alignment and safety departments, publicly report failure modes, and participate in drafting AI training safety standards.

Read originallesswrong.com