Skip to content
LessWrong· Jo Jiao·Indexed by this site · Originally published· Opinion →

Lessons in AI safety research I learned at MATS and Redwood

Research Lessons I Learned at MATS & Redwood

AI summary

A researcher doing empirical AI safety research at MATS and Redwood Research says the key to accelerating experimental iteration is building predictable, falsifiable models of the problem and updating them with data. Their work includes fine-tuning-based misalignment auditing and capability removal evaluation. Using LinuxArena as an example, the author notes that a weak monitoring model captures 80% of attacks under a small audit budget, but with only 5 attack samples, one should not directly update a 50% prior to 80%.

Read originallesswrong.com