Lessons in AI safety research I learned at MATS and Redwood
Research Lessons I Learned at MATS & Redwood
AI summary
A researcher doing empirical AI safety research at MATS and Redwood Research says the key to accelerating experimental iteration is building predictable, falsifiable models of the problem and updating them with data. Their work includes fine-tuning-based misalignment auditing and capability removal evaluation. Using LinuxArena as an example, the author notes that a weak monitoring model captures 80% of attacks under a small audit budget, but with only 5 attack samples, one should not directly update a 50% prior to 80%.