Differentiated Automation: Steering Automation R&D Toward Safety and Security

The IAPS report proposes a "differentiated automation" framework, urging the U.S. government to push AI developers to redirect a substantial portion of automation R&D from improving model capabilities to safety and security research. The report notes that automation R&D could accelerate known risks such as cyberattacks and bioweapons, and bring unpredictable new capabilities like evaluation awareness, while the models advancing automation themselves have unresolved safety issues such as scheming, backdoors, and secret loyalty. It proposes four workstreams: establishing visibility into R&D automation, driving industry investment in safety, enhancing non-industry defensive automation capabilities, and cross-domain processes for trusted defense.