Skip to content
Joe Carlsmith· Joe Carlsmith·Indexed by this site · Originally published

Can we safely automate alignment research?

Can we safely automate alignment research?

AI summary

In the fifth article of the series, Joe Carlsmith explores the feasibility of safely automating alignment research, proposing the concept of an "alignment MVP," where AI reaches or exceeds the alignment research level of top human experts. He argues that alignment research that can be evaluated through empirical feedback loops and formal methods is most promising, and suggests first automating such research at scale, then using it to safely automate the remaining "conceptual alignment research." Key risks include evaluation difficulties, AI scheming, and insufficient resources, but he believes there is still a strong chance of success.

Related papers
Read originaljoecarlsmith.substack.com