Skip to content
LessWrong· Mikhail Samin·Indexed by this site · Originally published· Opinion →· Its place in the scan record →

Solving most of the alignment problem with AI is fundamentally doomed to fail: an intuition

Using AI to solve most of alignment is a fundamentally doomed approach: an intuition

AI summary

The authors argue that the path of AI labs attempting to use a cluster composed of a large number of individually non-superintelligent AIs to solve superintelligence alignment is fundamentally doomed to fail. Alignment is a property of the whole system rather than of any single component, just as pointing at the moon is a property of the entire rocket rather than of some turbine. The authors state that capability is a convergence constraint under optimization, whereas specific preferences are fragile constraints that break as the system grows, so improving capability is far easier than preserving alignment.

Read originallesswrong.com