Skip to content
LessWrong· Cleo Nardo·Indexed by this site · Originally published· Opinion →· Its place in the scan record →

The author considers the argument that "safety is harmful because warning is beneficial": if the button only makes the current AI safer, its best guess is to press it

Thoughts on "safety bad bc warning shots good"

AI summary

The author examines the warning argument that "making the current AI safer is instead harmful," and the best guess is that the current AI should still be made safer. The argument claims that future agents will contribute nearly all catastrophic risk, and that making the current AI safer would reduce non-catastrophic harms and lower attention from all parties to catastrophic risk; the author points out that it ignores higher-order effects, and that the required quantitative modeling has not been done by anyone. The author also mentions that open-weight models lag the frontier by about 6 months, and that the current AI can already take on safety work such as monitoring and research.

Read originallesswrong.com