跳到正文
原文
Jasmine Wang· @j_asminewang · X·本站收录· 原文发表· 治理 →· 扫描记录里的位置 →

加州新 AI 安全报告建议紧急关停权与封顶递归自我改进算力

AI 导读

加州州长纽森召集的专家小组发布 AI 安全报告,建议授予紧急关停权、为 AI 开发者设定最低安全标准、强制第三方监督,并授权封顶用于递归自我改进的算力。报告还建议对自动化 AI 研发进行经验证的算力披露、强制失控报告并扩大吹哨人保护。作者 Jasmine Wang 称这是她见过的各国政府中针对递归自我改进最严肃的政策方案,并称 AI 开发者似乎正走向在 6-12 个月内实现递归自我改进,希望州立法机构下个月推进落实。

正文 · 原文

California’s new AI safety report is the most serious policy proposal for recursive self-improvement from any government I’ve seen. Very exciting!

It recommends the procedural machinery to actually slow down if & when we need to: verified compute disclosures for automated AI R&D, standby authority to cap that compute, emergency shutdown powers, mandatory loss-of-control reporting, and broadened whistleblower protections.

You can’t pace what you can’t see. If implemented, this would let the state see, and then act.

I hope the legislature moves on this when it’s back next month—AI developers seem on track to recursive self-improvement within 6-12 months, so we need these mechanisms practiced and working ASAP.

引用Nathan Calvin@_NathanCalvin
The recommendations from Governor Newsom's recently convened expert panel - Professor Alondra Nelson (Princeton), Jason Goldman, Professor Robert Reich (Stanford), and Professor Gillian Hadfield (Johns Hopkins) - were just released and are a clarion call for urgent action to address risks to public safety from advanced AI systems. Some of the most notable recommendations: • Emergency shutdown authority • Minimum safety standards for AI developers • Mandatory third party oversight • Authority to cap the amount of compute used for RSI The next weeks and months will determine whether these recommendations are words on paper, or whether the Governor and legislature can work together to accomplish something historic to protect CA and lead the country on guardrails to prevent catastrophic risks from runaway AI development.
在 X 查看被引用的帖子

来源:Jasmine Wang · x.com