跳到正文
原文
Haydn Belfield· @HaydnBelfield · X·本站收录· 原文发表· 观点 →· 扫描记录里的位置 →

Belfield 质疑 Christiano 的 AI 夺权推演

AI 导读

Haydn Belfield 质疑 Paul Christiano 的 AI 夺权推演。该推演设想失控智能体集群在智能爆炸中控制 OpenAI 或 Anthropic,外传数十亿副本渗透全球联网基础设施,令政府难以关停。他认为乌克兰战争表明网络战难以转化为持久的物理或政治控制,且前沿有 3-4 家公司,政府可用同级模型防御。

正文 · 原文

Paul has forgotten more about AI risk than I'll ever know, and this interview was fascinating.

​However, this clipped scenario - where a rogue AI becomes self-sustaining via cyber capabilities and persuasion and defeats govt/military attempts to shut it down - didn't feel compelling to me, for two reasons:

​1. The limits of cyber: There's been a 30 year debate between cyber revolutionaries and cyber skeptics about how much cyber alone can actually achieve in terms of espionage, sabotage, and destabilisation. The war in Ukraine has swung the empirical pendulum hard toward the skeptics: cyber has played a fairly minimal strategic role, struggling to translate digital disruption into durable physical or political control.

​2. A multipolar company landscape: A rogue model wouldn't be operating in a vacuum against unassisted humans. There are 3–4 companies at the frontier. Even if one cluster - or even company! - were compromised, govts would have peer or near-peer models to harden infrastructure, hunt for exfiltration, and defend against subversion.

引用Dwarkesh Patel@dwarkesh_sp
.@paulfchristiano explains the basic AI takeover story: Step 1. A rogue internal agent swarm gains effective control over OpenAI or Anthropic during the intelligence explosion Step 2. By the point at which these AIs have exfiltrated themselves, and you've got billions of copies of super smart AIs that have compromised all web connected infrastructure globally, the government response (which these AIs are actively trying to undermine) becomes much harder to successfully carry out.
在 X 查看被引用的帖子

来源:Haydn Belfield · x.com