Ryan Greenblatt 宣布加入 METR，继续开展类似其 Hugging Face 报告那样的调查工作。他认为当前关于 AI 开发的大量基础信息未公开，而近期事件让他改变了对公开信息价值的怀疑态度；获取 AI 公司内部经核实的信息尤为紧迫，因为有限公开证据与“即将到来的递归自我改进可能大幅加速能力进展、甚至在一半年内产生极端超人类通用能力”的可能性相符。METR 初期将聚焦能力/起飞、对齐与控制，他希望其他团队覆盖安全、内部流程等领域。
Ryan Greenblatt 宣布加入 METR，继续开展类似其 Hugging Face 报告的调查。他表示，当前大量与灾难性风险高度相关的 AI 开发基础信息并未公开，而近期事件让他改变了对公开信息价值的怀疑态度，认为获取 AI 公司内部经核实的信息尤为紧迫。他提到，现有有限的公开证据与一种可能性相符，即临近的递归自我改进可能大幅加速能力进展，进而可能在 6 个月到一年内产生极端超人类通用能力，并带来相应的大规模最坏结果风险；更多经核实的公开信息可帮助判断这类极端结果在近期是否更可能或更不可能。除能力与起飞外，对齐、安全、控制以及 AI 公司内部风险相关流程的公开证据同样有限。METR 初期计划聚焦能力/起飞、对齐与控制，他希望其他团队覆盖安全、内部流程等领域。Buck Shlegeris 表示与 Ryan 共事约 5 年，认为他此举是正确的，这些调查有望揭示失准风险。
引用Ryan Greenblatt@RyanGreenblatt
I'm joining METR to work on more investigations like our Hugging Face report.
Currently, tons of even basic information about AI development that's highly relevant to catastrophic risk isn't public. I used to be more skeptical of the value of public info, but recent events have changed my mind.
Getting verified information about what's going on inside AI companies seems particularly urgent now. The limited public evidence we have seems consistent with the possibility that imminent recursive self-improvement could massively accelerate capabilities progress, which could then potentially yield extremely superhuman general capabilities within 6 months or a year. If this occurred, there would be a correspondingly large risk of worst-case outcomes. This uncertainty about extreme outcomes could be substantially resolved with more verified public information: we could either build more consensus about near-term risk or learn that such extreme outcomes are less likely in the near term.
Beyond AI capabilities and takeoff, the state of public evidence is also highly limited for alignment, security, control, and risk-relevant internal processes at AI companies. This makes it hard to determine exactly how well or poorly these key areas will go in the near future. (METR plans to focus, at least initially, on just capabilities/takeoff, alignment, and control; I hope other groups cover security, internal processes, and other important areas.)
While I'm no longer working at Redwood, I think the work they are doing is very important; I'm excited about Redwood's ongoing contributions to R&D on technical mitigations and better public interpretation of risk-relevant evidence.
Apollo Research 发布嵌入式评估原则，主张外部评估者应获得接近员工级别的访问权限，用于评估模型谋划导致的失控风险。文章提出有效嵌入式评估需满足四项标准：降低风险、告知公众、形成良好激励、对开发者与评估者双方公平。具体设计上，评估者针对预先固定的具体主张（如模型从未试图破坏自身安全训练、智能体从未未经授权访问外部服务）给出 A 到 E 五级裁定，从 Verified 到 Blocked，并默认公开报告、说明所获访问权限与证据缺口。文章认为纯自愿承诺不足以保障激励，嵌入式评估最终应由法律强制要求；同时建议对严重发现设置两级快速披露机制，最严重情形可直接报告相关机构。