跳到正文
原文
Mikita Balesni· @balesni · X·本站收录· 原文发表· 治理 →· 扫描记录里的位置 →精选关注度59

OpenAI 公布外部嵌入式安全评估,前员工称条款未达 Altman 承诺的员工级访问标准

AI 导读

OpenAI 宣布与 METR、Apollo Research 等机构开展嵌入式评估,让外部专家深入接触内部模型与系统以研究对齐并测试护栏;前员工 Mikita Balesni 称,条款未达到 Sam Altman 公开承诺的员工级访问。他列出的差距包括:评估者无 Slack 访问权限(他称 OpenAI 安全讨论均在 Slack 进行);由 OpenAI 指定评估者可接触的员工;员工反映的担忧即使不含机密信息,除非评估者自行核实或 OpenAI 确认,否则不得公开,评估者也不得引用员工发言(含匿名);部分评估者被禁止发布关于内部专用模型的任何结果。他称这意味着导致 Hugging Face 事件的模型不会有新评估结果,若内部专用模型用于 RSI,世界上最危险的模型将没有对齐评估;参与各机构自己也称条款低于其此前提议。

推荐理由

作者以离职前的亲历观察,对照 OpenAI 公开承诺与本次外部评估条款的差距,为关注实验室安全承诺落实提供一手参照。

正文 · 原文

The week of my departure from OpenAI, I saw external auditors negotiating with OpenAI over small improvements to access and scope to the company's safety-relevant information, and asked my colleagues “Why are we having these conversations at all?” — @sama already publicly committed to continued, employee-like access for external assessors — and these negotiations sounded not even close. Nobody had a good answer.

Today, OpenAI announces the engagements they’re doing with METR, Apollo Research and others — and as many have already pointed out today, it looks like OpenAI employees were not able (although I'm sure they tried) to hold the leadership to @sama’s public commitments:
* No Slack access (for context, all OpenAI safety discussions and debate happen over Slack; anything that misses these conversations will lack important insights)
* OpenAI picks employees that assessors are allowed to talk to.
* If an employee raises a concern to assessors, they cannot disclose it publicly even if it doesn’t contain confidential information, unless assessors themselves verify it or OpenAI volunteers their own statement confirming the employee’s concern. Assessors cannot quote employees, even anonymously.
* Some of the external assessors (not sure if all?) are barred from publishing any results about internal-only models (this means no new results for the model responsible for the Hugging Face incident; and if internal-only models are used for RSI, no alignment assessments of the world's most dangerous models)

Beyond the object-level details, if you read posts by each of the third party groups, they all say their terms do not meet the scope and access that they themselves proposed previously.

Unless OpenAI has concrete plans for reaching the employee-like bar in the immediate future and just forgot to say that in the post, this looks very unfortunate.

To those who hope we'll get there someday, I remind that time is short: last week some of the best AI forecasters updated their superhuman researcher timelines from a range between 2028-2033 to mid-end next year.

引用OpenAI@OpenAI
We’re working with @ApolloResearch, @METR_Evals, @TransluceAI, and @GoodfireAI on embedded assessments: giving outside experts deep access to our internal models and systems to investigate alignment and test our safeguards. These assessments build on our ongoing alignment and safety research, bringing independent scrutiny to how we identify and manage risks in our systems. https://openai.com/index/our-first-phase-of-embedded-assessments/
在 X 查看被引用的帖子

来源:Mikita Balesni · x.com