研究者批评 OpenAI 外部评估者权限远低于九月承诺的员工级访问
研究者批评 OpenAI 公布的外部评估者访问安排远低于 Sam Altman 九月承诺的员工级访问,不足以支撑真正的监督。Jasmine Wang 转引 @alextmallen 的整理并列出,评估者不能访问 Slack、不能与员工私下交谈、范围需预先商定,还拿不到模型权重与架构等机密信息、不能运行训练实验、只能访问部分选定模型、无法用可解释性工具测量评测感知,也看不到用于对齐评估的内部记录以及 RSI 与自主性数据。帖子提到 HF 调查中调查者最初缺少约 70% 的相关记录,直到证明记录不完整后,OpenAI 才在最后两天提供剩余部分。
This level of access + control is not enough for real oversight. This is also very, very far from the “employee-level access” that Sama promised in September.
It's very limited:
- No access to Slack
- No freedom to talk freely in private with employees
- A narrow, pre-agreed scope
- Other limits @alextmallen outlines below
This is exactly what @tomekkorbak, @balesni, and I were worried about in October: OpenAI gesturing at embedded assessment without living up to the spirit (or even the letter) of the promise.
It’s why we wrote to OpenAI’s board and safety leadership asking them to make sure METR and other assessors got employee-like access. OpenAI responded on Twitter afterwards that they were following through on embedding external assessors.
Today’s news shows OpenAI hasn’t followed through.
I recognize that lots of hard work has gone into these agreements, and am grateful to my ex-colleagues for putting in that work. Notably no labs other than Anthropic & OpenAI have published *any* follow-on to their evaluator commitments. But OpenAI needs to do better here.
These announcements look to be a lot more limited than the "employee-like access" Sam promised two months ago.
The assessors don't get access to:
- Slack.
- Broad context. As OpenAI acknowledges, "assessors need the freedom to follow evidence". It's hard to know what to ask for when you can't see anything other than what's in scope. E.g., in the HF investigation, investigators were missing roughly 70% of the relevant transcripts until they proved the transcripts were incomplete, and OpenAI gave them the remaining transcripts in the last two days.
- Confidential information like model weights or architecture. Model architecture is relevant for understanding how much evaluators can trust what's written in the chain of thought. More importantly, this rules out assessors' access to all sorts of information about how AI is actually being used to research and build more powerful AI systems (e.g., actual transcripts of arbitrary internal use). Given that OpenAI expects assessors "to follow at minimum the same security standards as OpenAI employees", I don't see why they aren't allowed access to the same confidential information.
- Ability to run training experiments or training-based elicitation. Elicitation is a make-or-break component of many evals.
- All the models. They only have access to "selected models" and we don't know what the selection criteria were. OpenAI's OES report previously excluded all the models involved in the HF incident for unclear reasons.
- Ability to use interpretability tools to measure unverbalized evaluation awareness etc. (It looks like Goodfire's access is separate from others'.)
- Internal transcripts for assessing alignment (METR and Transluce). It's not possible for Transluce to properly assess the "quality and coverage" of post-training alignment evaluations without access to detailed information and transcripts from the deployment circumstances which they're meant to reflect. Model behavior in alignment assessments can be sensitive to exact details that might differ from internal usage.
- RSI and autonomy data. Almost all the risk from AI development comes from runaway RSI. But the assessors don't get to see how autonomously AIs are doing research, how much uplift developers are getting, or how spread thin human oversight is.在 X 查看被引用的帖子