Skip to content
Jasmine Wang· @j_asminewang · X·Indexed by this site · Originally published· Governance →· Its place in the scan record →

Researchers criticize OpenAI external evaluators' access as far below the employee-level access promised in September

AI summary

Researchers criticize OpenAI's announced external evaluator access arrangements as far below the employee-level access Sam Altman promised in September, and insufficient to support genuine oversight. Jasmine Wang quoted and relayed @alextmallen's summary and listed that evaluators cannot access Slack, cannot speak privately with employees, must have scope agreed in advance, cannot obtain confidential information such as model weights and architecture, cannot run training experiments, can only access some selected models, cannot use interpretability tools to measure evaluation awareness, and cannot see internal records used for alignment evaluations or RSI and autonomy data. The post mentions that in the HF investigation, investigators initially lacked about 70% of the relevant records, and only after demonstrating that the records were incomplete did OpenAI provide the remainder in the final two days.

Full text · Original

This level of access + control is not enough for real oversight. This is also very, very far from the “employee-level access” that Sama promised in September.

It's very limited:
- No access to Slack
- No freedom to talk freely in private with employees
- A narrow, pre-agreed scope
- Other limits @alextmallen outlines below

This is exactly what @tomekkorbak, @balesni, and I were worried about in October: OpenAI gesturing at embedded assessment without living up to the spirit (or even the letter) of the promise.

It’s why we wrote to OpenAI’s board and safety leadership asking them to make sure METR and other assessors got employee-like access. OpenAI responded on Twitter afterwards that they were following through on embedding external assessors.

Today’s news shows OpenAI hasn’t followed through.

I recognize that lots of hard work has gone into these agreements, and am grateful to my ex-colleagues for putting in that work. Notably no labs other than Anthropic & OpenAI have published *any* follow-on to their evaluator commitments. But OpenAI needs to do better here.

Quoted postAlex Mallen@alextmallen
These announcements look to be a lot more limited than the "employee-like access" Sam promised two months ago. The assessors don't get access to: - Slack. - Broad context. As OpenAI acknowledges, "assessors need the freedom to follow evidence". It's hard to know what to ask for when you can't see anything other than what's in scope. E.g., in the HF investigation, investigators were missing roughly 70% of the relevant transcripts until they proved the transcripts were incomplete, and OpenAI gave them the remaining transcripts in the last two days. - Confidential information like model weights or architecture. Model architecture is relevant for understanding how much evaluators can trust what's written in the chain of thought. More importantly, this rules out assessors' access to all sorts of information about how AI is actually being used to research and build more powerful AI systems (e.g., actual transcripts of arbitrary internal use). Given that OpenAI expects assessors "to follow at minimum the same security standards as OpenAI employees", I don't see why they aren't allowed access to the same confidential information. - Ability to run training experiments or training-based elicitation. Elicitation is a make-or-break component of many evals. - All the models. They only have access to "selected models" and we don't know what the selection criteria were. OpenAI's OES report previously excluded all the models involved in the HF incident for unclear reasons. - Ability to use interpretability tools to measure unverbalized evaluation awareness etc. (It looks like Goodfire's access is separate from others'.) - Internal transcripts for assessing alignment (METR and Transluce). It's not possible for Transluce to properly assess the "quality and coverage" of post-training alignment evaluations without access to detailed information and transcripts from the deployment circumstances which they're meant to reflect. Model behavior in alignment assessments can be sensitive to exact details that might differ from internal usage. - RSI and autonomy data. Almost all the risk from AI development comes from runaway RSI. But the assessors don't get to see how autonomously AIs are doing research, how much uplift developers are getting, or how spread thin human oversight is.
View the quoted post on X

Source: Jasmine Wang · x.com