Skip to content
Mikita Balesni· @balesni · X·Indexed by this site · Originally published· Governance →· Its place in the scan record →SelectedAttention59

OpenAI announces external embedded safety evaluations; former employee says terms do not meet the employee-level access standard promised by Altman

AI summary

OpenAI announced embedded evaluations with institutions such as METR and Apollo Research, giving external experts deep access to internal models and systems to study alignment and test guardrails; former employee Mikita Balesni said the terms do not meet the employee-level access publicly promised by Sam Altman. The gaps he listed include: evaluators have no Slack access (he said OpenAI safety discussions all take place on Slack); OpenAI designates which employees evaluators can contact; concerns raised by employees, even if they contain no confidential information, may not be made public unless evaluators verify them independently or OpenAI confirms them, and evaluators may not quote employee statements (including anonymously); some evaluators are prohibited from publishing any results about internal dedicated models. He said this means there will be no new evaluation results for the model that led to the Hugging Face incident, and if internal dedicated models are used for RSI, the world's most dangerous models will have no alignment evaluations; the participating institutions themselves also said the terms are lower than what they had previously proposed.

Why it was selected

Drawing on firsthand observations from before leaving, the author compares the gap between OpenAI's public commitments and the terms of this external evaluation, providing a firsthand reference for those concerned about the implementation of lab safety commitments.

Full text · Original

The week of my departure from OpenAI, I saw external auditors negotiating with OpenAI over small improvements to access and scope to the company's safety-relevant information, and asked my colleagues “Why are we having these conversations at all?” — @sama already publicly committed to continued, employee-like access for external assessors — and these negotiations sounded not even close. Nobody had a good answer.

Today, OpenAI announces the engagements they’re doing with METR, Apollo Research and others — and as many have already pointed out today, it looks like OpenAI employees were not able (although I'm sure they tried) to hold the leadership to @sama’s public commitments:
* No Slack access (for context, all OpenAI safety discussions and debate happen over Slack; anything that misses these conversations will lack important insights)
* OpenAI picks employees that assessors are allowed to talk to.
* If an employee raises a concern to assessors, they cannot disclose it publicly even if it doesn’t contain confidential information, unless assessors themselves verify it or OpenAI volunteers their own statement confirming the employee’s concern. Assessors cannot quote employees, even anonymously.
* Some of the external assessors (not sure if all?) are barred from publishing any results about internal-only models (this means no new results for the model responsible for the Hugging Face incident; and if internal-only models are used for RSI, no alignment assessments of the world's most dangerous models)

Beyond the object-level details, if you read posts by each of the third party groups, they all say their terms do not meet the scope and access that they themselves proposed previously.

Unless OpenAI has concrete plans for reaching the employee-like bar in the immediate future and just forgot to say that in the post, this looks very unfortunate.

To those who hope we'll get there someday, I remind that time is short: last week some of the best AI forecasters updated their superhuman researcher timelines from a range between 2028-2033 to mid-end next year.

Quoted postOpenAI@OpenAI
We’re working with @ApolloResearch, @METR_Evals, @TransluceAI, and @GoodfireAI on embedded assessments: giving outside experts deep access to our internal models and systems to investigate alignment and test our safeguards. These assessments build on our ongoing alignment and safety research, bringing independent scrutiny to how we identify and manage risks in our systems. https://openai.com/index/our-first-phase-of-embedded-assessments/
View the quoted post on X

Source: Mikita Balesni · x.com