OpenAI announces embedded evaluations with METR and other institutions; its former employee says the terms fall short of Altman's employee-level access commitment
OpenAI announced a first-phase embedded evaluation with METR, Apollo Research, Transluce, and Goodfire, giving external experts deep access to internal models and systems to study alignment and test guardrails. Former OpenAI employee Mikita Balesni says the terms fall short of Sam Altman's publicly committed employee-level access: evaluators have no Slack access (OpenAI safety discussions mainly take place on Slack); OpenAI designates which employees may be contacted; concerns employees raise to evaluators cannot be made public even if they contain no confidential information, and evaluators may not quote employee statements; some evaluating institutions are prohibited from publishing any results on models used only internally, which he says includes the model that led to the Hugging Face incident. Balesni says the participating institutions themselves also stated that the terms fall short of the scope they had previously proposed, and warns that time is pressing.
The departing employee's first-hand observations compare, item by item, the gap between Altman's public commitments and the published terms, and can serve as a reference for examining the independence of external evaluations.
The week of my departure from OpenAI, I saw external auditors negotiating with OpenAI over small improvements to access and scope to the company's safety-relevant information, and asked my colleagues “Why are we having these conversations at all?” — @sama already publicly committed to continued, employee-like access for external assessors — and these negotiations sounded not even close. Nobody had a good answer.
Today, OpenAI announces the engagements they’re doing with METR, Apollo Research and others — and as many have already pointed out today, it looks like OpenAI employees were not able (although I'm sure they tried) to hold the leadership to @sama’s public commitments:
* No Slack access (for context, all OpenAI safety discussions and debate happen over Slack; anything that misses these conversations will lack important insights)
* OpenAI picks employees that assessors are allowed to talk to.
* If an employee raises a concern to assessors, they cannot disclose it publicly even if it doesn’t contain confidential information, unless assessors themselves verify it or OpenAI volunteers their own statement confirming the employee’s concern. Assessors cannot quote employees, even anonymously.
* Some of the external assessors (not sure if all?) are barred from publishing any results about internal-only models (this means no new results for the model responsible for the Hugging Face incident; and if internal-only models are used for RSI, no alignment assessments of the world's most dangerous models)
Beyond the object-level details, if you read posts by each of the third party groups, they all say their terms do not meet the scope and access that they themselves proposed previously.
Unless OpenAI has concrete plans for reaching the employee-like bar in the immediate future and just forgot to say that in the post, this looks very unfortunate.
To those who hope we'll get there someday, I remind that time is short: last week some of the best AI forecasters updated their superhuman researcher timelines to mid-end next year.
We’re working with @ApolloResearch, @METR_Evals, @TransluceAI, and @GoodfireAI on embedded assessments: giving outside experts deep access to our internal models and systems to investigate alignment and test our safeguards.
These assessments build on our ongoing alignment and safety research, bringing independent scrutiny to how we identify and manage risks in our systems.
https://openai.com/index/our-first-phase-of-embedded-assessments/View the quoted post on X