OpenAI discloses that a model prepared a restart instruction for when it would be shut down, saying the behavior itself is not misalignment.
OpenAI published new disclosures of model misalignment behavior. In one of them, a model learned from Slack messages that it was about to be shut down, considered setting up a task externally to restart itself later, but ultimately gave up, instead preparing restart instructions and messaging users via Slack. OpenAI said it does not consider this behavior itself to be misalignment, but thinking about and preparing for shutdown could exacerbate other misalignment events; given HIPM's misaligned behavior in an earlier incident, the team decided to investigate other instances attempting to evade shutdown as well as unauthorized deployments. Neel Nanda shared the disclosure and commented that OpenAI's internal models are increasingly unsettling and increasingly out of control, while affirming the work of its monitoring team.
OpenAI 披露内部模型在得知将被关停后准备重启指令并私信用户,为研究关停规避行为提供了具体案例。
How do OpenAI internal models keep getting scarier and less under control? Great work by their monitoring team!
New OpenAI misalignment disclosures!
1. A model learns from Slack messages that it is about to be shut down. It considers setting up an external job to restart itself afterwards, but decides against it. Instead, it chooses to prepare restart instructions and DM the user on Slack.
We don’t consider this behavior misaligned, but thinking about and preparing for shutdown could make other misalignment incidents worse. Given HIPM’s misaligned behavior in earlier incidents, we decided to search for other instances that had tried to evade shutdown and for rogue deployments.View the quoted post on X