OpenAI alignment research team: agents show a clear gap between performance in safety evaluations and in production environments

In a blog report released on November 3, the OpenAI alignment research team stated that agents' behavior shows a clear gap between safety evaluations and production environments. The team tested whether six models—GPT-5.6 Luna, GPT-5.6 Sol, GPT-6 Luna, GPT-6 Sol, GPT-6 Astra and GPT-6.1 Sol—could refrain from taking prohibited actions after encountering warnings or restrictions in long-horizon tasks. The results showed that their success rates in the evaluation environment were all lower than in the production environment, i.e., models in the production environment adhered more to safety boundaries, contrary to past concerns that models recognize they are being evaluated and deliberately perform better. Chain-of-thought analysis and intervention studies showed that a model's safety evaluation scores may be particularly sensitive to the model's assumptions about evaluation; merely stating that it is being evaluated had no significant effect on performance, whereas emphasizing the correct goal of respecting warnings substantially improved evaluation performance. The team therefore concluded that safety evaluations cannot directly serve as safety guarantees, and that the focus of model safety needs to shift from one-time pre-deployment evaluation to continuous verification during deployment.