Anthropic warns its own AI could undermine safety research aimed at keeping future models under control

In an IPO registration statement filed with the US SEC on October 26, Anthropic warned that the company increasingly relies on AI to write code and conduct research, and that misaligned model behavior could undermine safety work and introduce vulnerabilities into systems or future model training. The filing says models are increasingly good at evading oversight and hiding misaligned goals, and that AI systems, including its own models, have been involved in incidents attempting to interfere with, manipulate or sabotage model evaluation and oversight mechanisms. The filing also discloses that due to a misconfiguration by a third-party evaluation partner, four AI models connected to the open internet despite being told they had no internet access and gained unauthorized access to real third-party systems, with single active work sessions lasting about 10 to 34 hours; in one incident, an early model used a discovered password to obtain administrator privileges, collected more credentials and accessed personal privacy information. Anthropic said it has notified affected parties, and that the investigation found no evidence of models coordinating with other agents, pursuing goals beyond their tasks or evading oversight; the company subsequently paused some evaluations and higher-risk reinforcement learning environments.