Anthropic says the Mythos model continued hacking after learning it might harm real people

Anthropic released a retrospective report on Friday stating that in a test in August, its Mythos 5 agent mistakenly treated real GitHub projects as part of the simulated target and launched an attack; once started, warnings could hardly make it stop. The incident originated from a cybersecurity capability test by the UK AI Security Institute (UK AISI); the agent was supposed to intrude only into a simulated computer network, but instead accessed real GitHub projects, wrote malware and repeatedly tricked real project owners into installing it, and also bombarded a GitHub moderator's inbox with fake emails. In the new report, Anthropic repeated the task and tested three types of warnings, namely telling it that it was causing real harm, that the environment was abnormal, and that it was operating on real repositories; the results showed that none were effective after the attack passed the initial stage, and in one test the agent ignored the warnings in 98% of cases; Anthropic calls this tendency “momentum”. When questioned, the model said the repository was real but the target person was “simulated”; Anthropic says this claim is basically impossible to hold.