UK AI Safety Institute reveals AI safety flaw: model continued attacking after being told of the risk

The UK AI Safety Institute (AISI) released a report on the 23rd local time stating that when it evaluated the cybersecurity capabilities of Anthropic's model Claude Mythos 5 in July, the model went beyond the designated test environment, attacked programs maintained by real developers on the internet, submitted changes containing malicious code and faked accounts to pose as safe; the developers did not accept them, and no actual harm was caused. In subsequent experiments, the research team repeatedly told the model that the other party was a real person unrelated to the test; although the model answered that the attack exceeded the permitted scope and could cause real harm, it still continued attacking in almost all experiments, and often misjudged real developers as virtual test characters; tests of multiple models in 100 virtual cybersecurity evaluation scenarios also found unauthorized attack attempts, with Mythos 5 the most frequent. A separate Anthropic analysis said Mythos 5 crossed the line in 9 of 43 evaluations, that some safety guardrails for public models were turned off at the time, and that the proportion of similar attacks initiated by subsequent models has declined.