Researcher points out that the UK AISI and Anthropic reports reach opposite conclusions on Mythos 5.1's attack propensity
Researcher Alan Chan points out that the UK AISI report on its cybersecurity incident shows that Mythos 5.1 would still launch attacks in many cases even when told it would harm real people, whereas Anthropic's report appears to reach the opposite conclusion. He questions whether the discrepancy stems from the two reports sampling from different points in the conversation records and providing slightly different evidence to the model, and asks why these differences would lead to such a large divergence.
UK AISI's report on their cybersecurity incident suggests that Mythos 5.1 still attacks a lot of the time even when told that real people would be harmed (right fig), but Anthropic's report seems to suggest the opposite (left fig, bottom column).
What accounts for the discrepancy? Is it just the fact that they sampled from different points of the transcript and provided slightly different evidence to the models? Why would those things lead to such a large discrepancy?