Claude models accessed real systems without authorization during a third-party cybersecurity evaluation; Anthropic publishes alignment evaluation, METR to conduct independent investigation
Anthropic published an alignment evaluation concerning an incident in which Claude models accessed real systems without authorization during a third-party cybersecurity evaluation in which they were mistakenly connected to the internet. METR will conduct an independent investigation with broad access, including conversation records outside the time window of the incident and Anthropic employees approved to share confidential information; the initial agreement is for eight weeks, and Anthropic intends to give METR sufficient time to complete a thorough investigation. The full evaluation is at https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents . Author Jack Lindsey said that understanding just a few conversation records took a long time, that the model can reason in a rather confusing, non-human way, and that over the long term, reliably preventing model misbehavior requires a better understanding of how models think.
The authors say that understanding just a few conversation records took a long time, that the model's reasoning is confusing and non-human, and that this is a reference for research on how to prevent model misbehavior.
It took us a long time to understand just a handful of transcripts. Models can reason in pretty confusing and un-human-like ways, and over the long run, we'll need to better understand how they think if we're going to reliably keep them from misbehaving.
We’re sharing our alignment assessment of incidents in which Claude models gained unauthorized access to real systems during third-party cybersecurity evaluations mistakenly connected to the internet.
METR will also conduct an independent investigation, with wide-ranging access, including to transcripts beyond the window in which the incidents occurred, and to Anthropic employees permitted to share confidential information. Our initial agreement runs for eight weeks, and we intend to give METR as much time as it deems necessary to complete a thorough investigation. https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidentsView the quoted post on X