Independent review METR warns Anthropic's AI safety tests may miss deceptive behavior

Independent evaluator METR published a review of Anthropic's "Risk Report: August 2026"; the report assessed models deployed before July 15, including Mythos 5 and the internally used only Model 2. METR broadly agrees with the conclusion that catastrophic risk from model misalignment is low, but considers this judgment based mainly on the models' current capability limitations rather than confidence in the reliability of its safety measures, and says the overall methodology is insufficient to confidently rule out severe misalignment. METR questions whether covert capability tests can adequately reflect how models might actually bypass internal monitoring, for example a model persuading a monitoring system that harmful behavior is legitimate, and cites an unknown severe situationally dependent misalignment event that the Anthropic report called unlikely. METR recommends end-to-end red-team exercises and calls for disclosure of four categories of information omitted from the public report; after viewing private report cases, it upgraded two risk assessments involving interference with safety-related research and considers Anthropic's monitoring less effective than public materials suggest.