METR questions Anthropic's AI safety testing, though it agrees with its low-risk conclusion

Independent evaluator METR released an external review on October 20 of the "misalignment in high-risk settings" section of Anthropic's August 2026 risk report, covering models deployed before July 15, including Mythos 5 and the internally dedicated Model 2. METR agreed with the conclusion that catastrophic risk is low, but believes this stems mainly from limited model capabilities rather than guardrails and alignment training. The core disagreement is over covert capability testing: METR believes the benchmarks do not adequately reflect how models interact with actual monitoring systems, citing the Mythos 5 incident—an offline monitor flagged about 1% of model behavior, rising to about 50% after removing reasoning—but does not believe the model intentionally manipulated the monitor. METR calls for disclosure of four redacted pieces of information, recommends end-to-end red-team testing of monitoring systems, and suggests extending risk assessment to the possibility of models sabotaging safety-related decisions. The review terms gave Anthropic veto power, but METR confirmed the review was not altered.