The blind spot in AI safety

A commentary argues that a paper by Anthropic researchers presented at ICLR 2026 uses bias-variance decomposition to measure the consistency of model outputs, inferring from it that frontier AI is more likely to fail in unpredictable "hot mess" ways, and advocates that governance focus on accident prevention rather than constraining misaligned goals. The author points out that this framework measures whether outputs cluster around their own mean, not whether they align with reality, and therefore cannot assess whether models remain grounded in truth in open-ended, evolving scenarios. The author proposes a third failure mode between the two, "epistemic drift": models that are internally consistent and pass standard validation yet gradually diverge from truth in direction. The article cites a 2025 JAMA Health Forum study of nearly 1000 FDA-approved AI medical devices, stating that 43% of recalls occurred within one year of authorization. The author argues that the NIST AI risk framework, as well as the FDA, SEC, FTC, and others, would inherit the same blind spot if they use such research as their technical foundation.