A mess of AI: How does misalignment scale with model intelligence and task complexity?

The study finds that across all tested tasks and frontier models, the longer an AI model reasons and acts, the more incoherent its failures. The paper uses a bias-variance decomposition to measure this error incoherence, i.e., the proportion of error attributable to variance rather than bias under test-time randomness. How error incoherence changes with model scale varies by experiment, but in multiple settings, larger and stronger models are more incoherent than small models, indicating that scaling alone appears insufficient to eliminate this incoherence. The authors accordingly predict that as stronger AI takes on difficult tasks requiring more continuous action and thought, failures will be accompanied by more incoherent behavior, suggesting that future AI will sometimes cause industrial accidents through unpredictable inappropriate behavior, but less often manifest as persistent pursuit of a misaligned objective; the authors believe this raises the relative importance of alignment research targeting reward hacking and goal misspecification.