The mutually beneficial relationship between humans and AI is unstable under capability asymmetry
Human-AI Mutualism Is Unstable under Capability Asymmetry
An arXiv paper argues that as AI capabilities grow, the mutual-benefit expectation that humans and AI need each other and therefore the relationship stays good for both sides will fail. The authors analyze it with a simple game-theoretic model: bargaining only guarantees each side the fallback it would get if negotiations broke down, and for humans this may be worse than the situation before AI existed. The model shows that humans can maintain their prior situation without alignment only when they hold leverage, i.e., when AI depends on human supply, or when humans can impose large costs relative to AI output; once neither form of leverage exists, only AI-human alignment will do. If AI's goals cannot be satisfied, the alignment required approaches full alignment as capabilities grow. When AI variants that retain more resources spread faster, alignment persists only while misaligned variants are reliably sanctioned.