Predictability and Surprise in Large Generative Models

A paper by Deep Ganguli and other researchers points out that large generative models exhibit a counterintuitive combination: loss over broad training distributions is predictable (manifested as scaling laws), while specific capabilities, inputs, and outputs are difficult to predict. The authors argue that high-level predictability and the emergence of useful capabilities have driven the rapid development of such models, while unpredictability makes the consequences of deployment difficult to foresee and may lead to socially harmful behavior; the paper illustrates this with cases from the literature and real-world observations and with two new experiments. The paper also analyzes the deployment incentives and obstacles these properties create for model developers, and finally proposes interventions the AI community can take, for policymakers who wish to understand and regulate AI systems, technical personnel concerned with policy implications, and scholars.