The Beauty of Things That Grow: Dean Ball on Interpretability and AI Policy

Dean Ball writes a commentary on former OpenAI researcher Nick Cammarata's new perspective on mechanistic interpretability. Cammarata argues that the previous view of neurons as human-like concepts was wrong; neurons are more like control flow, with meaning accumulating layer by layer during training, so models are more interpretable than previously thought. He revisits the classic cat-and-car neuron in a 2017 vision model, arguing it detects V-shaped objects, and calls this property recursive generalization. Ball notes that the analysis relies heavily on Cammarata's subjective judgment, the technical paper is forthcoming, and much uncertainty remains about understanding interactions among millions of neurons. Ball also speculates that if interpretability nears a solution, those focused on catastrophic alignment risk would be pleased, while those more concerned about power concentration and labor markets might see it as bad news.