Goodfire report says Qwen3.6-35B-A3B neurons can serve as interpretable units
Linkpost: "Neural Networks Are Already Clean" Followup Technical Paper
Goodfire released the technical report "Neurons as an Interpretable Basis," arguing that neurons themselves can serve as interpretable computational units. As a follow-up to Nick Cammarata's widely shared essay, the report uses sparsification, attribution tools and a research agent to analyze Qwen3.6-35B-A3B, claiming that ablating the weakest 92.2% of MoE neuron activations in each forward pass increases language model loss by only 2.9%; that natural-language descriptions allow a judge model to predict with 95% accuracy whether a neuron fires; that simulation recovers 73% of the ablation damage (measured against the recoverable upper bound); and that it can write activation text stronger than any text in a 10B-token corpus. The appendix analyzes Colorado/Tornado neurons and finds that about 98.8% of the causal effect comes from Colorado and other related-location text, and that Tornado is not a hidden secondary function; the report suspects this association stems from Qwen3-32B copying the initialization of Qwen3-14B layers, and considers this overlap benign.