A long LessWrong post reflects on the concept of "circuits": the author says neural network computation needs a clearer ontology
Slouching Towards Interpretability: Thinking About Computation in Neural Networks
AI summary
During the Iliad Fellowship, the author wrote an article arguing that neural network computation needs a clearer ontology, and that the part circuit Darwinism treats as an "implementation detail" is where the deep confusion lies. The article contrasts two pictures of circuit origins, "pruning" and "precipitation," and states that predictions change greatly with settings such as "what a circuit is," and that naive guesses do not seem self-consistent. Part three gives an unfinished synthesis of "defining circuits," and the conclusion envisions a "foundational interpretability theory."