Link post: neural networks are now clearly readable

Yair Halberstadt posted a link on LessWrong, relaying Nick Cammarata's long post on mechanistic interpretability, in which he says neural networks are not as entangled and difficult as he once thought. Cammarata says he became pessimistic after early mechanistic interpretability work at OpenAI, but later launched OpenAI's first large-scale automated interpretability project, finding only about 2% of neurons readable with the then-primitive tools; with modern agent tools and more careful observation, the vast majority of neurons in models up to trillion-parameter scale are readable, and many famous cases of entangled neurons are actually misunderstandings. He believes ambitious interpretability will be solved when it is most needed, possibly even soon. Roon quoted and shared it, calling it a potentially important moment in the history of science, while the post author asks how much this advances known results and what use it is for alignment practice.