Neel Nanda says Astra's weight editing is more like old-school mechanistic interpretability research than some novel capability
Researcher Rikiya Takehi and others proposed Weight-Space Programming, letting GPT-6 Astra directly rewrite the weights of Qwen3.5-0.8B; by changing only 0.016% of the weights, without gradients or training, they raised MATH-500 from 7.9% to 48.1%, with math scores improving by up to 6 times and instruction following improving by 3 times. Neel Nanda commented that this looks more like Astra being unexpectedly good at old-school mechanistic interpretability research than some novel capability; he noted that most of these cases use known mechanistic interpretability techniques and rely on clean surface structure in the data, and that this kind of analysis gradually fell out of favor after 2022 because it is hard to scale to the complexity of real LLMs. He said this updates expectations for LLMs' ability to do this kind of analysis, but does not update the judgment of how far old-school mechanistic interpretability itself can go.
This is a cool paper, but I think it's best thought of as Astra is surprisingly capable of doing old school mechanistic interpretability research rather than anything spooky. The things it's doing are clever, but typically exploiting structure in the data and using known method mechanistic interpretability techniques (which Astra was surely pretrained on), though it is better at this than I would have predicted!
This is an exciting development for the prospects of ambitiously scaling interpretability or doing very laborious analysis like this, on more real models. However, I'm not holding my breath. This kind of analysis was pretty common up until about 2022, it largely fell out of favor because a lot of it is just exploiting fairly clean surface-level patterns that doesn't scale to the annoying complexity of real LLMs or which only works for the more simpler and superficial properties. And as far as I can tell, basically all the case studies here are just using a bag of interpretability tricks rather than anything spooky. The piece doesn't really update me on how far old school mech interp can go, just on how good LLMs are at it. It's very fun to see the creative solutions Astra came up with though!
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: https://rikiyat.github.io/blog/weight-space-programming/View the quoted post on X