Neel Nanda 称 Astra 的权重编辑更像旧派机制可解释性研究，而非什么新奇能力
研究者 Rikiya Takehi 等人提出 Weight-Space Programming，让 GPT-6 Astra 直接改写 Qwen3.5-0.8B 的权重，仅改动 0.016% 的权重、不经梯度和训练，就把 MATH-500 从 7.9% 提升到 48.1%，数学成绩最高提升 6 倍，指令跟随提升 3 倍。Neel Nanda 评论认为，这件事更像是 Astra 出人意料地擅长旧派机制可解释性研究，而不是什么新奇能力；他指出这些案例大多使用已知的机制可解释性技巧，并依赖数据中干净的表层结构，这类分析在 2022 年后逐渐失宠，因为很难扩展到真实 LLM 的复杂性上。他认为这让人更新了对 LLM 做这类分析的能力预期，却没有更新旧派机制可解释性本身能走多远的判断。
This is a cool paper, but I think it's best thought of as Astra is surprisingly capable of doing old school mechanistic interpretability research rather than anything spooky. The things it's doing are clever, but typically exploiting structure in the data and using known method mechanistic interpretability techniques (which Astra was surely pretrained on), though it is better at this than I would have predicted!
This is an exciting development for the prospects of ambitiously scaling interpretability or doing very laborious analysis like this, on more real models. However, I'm not holding my breath. This kind of analysis was pretty common up until about 2022, it largely fell out of favor because a lot of it is just exploiting fairly clean surface-level patterns that doesn't scale to the annoying complexity of real LLMs or which only works for the more simpler and superficial properties. And as far as I can tell, basically all the case studies here are just using a bag of interpretability tricks rather than anything spooky. The piece doesn't really update me on how far old school mech interp can go, just on how good LLMs are at it. It's very fun to see the creative solutions Astra came up with though!
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: https://rikiyat.github.io/blog/weight-space-programming/在 X 查看被引用的帖子