MIT team has GPT-6 Astra directly rewrite Qwen weights
Start here
The MIT CSAIL research team announced experimental results called "weight space programming": GPT-6 Astra directly reads and modifies the weights of Qwen3.5-0.8B through natural language instructions, changing only about 0.016% of parameters (131,072 weights), without retraining or using gradient descent, and the MATH-500 score rose from 7.9% to 48.1%. According to TechFlow, the entire process took 20.3 hours, including 158 rounds of dialogue, 80 Python executions and 50 model evaluations; GSM8K rose from 13.8% to 67.1%, and IFEval improved from around 21%. QbitAI reported that instruction-following capability increased 3-fold, and the proportion of MATH-500 hitting the 32768 token length limit dropped from 93.0% to around 33%. India Today reported that the experiment was documented in a research blog post titled "LLMs can natively read & write neural network weights." Researcher Neel Nanda commented that this looks more like Astra being unexpectedly good at old-school mechanistic interpretability research rather than any novel capability. Another poster said that Qwen's improvement mainly came from fixing reasoning infinite loops, and that the research team also observed basically no improvement on Gemma 3 270M, which has no obvious infinite loops.
Generated by AI from reports · Updated 45 min ago
Latest developments
Progress timeline
- X@ZJM330· DiscussionIn Chinese, as written没跑一步梯度,小模型数学分 7.9% 变 48.1%。 MIT 让 GPT-6 Astra 直接读 Qwen3.5-0.8B 的权重,找出导致推理死循环的部分,只改 0.016%。 我认为以后调模型可能先让大模型读权重。但只在 0.8…(opens the original post in a new tab)
- X@ScarletKc· DiscussionIn Chinese, as writtenMIT CSAIL 的 Omar Khattab 团队刚发了一篇很有意思的博客,提出了一个叫 Weight Space Programming(权重空间编程)的概念:现在的前沿大模型已经可以不靠梯度下降,直接用自然语言推理去读写和修改神经…(opens the original post in a new tab)
- X@erguoyi· DiscussionIn Chinese, as writtenMIT CSAIL 最近做了个挺夸张的实验:让 GPT-6 Astra 不用梯度下降,也不重新训练,直接动另一个神经网络的原始权重。 拿 Qwen3.5-0.8B 开刀,只改了约 13 万个权重,占整个模型的 0.016%,MATH-5…(opens the original post in a new tab)
- XX @haider1· Discussionwell that's a big deal(opens the original post in a new tab)
- X @MinLiBuildsMIT CSAIL research says AI directly modifies Qwen3.5-0.8B weights, MATH-500 rises from 7.9% to 48.1%
MIT CSAIL research is said to have an AI directly modify about 0.016% of Qwen3.5-0.8B's weights, raising MATH-500 from 7.9% to 48.1%, about 6 times, without training or LoRA. The poster says Qwen's improvement mainly comes from fixing a reasoning infinite loop; the research team also observed essentially no improvement on Gemma 3 270M, which has no obvious infinite loop. The poster also says their similar weight-editing attempt on a person-recognition network showed no effect.
- India TodayCan AI repair another AI's brain? MIT researchers use GPT-6 Astra to improve the math ability of Qwen3.5-0.8B
MIT CSAIL researchers had OpenAI's GPT-6 Astra directly read and write the weights of Alibaba's Qwen3.5-0.8B through a Python environment, fixing, without retraining or gradients, the problem of the model falling into repetitive reasoning, sometimes up to the 32768 token limit. According to the paper, Astra changed only 131072 weights, 0.016% of the total 800 million weights; Qwen's score on MATH-500 rose from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval from 21% to 62.3%. The researchers say Astra did not teach Qwen new mathematical concepts, but removed a fault that prevented it from using its existing abilities. This technique, called Weight-Space Programming, still has limitations: Astra cannot tell from weights alone which of the two models was trained to cater to users, and needs attention information. The team has released the experimental logs and is developing related benchmarks.
- WebSynced (Jiqizhixin)· Media coverageMIT lets GPT-6 Astra directly modify Qwen weights! Math scores surge(opens the original post in a new tab)
- TechFlowSelectedGPT-6 Astra directly rewrites Qwen3.5-0.8B weights, MATH-500 score rises from 7.9% to 48.1%
The MIT CSAIL research team published the results of an experiment called "weight space programming": GPT-6 Astra directly modified 131,072 weights in Qwen3.5-0.8B via natural language instructions, accounting for 0.016% of all parameters, and the model's MATH-500 score rose from 7.9% to 48.1% as a result. The whole process took 20.3 hours, including 158 rounds of dialogue, 80 Python executions and 50 model evaluations; GSM8K rose from 13.8% to 67.1%, IFEval rose from 21.0% to 62.3%, and the proportion of responses that hit the 32,768 token limit because they fell into repetitive loops fell from 93% to 33.1%. The team said Astra did not inject new mathematical ability, but only removed obstacles preventing the model from using abilities it already had; when the same experiment was repeated on Gemma 3 270M, the MATH-500 score fluctuated by no more than 1 percentage point.
- WeChat · QbitAISelectedMIT: GPT-6 Astra can directly modify Qwen weights, math capability surges 6-fold
MIT experiments show that GPT-6 Astra, without gradient training, changed only 0.016% of the weights in Qwen3.5-0.8B and raised its MATH-500 score from 7.9% to 48.1%, while instruction-following ability also improved 3-fold. The team calls this method Weight-Space Programming: a large model analyzes neural network weights, locates problems, and modifies them as required—for example, reducing the proportion of MATH-500 cases hitting the 32768-token length limit from 93.0% to 33.1%. Astra also identified, by analyzing weights, the games and languages a small network had learned, found induction heads, and in a small-car hill-climbing task changed 3.5% of the weights so that all 1024 tests succeeded. In MinAtar experiments, networks whose weights it wrote from scratch scored 517.4 on average, about 20 times the 25.3 of a PPO baseline trained for 500 million steps. The article also notes that Astra's capabilities still come from its prior training, and that analyzing weights and evaluation also consume computational resources.
- WeChat · Synced (Jiqizhixin)SelectedThe MIT CSAIL team had GPT-6 Astra directly modify the weights of Qwen3.5-0.8B, raising the MATH-500 score from 7.9% to 48.1%
The MIT CSAIL team had GPT-6 Astra directly read and modify the weights of Qwen3.5-0.8B through natural language, and by changing only about 0.016% of the parameters, it raised the MATH-500 score from 7.9% to 48.1%. Within a limited number of forward computations, Astra located a loop direction written into the MLPs of layers 15 and 19, subtracted that direction from each of the 64 output columns of these two layers, and modified a total of 131,072 weights, without retraining or using gradient descent. On Qwen3.5-0.8B, GSM8K rose from 13.8% to 67.1%, IFEval rose from 21.0% to 62.3%, and the proportion hitting the 32,768 Token limit fell from 93% to 33%. Blog address: https://rikiyat.github.io/blog/weight-space-programming/
- X @NeelNanda5Neel Nanda says Astra's weight editing is more like old-school mechanistic interpretability research than some novel capability
Researcher Rikiya Takehi and others proposed Weight-Space Programming, letting GPT-6 Astra directly rewrite the weights of Qwen3.5-0.8B; by changing only 0.016% of the weights, without gradients or training, they raised MATH-500 from 7.9% to 48.1%, with math scores improving by up to 6 times and instruction following improving by 3 times. Neel Nanda commented that this looks more like Astra being unexpectedly good at old-school mechanistic interpretability research than some novel capability; he noted that most of these cases use known mechanistic interpretability techniques and rely on clean surface structure in the data, and that this kind of analysis gradually fell out of favor after 2022 because it is hard to scale to the complexity of real LLMs. He said this updates expectations for LLMs' ability to do this kind of analysis, but does not update the judgment of how far old-school mechanistic interpretability itself can go.
- RedditReddit r/LocalLLaMA· DiscussionGPT-6 Astra Edits Qwen Weights: MATH-500 7.9% to 48.1%(opens the original post in a new tab)
- XX @Thom_Wolf· Discussionwhat?(opens the original post in a new tab)
- RedditReddit r/singularity· DiscussionSparks of Weight-Space Programming: LLMs can natively read & write neural network weights(opens the original post in a new tab)
- XX @AndrewCurran_· DiscussionFrom the blog:(opens the original post in a new tab)
Related discussion
What people think
By account: 13 hold one of the views above, 18 posted without a clear one, and 35 more only relayed the news. AI drew the views from the posts; a post is placed only when three models, judging apart, all agree. The posts themselves are the record.
- X@jialiwanaiWait... AI can now edit another AI's brain? 🤯 MIT researchers asked GPT-6 Astra to directly modify Qwen3.5-0.8B's neur…(opens the original post in a new tab)
- X@mtsmysMIT CSAILのWeight-Space Programmingは言ってみればコンパイラーのビルドと同じなので,既に通ってきた道ではある.実現している事は面白いけれど.と同時に,この方向での研究が進むと学習環境が不要になるので,環境を…(opens the original post in a new tab)
- X@yonifraלפי חוקרי MIT, GPT-6 Astra שינה ישירות 0.016% ממשקלי מודל מתמטיקה קטן, בלי אימון או גרדיאנטים. ציון MATH-500 עלה מ-7.9%…(opens the original post in a new tab)
- X@justnadeem7MIT researchers used GPT-6 Astra to edit just 0.016% of a Qwen model’s weights. Its math-test score jumped from 7.9% t…(opens the original post in a new tab)
- X@ZJM330In Chinese, as written没跑一步梯度,小模型数学分 7.9% 变 48.1%。 MIT 让 GPT-6 Astra 直接读 Qwen3.5-0.8B 的权重,找出导致推理死循环的部分,只改 0.016%。 我认为以后调模型可能先让大模型读权重。但只在 0.8…(opens the original post in a new tab)
- X@umiyuki_aiMITのRikiyaTakehi氏らによるヤバ研究。Qwen3.5-0.8Bに数学問題を解かせると同じ思考を延々とループして一向に答えが出ないバグ的な挙動があった。そこで研究者はAstraに「Qwenを修正してくれない?」って丸投げしてみ…(opens the original post in a new tab)
40 likes·Nov 3 16:10Collect
- X@goyashySource: "Sparks of Weight-Space Programming" by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (MIT CSAIL): ht…(opens the original post in a new tab)
- X@ScarletKcIn Chinese, as writtenMIT CSAIL 的 Omar Khattab 团队刚发了一篇很有意思的博客,提出了一个叫 Weight Space Programming(权重空间编程)的概念:现在的前沿大模型已经可以不靠梯度下降,直接用自然语言推理去读写和修改神经…(opens the original post in a new tab)
2 likes·Nov 3 16:10Collect
- X@goyashyGPT-6 Astra did not teach Qwen math. It stopped Qwen looping. MIT CSAIL researchers had Astra edit 0.016% of Qwen3.5-0…(opens the original post in a new tab)
- X@FegaPierreUn modèle d'IA a corrigé un autre modèle en modifiant directement ses poids, sans aucun entraînement. C'est ce que décr…(opens the original post in a new tab)
- X@erguoyiIn Chinese, as writtenMIT CSAIL 最近做了个挺夸张的实验:让 GPT-6 Astra 不用梯度下降,也不重新训练,直接动另一个神经网络的原始权重。 拿 Qwen3.5-0.8B 开刀,只改了约 13 万个权重,占整个模型的 0.016%,MATH-5…(opens the original post in a new tab)
- X@blackright0On November 2, 2026, Rikiya Takehi, Jayden Teoh, Alex Zhang, Omar Khattab and others at MIT CSAIL announced an approach called "Weight-Space Programming." OpenA…(opens the original post in a new tab)
4 likes·Nov 3 16:10Collect
Attention trend
New sources per day
- Nov 3: 98 new sources (10 outlets, 88 accounts)