Microsoft and others propose iWSP, LLMs hand-write Transformer weights
Start here
Microsoft Research, together with Johns Hopkins and others, proposed interpretable Weight-Space Programming (iWSP): LLM agents write model weights and features directly in code without backpropagation or gradient descent, propose hypotheses, and manually build and improve human-interpretable computations; LLM audits confirmed the interpretability of these hand-built models. The paper first had iWSP build compact and accurate Transformers on benchmarks such as arithmetic and sentiment classification; according to New Zhiyuan, Claude Opus 5.5 achieved 100% accuracy on hidden test sets across all 13 categories of algorithmic tasks, averaging about 39 iterations, and comparisons within the same family showed that stronger models wrote smaller networks, with Opus 5.5 using only 166 parameters. The method was also applied to brain encoding models: predictions in visual cortex exceeded existing interpretable baselines, data confounds were revealed in language fMRI, and in auditory ECoG the system systematically found interpretable acoustic features that predict neural activity. However, NLP tasks such as sentiment classification were limited, reaching only 72% even with parameters inflated to about 190,000; the paper emphasizes that complex representations still require massive data and gradient descent, and this cannot be taken to mean large models no longer need training.
Generated by AI from reports · Updated 17 hr ago
Progress timeline
- WeChat · AI Era (Xinzhiyuan)SelectedNo training needed! Opus 5.5 directly hand-writes Transformer weights, scoring 100% on all 13 types of algorithm tasks
Microsoft Research, together with Johns Hopkins University and other universities, proposes interpretable weight-space programming, letting Claude Opus 5.5 write Transformer weights directly in code without backpropagation or gradient descent, reaching 100% accuracy on hidden test sets across 13 types of algorithm tasks, with each iteration relying only on feedback from a single validation score, averaging about 39 rounds. Comparison within the same family shows that stronger models write smaller networks: Haiku 5.5 has a median of about 100,000 parameters, Sonnet 5.5 about 41,600, and Opus 5.5 only 166. The method is also applied to brain encoding models, where on language fMRI data it found a dataset confounding flaw, and on visual cortex tasks its predictions exceed those of existing interpretable models. However, it is limited on NLP tasks such as sentiment classification, reaching only 72% even with parameters inflated to about 190,000; the paper emphasizes that complex representations still require massive data and gradient descent, and this should not be taken to mean that large models no longer need training. Paper: arXiv 2611.18114.
- arXivInterpretable weight-space programming builds transparent Transformers and brain encoding models
The authors propose interpretable weight-space programming (iWSP), in which LLM agents iteratively build interpretable models by writing model weights and features directly in code without backpropagation; the agents must rely on prior knowledge and reason about past trials, proposing hypotheses, manually constructing and improving human-interpretable computations, and an LLM audit confirmed the interpretability of these hand-built models. The authors first had iWSP build compact and accurate Transformers on benchmarks such as arithmetic and sentiment classification, then applied it to neural encoding models of vision, language and hearing: on visual cortex it outperformed existing interpretable baselines in prediction, revealed different computations in early and higher visual areas, and generalized to unseen data; in language fMRI it revealed confounders in the data, and in auditory ECoG it systematically found interpretable acoustic features that predict neural activity. The authors say these results establish iWSP as a general paradigm combining high predictive performance with human-interpretable scientific discovery.
Related discussion
- X@arnaudmercierInterpretable weight-space programming builds transparent transformers and brain encoding models https://t.co/k47CfZXopr(opens the original post in a new tab)
- X@thesupermannxMicrosoft just killed backpropagation. They built an AI that writes a neural network’s weights by hand, in code, then…(opens the original post in a new tab)
585 likes·Nov 19 11:45Collect
- X@Yamamoto_NeuroXThis is an extremely groundbreaking research result that solves alignment… I am literally trembling. A joint research team from Microsoft Research, Johns Hopkins University, Columbia University and New York University published the paper "Interpretable w…(opens the original post in a new tab)
427 likes·Nov 19 11:45Collect
- X@S_Van_SeverenAgents are now sufficiently capable of directly programming the weight space of brain-encoding models (🤯!): an LLM writ…(opens the original post in a new tab)
3 likes·Nov 19 11:45Collect
Attention trend
New sources per day
- Nov 18: 3 new sources (0 outlets, 3 accounts)
- Nov 19: 2 new sources (1 outlets, 1 accounts)
- Nov 20: 0 new sources (0 outlets, 0 accounts)