Skip to content
EventUpdating

Microsoft and others propose iWSP, LLMs hand-write Transformer weights

2 reports2 reporting sourcesLatest report

Start here

AI overview

Microsoft Research, together with Johns Hopkins and others, proposed interpretable Weight-Space Programming (iWSP): LLM agents write model weights and features directly in code without backpropagation or gradient descent, propose hypotheses, and manually build and improve human-interpretable computations; LLM audits confirmed the interpretability of these hand-built models. The paper first had iWSP build compact and accurate Transformers on benchmarks such as arithmetic and sentiment classification; according to New Zhiyuan, Claude Opus 5.5 achieved 100% accuracy on hidden test sets across all 13 categories of algorithmic tasks, averaging about 39 iterations, and comparisons within the same family showed that stronger models wrote smaller networks, with Opus 5.5 using only 166 parameters. The method was also applied to brain encoding models: predictions in visual cortex exceeded existing interpretable baselines, data confounds were revealed in language fMRI, and in auditory ECoG the system systematically found interpretable acoustic features that predict neural activity. However, NLP tasks such as sentiment classification were limited, reaching only 72% even with parameters inflated to about 190,000; the paper emphasizes that complex representations still require massive data and gradient descent, and this cannot be taken to mean large models no longer need training.

Generated by AI from reports · Updated 17 hr ago

Progress timeline

Nov 19
  1. WeChat · AI Era (Xinzhiyuan)Selected
    No training needed! Opus 5.5 directly hand-writes Transformer weights, scoring 100% on all 13 types of algorithm tasks

    Microsoft Research, together with Johns Hopkins University and other universities, proposes interpretable weight-space programming, letting Claude Opus 5.5 write Transformer weights directly in code without backpropagation or gradient descent, reaching 100% accuracy on hidden test sets across 13 types of algorithm tasks, with each iteration relying only on feedback from a single validation score, averaging about 39 rounds. Comparison within the same family shows that stronger models write smaller networks: Haiku 5.5 has a median of about 100,000 parameters, Sonnet 5.5 about 41,600, and Opus 5.5 only 166. The method is also applied to brain encoding models, where on language fMRI data it found a dataset confounding flaw, and on visual cortex tasks its predictions exceed those of existing interpretable models. However, it is limited on NLP tasks such as sentiment classification, reaching only 72% even with parameters inflated to about 190,000; the paper emphasizes that complex representations still require massive data and gradient descent, and this should not be taken to mean that large models no longer need training. Paper: arXiv 2611.18114.

Nov 13
  1. arXiv
    Interpretable weight-space programming builds transparent Transformers and brain encoding models

    The authors propose interpretable weight-space programming (iWSP), in which LLM agents iteratively build interpretable models by writing model weights and features directly in code without backpropagation; the agents must rely on prior knowledge and reason about past trials, proposing hypotheses, manually constructing and improving human-interpretable computations, and an LLM audit confirmed the interpretability of these hand-built models. The authors first had iWSP build compact and accurate Transformers on benchmarks such as arithmetic and sentiment classification, then applied it to neural encoding models of vision, language and hearing: on visual cortex it outperformed existing interpretable baselines in prediction, revealed different computations in early and higher visual areas, and generalized to unseen data; in language fMRI it revealed confounders in the data, and in auditory ECoG it systematically found interpretable acoustic features that predict neural activity. The authors say these results establish iWSP as a general paradigm combining high predictive performance with human-interpretable scientific discovery.

Related discussion

Total 4 items

Attention trend

New sources per day

5 in 3 days·most 3 (Nov 18)·today 0

  • Nov 18: 3 new sources (0 outlets, 3 accounts)
  • Nov 19: 2 new sources (1 outlets, 1 accounts)
  • Nov 20: 0 new sources (0 outlets, 0 accounts)

Hourly attention

Current attention 42·Peak in the comparable period 46 (Nov 19 11:00)·Change over the comparable past 24 hours –

0204060Nov 1911:00Nov 1916:00Nov 1921:00Nov 2002:00