Exploring weight-space programming: capabilities, generalization and limitations

The researchers systematically examined weight-space programming (WSP), a method that lets reasoning agents directly inspect and modify neural network parameters, as an alternative to gradient-based model adaptation. Evaluating across seven benchmarks and two model scales, Qwen3.5-0.8B and Qwen3.5-9B, over 16 edit trajectories, the study found that WSP can substantially improve performance through highly localized interventions; on the full held-out evaluation, accuracy on MATH-500, GPQA Diamond, IFEval and SciBench improved by up to 29.45, 4.42, 36.72 and 17.11 percentage points respectively, with modifying only 1,023 parameters yielding a 29.45 percentage point improvement. Cross-task generalization is promising but uneven: checkpoints edited on math improved GPQA Diamond, IFEval and SciBench by 3.03, 12.18 and 20.10 percentage points respectively without further editing, while SciCode scores were unchanged.