Sakana AI's self-improving model MASS has self-blind spots

Researchers at Sakana AI and UC Berkeley proposed a multi-agent self-supervision method called MASS, in which the same language model simultaneously takes on three roles—solving problems, optimizing workflows, and judging results—to achieve recursive self-improvement without a stronger teacher model. The base model is Qwen3.6-27B, and the tasks are 12 financial, robotics, and pharmaceutical research tasks generated by GPT-5.5. On the test tasks, the win rate against the base model rose from 53.9% after one round to 69.9% after two rounds; on MLR-Bench, DSBench, ScienceAgentBench, and AstaBench, the score per output token was 1.2 to 1.6 times that of the base model, but on Terminal-Bench 2.0 and SWE-bench Verified the scores dropped slightly; the agreement rate between the self-evaluator and external judges rose from 73% to 93%. The code has been released.

Sakana AI proposes the MASS method, having the same model serve as task executor, workflow optimizer, and evaluator simultaneously, using multi-agent self-supervision to provide training signals for recursive self-improvement. The inner loop uses evaluator feedback to search for better team workflows; the outer loop runs selected workflows with multiple random seeds, then fine-tunes the top-ranked trajectories back into shared model weights via LoRA. Starting from Qwen3.6-27B and running two rounds, external judge win rate against the base model on three held-out synthetic research tasks increased from 53.9% to 69.9%; on MLR-Bench, DSBench, ScienceAgentBench, and AstaBench, score per output token reached 1.2–1.6 times the base, but on Terminal-Bench 2.0 and SWE-bench Verified it was 0.96 times and 0.94 times, respectively.