MASS: Recursive Self-Improvement through Multi-Agent Self-Supervision

On open research tasks that are difficult to externally verify, the improved model is also the best available supervisor, and homogeneous self-feedback may reinforce errors.

• Scenario: The authors state that frontier systems are already involved in original scientific results, and to maintain scale and pace, the natural direction is recursive self-improvement (RSI), driven by evaluation and improvement. The difficulty of evaluation is that the work exceeds the evaluation capacity of available human experts (Burns et al., 2023), or feedback is too slow; the authors give the example that in nuclear fusion, a single tokamak plasma study may require more than 120 million CPU hours. Improvement relies on high-quality execution data, and manually revising task decomposition, tool use, and verification is, according to the authors, prohibitively expensive across tasks and domains.
• Existing shortcomings: Having the model serve as evaluator can alleviate the evaluation side, but the central difficulty is that the improved model is also the best available supervisor. The authors state that generation, evaluation, and revision share the same learning prior, and self-feedback may reinforce rather than correct errors (Huang et al., 2024); a single instance also struggles to critique and improve its own complex reasoning.
• Hypothesis: The authors borrow from Minsky's Society of Mind, using multiple copies of the same model with division of labor, then distilling collective experience into shared weights. Two assumptions are: optimizing what agents communicate, to whom, and when—not just increasing the number of agents—can provide self-supervision different from strengthening the reasoning chain; jointly learning orchestration and bounded sub-agent execution may be more efficient than learning from a single long trajectory interleaving two levels of abstraction. The latter is written by the authors as a conjecture, inspired by hierarchical imitation learning.
• Method: The authors propose Multi-Agent Self-Supervision (MASS). The same base model first evolutionarily optimizes the multi-agent workflow under fixed weights, then performs supervised fine-tuning on orchestrator and sub-agent trajectories; the updated model again serves as executor, optimizer, and evaluator.

Section 6 is divided into self-improvement/workflow optimization, and model-workflow co-evolution; Appendix J adds positioning with mechanism comparisons.

Self-improvement and self-supervised training

• Schmidhuber: 1993 self-referential weight modification, 1997 reward-based policy change, and 2007 Gödel machine with proof-gated self-rewriting. The authors state MASS uses empirical workflow search and SFT, without formal guarantees.
• Self-Rewarding LMs (Yuan et al., 2024), SiriuS (Zhao et al., 2026), Multi-Agent Evolve (Chen et al., 2025a): respectively train on self-scored responses, train on filtered multi-agent trajectories, and let a shared model ask questions, solve, and judge. Appendix J states Self-Rewarding has no workflow search; Multi-Agent Evolve's proposer generates questions, not execution workflows.
• Expert Iteration, STaR, ReSTEM: Appendix J writes the precedent of search-then-distill as policy learning, learning from reasoning that reaches known answers, and alternating generation, reward filtering, and fine-tuning. The authors write MASS as applying this line to optimized multi-agent research workflows, with orchestrator and worker sharing weights.

Workflows and communication structures

• ADAS, AFlow, GPTSwarm (Hu et al., 2025; Zhang et al., 2025; Zhuge et al., 2024): search agent programs, workflow graphs, or prompts and communication edges. Appendix J states these works establish automatic design of agentic computation; MASS additionally learns shared weights from execution trajectories.
• RHI (Lee et al., 2026): revises workflow prompts with pairwise comparisons. The authors state RHI provides the workflow search method and information flow motivation for this paper, and MASS adds trajectory-based post-training.
• TTHE (Nie et al., 2026) and Park et al., 2026: TTHE uses a frozen backbone for execution, harness proposal, and judgment; Park et al. let communicating agents share discoveries at inference time. Appendix J states TTHE embodies roles within a homogeneous loop, but weights remain fixed.

Model-workflow co-evolution

• SIA (Hebbar et al., 2026): a separate feedback model modifies the scaffold and initiates weight updates, relying on task verifiers.
• HASE (Luo et al., 2026): a single policy uses reinforcement learning to simultaneously learn task solutions and harness edits, with training signals from task rewards.
• HELIX (Fan and Huang, 2026): evolves harnesses and exports verified training records. Appendix J states its reported experiments stop at data generation, without weight updates.

Baselines and benchmarks

• Baselines: S, S+, S++ are fine-tuned from L(0) on successful single-agent trajectories, excluding rounds with sub-agent calls; M, X, M+ use multi-agent or mixed teachers. Public evaluation controls are L(0) under the same qwen-code scaffold.
• Benchmarks: private synthetic research programming tasks, plus MLR-Bench, ScienceAgentBench, DSBench, AstaBench, Terminal-Bench 2.0, SWE-bench Verified.

Appendix J positions this paper as: searching multi-agent workflows, fine-tuning filtered orchestrator and worker dialogues into shared weights, then having the student serve the three roles. The homogeneous condition refers to execution, evaluation, and optimization within the loop; role ablations change this allocation.

MASS lets the current model L(g) first rewrite how its own copies collaborate, then updates weights from collaboration trajectories; in the next cycle all three roles switch to L(g+1). Experiments run for two cycles, i.e., L(0) to L(1) to L(2).

Problem setup

• Tasks and roles: T is a finite task set; the training subset and test set are disjoint. The coding agent A's tools and memory text are fixed in full; the scaffold is qwen-code 0.20.0. One execution yields trajectory τ_t and deliverable workspace y_t. In the homogeneous setting, L_agent, L_eval, and L_opt all equal the current L(g).
• Objective: Increase the probability, judged by an external strong model L_ext, that the current generation's output is better than outputs from previous generations. L_ext's judgments do not enter Algorithm 1 and are not an SFT objective.
• Workflow: A workflow is a text specification appended to the task prompt. For task t, w_t={h_t,(r_t,j,u_t,j,c_t,j)_j=1^m_t. h_t is the call order, r is the role, u is the instruction, c is the output contract returned to the orchestrator, and m_t is the number of sub-agents. The orchestrator and all sub-agents share L_agent.

Inner loop: workflow optimization

• Elite retention: Each training task independently maintains a workflow sequence and the best-so-far w_t*. L_opt proposes and executes the next workflow based on the current best, feedback history, and optimization prompt. L_eval compares the current output with the best output pairwise; the elite is replaced only if the current one is better. The authors call this evolution with an elite size of 1. Unselected proposals are still executed, so subsequent execution quality need not be monotonic.
• Search bias: The optimization prompt explicitly emphasizes communication structure, i.e., call order and output contracts, above roles and instructions. The retained workflow is the best under sequential pairwise self-evaluation within the loop; the authors state it is not a certified global optimum. Tasks that do not exceed the reference workflow do not enter teacher collection.
• Prompt removed from training input: The authors state that writing the multi-agent workflow into the prompt is one way to make the model act according to organizational instructions. Teacher collection additionally attaches execution instructions requiring actual calls to workers via agent tools; Appendix B.3 gives the full text. When constructing data, this instruction and w_t* are removed from the orchestrator's initial input, while the assignments received by workers are retained.

Outer loop: supervised fine-tuning

• Filtering: Each training task is executed independently M=18 times with the retained workflow. L_eval compares workspaces within a task, and a Bradley–Terry model converts pairwise judgments into scores. The first K−1 trajectories are used for training and the K-th for validation; experiments use K=16. After one trajectory is excluded due to early scaffold termination, 8 tasks yield 119 training trajectories, i.e., (9−1)×15−1.
• Window and sampling: The window is at most L_max=49,152 tokens; up to 8,192 tokens of preceding context may be retained but masked and excluded from loss. Loss is only on assistant reasoning, text, and tool calls. The main text uses a fixed 2:1 sampling of sub-agents to orchestrator; Appendix B.2 writes it as orchestrator window probability 1/3, with worker and auxiliary windows totaling 2/3, with replacement.
• Optimizer: LoRA is newly initialized on the BF16 parent model, rank 64, scaling factor 128, dropout 0.05. AdamW with beta1=0.9, beta2=0.95, peak learning rate 3×10^-5, 5-step warmup, cosine decay to 10% of peak. First cycle: 1,624 steps, 3 windows per step. Checkpoints are selected by lowest validation loss; for the first cycle this is step 1200; external workspace judgments do not participate. The second cycle reinitializes LoRA on the merged L(1), with checkpoints still selected by validation loss; the paper does not give the step count for this cycle.

External success judgment

• Win rate: The reported L_ext is GPT-5.5 and Claude Opus 4.8, both at maximum reasoning effort, seeds 42, 43, 44, with 6 judgments per workspace pair. Win rate is ŝ(a,b)=W_a,b/(W_a,b+L_a,b), with ties excluded. The authors state this is repeated evaluation of the same output pair and does not infer statistical significance from the number of judgments.
• Separate from in-loop signals: Workflow search and teacher ranking use the current model; checkpoints use validation loss. Self-evaluation agreement rates are computed after self-evaluation cannot see external judgments. The teacher composition comparison in Appendix D switches to external judgments for checkpoint selection.

Experimental Setup

• Model under test: L(0) is Qwen3.6-27B-FP8; trained with a BF16 parent model plus LoRA, merging adapters and quantizing to block-FP8 before inference. L(1) and L(2) are the same base after one and two MASS cycles. The scaffold is qwen-code 0.20.0 in all cases.
• Data: GPT-5.5 generated 12 private long-horizon research programming tasks, 4 each in finance, robotics, and pharmaceuticals. 9 were designated for training; task 221 was excluded because the search did not exceed the reference, leaving 8 for SFT. The other 3 synthetic tasks were held out as test tasks, 1 per domain.
• Public benchmark scale: Table 7 is ScienceAgentBench 102×3, MLR-Bench 12×5, AstaBench E2E-Bench-Hard 40×3, DSBench 74×3, Terminal-Bench 2.0 89×3, SWE-bench Verified 500×3. For the last two, each trial kept only tasks with verified results for all three models, with denominators 86/88/87 and 499/499/496, respectively. MLR-Bench scores come from the 12-task subset, not the full 107 tasks listed in Section 4.1.
• Baselines: L(0) under bare prompting; workflow transfer hands workflows from different generations to a fixed executor. S, S+, S++, M, X, and M+ are each fine-tuned independently from L(0).
• Metrics and evaluation: Synthetic tasks use external win rate. Public metrics are success rate, MLR Overall (0–10, 0 for missing papers), checklist satisfaction proportion, normalized score (negatives truncated to 0), and pass rate, respectively. Synthetic tasks and MLR are judged by GPT-5.5 and Claude Opus 4.8, 3 seeds each; the AstaBench official rubric judge is Claude Sonnet 4.6 at temperature 0. The remaining public benchmarks use official execution or tests, not LLM win rates.
• Repeats: Public benchmarks use 3 trials except MLR, which uses 5. The ± in the table is the sample standard deviation of trial means. The synthetic main comparison is 10 bare-prompt rollouts per task. Public benchmark sampling is temperature 0.6, top-p 0.95, top-k 20.

Main Results

The table below follows Table 7 and is consistent with Figure 3b. These are scores, not per-token scores.

Swipe horizontally to read the table

Benchmark and metric	L(0)	L(1)	L(2)
ScienceAgentBench success rate (%)	28.8±1.5	30.4±2.0	31.7±2.0
MLR-Bench Overall (0–10)	1.58±0.38	1.80±0.30	2.40±0.28
AstaBench rubric	0.053±0.002	0.055±0.004	0.067±0.017
DSBench normalized score	0.473±0.029	0.465±0.011	0.482±0.025
Terminal-Bench 2.0 pass rate (%)	37.5±2.2	35.3±2.1	35.2±1.7
SWE-bench Verified pass rate (%)	62.6±1.4	61.8±1.9	61.1±0.6

• Per token: Figure 2 labels L(2) vs. L(0) per output token score ratios: MLR-Bench 1.60×, DSBench 1.35×, ScienceAgentBench 1.20×, AstaBench 1.28×; Terminal-Bench 2.0 0.96×, SWE-bench Verified 0.94×. The authors call the four research benchmarks 1.2–1.6×.
• Synthetic win rates: In Figure 3b, on the three test tasks, L(1) and L(2) vs. L(0) are 53.9% and 69.9%, and L(2) vs. L(1) is 60.8%; on the eight training tasks, they are 65.6%, 82.5%, and 68.1%.
• Authors' interpretation: The authors say post-training uses only solution trajectories, but claim optimization and evaluation also change. DSBench L(1) is 0.465, below L(0)'s 0.473; pass rates on the two coding benchmarks do not rise by L(2). Section 4.2 also writes ScienceAgentBench from 28.7% to 31.8%, inconsistent with Table 7; here the table is authoritative.

Role Transfer and Completion Rates

• Evaluation and workflows: Agreement between self-evaluation and external judgment is 0.73, 0.91, 0.93 (Figure 3c). Table 1 averages per-round win rates over i∈[11] and 12 tasks: L(0) executing L(1) workflows vs. L(0) executing L(0) workflows is 0.64; L(1) executing its own-generation workflows vs. L(0) executing its own-generation workflows is 0.80; L(0) executing L(1) workflows vs. bare L(0) is 0.63, and vs. L(1) executing the same batch of workflows is 0.29.
• Authors' interpretation: The authors say the 0.64 fixed-executor comparison better supports the optimizer improving after post-training. Figure 3a's text says the post-trained model discovers successful workflows earlier; the main text does not give the endpoint task count for that figure.
• Completion rates: MLR submissions are 31/60, 36/60, 50/60, with submitted Overall means of 3.07, 3.00, 2.88. AstaBench submissions are 60/120, 69/120, 91/120, with submitted means of 0.105, 0.095, 0.088. The authors say these conditional means are not quality changes on the same completion set; the aggregate score rise accompanies higher completion rates.

Information Flow and Training Tokens

• Information flow: On retained workflows from the full-base search, at i∈{3,6,9,11}, estimated with text-embedding-3-large and all-mpnet-base-v2. Figure 4's caption says by i=11, task information for role and instruction decreases, while contract and hop increase, across all four configurations. Figure 5's caption says unconditional total correlation decreases for all four; the task-centered version decreases for three, while permutation-adjusted MPNet increases. Absolute estimates are in Appendix Tables 12–13. The authors' Claim 1 is: task-specific information increasingly enters communication content and routing, and redundancy among components decreases.
• Tokens: Figure 6, on four shared training tasks plus test task 60, shows win rates vs. L(0) relative to processed supervised tokens: S is 1.82M, 44.0%; S+ is 9.11M, 59.0%; S++ is 42M, 64.0%; M+ is 28.82M, 68.3%. The authors say M+ beats S++ by 4.3 percentage points using 31.4% fewer tokens, and say M and M+ exceed the single-agent piecewise-linear reference by 8.3 and 6.3 percentage points.
• Authors' interpretation: Claim 2 is that learning coordination from orchestrator trajectories and bounded execution from subagent trajectories may provide more effective RSI supervision than learning both from a single agent's long trajectory. Appendix D states that M+'s sampling, data coverage, and checkpoint selection do not fully match the controls.

Search Bottlenecks and Behavior

• Bottlenecks: Figure 7 fixes the executor at L(0), varies the evaluator, optimizer, and history, and iterates to i∈[20]. The authors' main claim is that optimizer capacity, not evaluator capacity, becomes the bottleneck on RSI speed. Replacing only the evaluator with GPT-5.5 improves less than replacing both evaluator and optimizer; full L(0) ends with 6/6 unanimous wins on 11 of 12 tasks. The authors say the evidence concerns speed, not that weak models cannot discover useful workflows.
• Relapse: In Figure 7b, improved reachability does not guarantee stability; strong iterations are followed by relapse. The authors say the Spearman correlation between text edit magnitude and judgment change is weak, |ρ_edit|≤0.12; near-unanimous results flip under both small and large edits. Appendix E records two byte-identical workflows and prompts on task 53, one 6–0–0 and one 0–6–0 due to loop detection.
• Bare prompting: In Table 2a, delegation goes from 1/110 to 48/110 and 87/110, and code handoff from 1/110 to 42/110 and 83/110. Statistics do not provide optimized workflows. Of 114 audited teacher trajectories, 98 have code handoff and 57 have subagent revisions, with 79 and 37 sampled (Table 2b). The authors believe post-training internalizes coordination behavior; Appendix G states the base can already do these behaviors under general instructions, and the change under bare prompting is more frequent use.

Other Ablations and Analyses

• Under general instructions, M+ vs. base is 352 wins, 306 losses, 2 ties, a 53.5% share (Appendix C.4).
• In the first cycle, M+'s share vs. the ranking teacher is 17.3% (112–536–0, 108 pairs); on task 205 vs. bare base it is 20.0% (Table 5).
• Teacher output preference: multi-agent vs. single-agent is 85.1% (206 pairs); on 10 pairs of similar length it is 80.0% (32–8, Appendix D.3).
• MLR-Bench: Welch p=0.0053 for L(2) vs. L(0), Fisher p=0.0004 for submissions; paired sign test on the 12-task means gives p=0.2266. The authors say the evidence depends on the aggregation unit.
• None of the 180 MLR-Bench runs call the delegation tool; the authors say this cannot establish delegation as the mechanism for public benchmark transfer.
• ScienceAgentBench reference program local check success rate is 80.4% (Appendix C.5).

Limitations and Future Directions Noted by the Authors

• Speeding up gains: Section 7 states that speeding up these performance gains within each cycle remains an important direction for future work.
• No step-by-step guarantee: Appendix J states the recursive loop is empirical; unlike the Gödel machine's proof-based self-rewriting, it does not guarantee improvement in every future cycle.
• Environment comparison: Appendix C.5 states some ScienceAgentBench reference program failures come from package differences or the 900-second evaluation time limit, which limits comparison with scores in other environments.
• Attribution of teacher controls: Appendix D states differences in sampling, data, and checkpoint selection limit conclusions that can be attributed solely to teacher composition.

Experimental Coverage

• The tested generations are L(0), L(1), L(2) on Qwen3.6-27B-FP8, with scaffold qwen-code 0.20.0; the three roles in the homogeneous loop share the current model (Section 4.1).
• SFT teachers come from 8 synthetic tasks, 119 trajectories, 17,221,055 supervised tokens; testing includes 3 synthetic tasks and six public benchmarks. MLR-Bench reported scores are the 12-task subset × 5 trials (Appendix C.5, Table 7).
• External reporting evaluation is GPT-5.5 and Claude Opus 4.8, each with 3 seeds; AstaBench rubric evaluation is Claude Sonnet 4.6. Figure 7 replaces the evaluator and/or optimizer with GPT-5.5, with the executor still L(0).
• The paper reports Welch, Fisher, and paired sign tests for MLR-Bench, and states the sign test does not support consistent gains across tasks; none of the 180 MLR executions called the delegation tool (Appendix C.5).
• The first-cycle checkpoint is step 1200 with the lowest validation loss; teacher composition controls instead use external judgment for checkpoint selection. The paper does not report the optimization step count for the second cycle.

MASS is a homogeneous recursive self-improvement method: the same language model takes turns doing research task solving, workflow optimization, and evaluation, then uses multi-agent trajectories to update shared weights.

• Problem: On open research tasks, outputs may exceed human evaluation, and the model itself becomes the best available optimizer and reviewer. Shared priors make self-feedback potentially reinforce errors. The authors use information flow reorganization among copies, and joint learning of orchestration and bounded execution, to obtain self-supervision.
• Approach: The inner loop writes roles, instructions, output contracts, and call order as text workflows, evolves with elite size 1, and the optimization prompt emphasizes communication structure. The outer loop uses Bradley–Terry to select from M=18 executions, removes the workflow prompt, performs LoRA SFT on orchestrator and sub-agent windows, then lets the updated model enter the next cycle. External strong models are used only for post-hoc reporting.
• Results: After two cycles of Qwen3.6-27B, per output token scores on the four research benchmarks in Figure 2 are 1.20× to 1.60× the base; on three synthetic test tasks, L(2) vs L(0) win rate is 69.9%, and self-evaluation agreement rises from 0.73 to 0.93. Terminal-Bench 2.0 and SWE-bench Verified pass rates at L(2) are 35.2±1.7% and 61.1±0.6%, below L(0)'s 37.5±2.2% and 62.6±1.4% (Table 7). M+ uses 28.82M processed supervised tokens to reach 68.3% win rate, while S++ with 42M tokens is 64.0%. Under bare prompting, delegation increases from 1/110 to 87/110.
• Authors' conclusion: The authors believe joint learning of orchestration and bounded sub-agent execution can provide effective signals for RSI; post-training only on solution trajectories also changes optimization and evaluation. The authors list speeding up per-cycle gains as a future direction and state the loop has no formal step-by-step improvement guarantee.