Harness^2: Recursive Agent Execution Framework Improvement for Open Worlds

Open-world tasks lack ground-truth verifiers, and single-task harness modifications are also hard to transfer.

• Scenario: the authors believe agent effectiveness depends on the harness beyond the frozen model (prompts, memory, skills, tools, and orchestration). As tasks expand to domains like office and legal, manually designing harnesses repeatedly is costly, hence the need for recursive self-improvement (RSI).
• Existing shortcomings: early methods select static harnesses against fixed training tasks, unit tests, or verification feedback; the authors state these often generalize poorly to the open world. Online methods either modify the harness only for the current task, or carry a successful execution harness to later batches, relying on proxy signals, assumed answers, or pairwise output judgment.
• Core observation: single-task edits fail when requirements change, and directly inheriting a highly customized execution harness introduces noise under distribution shift. The improvement process itself—how diagnosis was done, which changes were tested, what execution revealed—the authors believe can be reused.
• What this paper does: the authors propose Harness2. Task-level recursion uses contrastive Propose–Probe–Compose to modify the current execution harness without ground truth; harness-level recursion writes adopted results and experience into a persistent improving harness to guide subsequent tasks.

Recursive self-improvement

• Definition and object of RSI: Duan et al. (2026) and Weng (2026) describe RSI as: a system turns execution results into persistent changes to itself, and this affects how subsequent improvements are generated, evaluated, and retained. The object can be weights (Yuan et al., 2024), improvement programs (STOP, Zelikman et al., 2024), or agent code (Darwin Gödel Machine, Zhang et al., 2026b).
• Meta-level and skills: Hyperagents (Zhang et al., 2026c) makes the meta-level modification process editable. MetaSkill-Evolve (Wang et al., 2026b) and MCE (Ye et al., 2026) co-evolve task artifacts and the skills governing their construction. Output-level methods include Self-Refine (Madaan et al., 2023) and Reflexion (Shinn et al., 2023).
• The authors' contrast: the authors state that these systems mostly optimize via utility, benchmark, or verification signals. Harness2 studies how to turn task-specific improvements into reusable experience on a deployment task stream, without ground truth, and without replaying completed tasks.

Harness optimization and online evolution

• From components to full harnesses: automatic optimization expands from prompts and workflows (DSPy, Khattab et al., 2024; GEPA, Agrawal et al., 2026; ADAS, Hu et al., 2025; AFlow, Zhang et al., 2025) to memory and skills, and then to complete executable harnesses (Meta-Harness, Lee et al., 2026b; AHE, Lin et al., 2026; HarnessX, Chen et al., 2026).
• Offline and test-time: offline methods repeatedly evaluate candidates on fixed training or development tasks. Test-time methods modify the harness for the current task, or carry an evolving harness across batches (RHI, Lee et al., 2026a; Harness Scaling, Wang et al., 2026a; TTHE, Nie et al., 2026b). Adaptive Auto-Harness (Liu et al., 2026) uses outcomes revealed only after task solving.
• Concurrent work: Continual Harness (Karten et al., 2026) improves within continuous episodes. HCL (Kang et al., 2026) and HSI (Zhou, 2026) study retention-aware adaptation and editable improvement strategies, respectively. The authors state that Harness2 simultaneously performs contrastive task-level improvement and updates to the improving harness in a single-pass deployment stream, without updating model weights.

Positioning in the comparison table

• Table 4: the authors compare using object, stage, whether there is an external correctness label, whether it is single-pass streaming, whether test-time scaling is done, and whether the improver is updated. Harness2 is marked Online, no Label, Streaming, Scaling, and Improver Update all as yes. The authors place the distinction in: simultaneously improving both harnesses on an unlabeled task stream, and learning from task-level exploration, rather than having only one of these properties.

Baseline methods and benchmarks

• Baselines: Base harness does not change the initial configuration. ReasoningBank (Ouyang et al., 2026b) fixes the harness and retrieves policies distilled from past trajectories into context. Meta-Harness (Lee et al., 2026b) searches offline on a training split, then deploys as-is. Harness Scaling (Wang et al., 2026a) spends extra inference per evaluation instance to modify the harness, without carrying state across instances.
• Benchmarks: WorkBuddy-Bench (Team et al., 2026), JobBench (Li et al., 2026), LAB (Harvey AI, 2026).

Harness2 performs recursive agent harnessing on a frozen base model: the task level improves the current execution harness, and the harness level updates the improving harness that guides future revisions.

Setup and editable surface

• Execution agent: model M's weights are frozen. The harness H consists of K editable components, which combine with M to form an agent without changing parameters. The authors adopt AHE's file-level decomposition plus one additional component, K=8: system prompt, guardrails, memory, skills, tool guidance, scripts, subagents, middleware plugin. Each component is mounted as a file. Figure 2 and Appendix Table 5 give the activation timing.
• Open-world stream: N tasks, with no ground-truth answers or labeled verification feedback. Each question starts from the common initial execution harness H0, and can rerun the current task within a bounded budget before submitting. Completed tasks are not replayed.
• Two-level division of labor: task-level recursion uses the generated harness and execution evidence to improve the current execution harness. Harness-level recursion, after submission, uses the adopted harness and improvement experience to update the improving harness. The improving harness is fixed during task-level recursion.

Contrastive Propose–Probe–Compose

One-step contrastive improvement is written as (H′, e′, ξ)=Φ(q_t, H, E; R_t): input the current task, execution harness, within-task history, and the fixed improving harness; return the successor harness, its execution record, and a step record containing reference executions, candidates, probes, composition decisions, and successor executions.

• Propose: first sample reference executions, only observing how the harness behaves, without judging whether the deliverable is correct. Then propose two candidates. Invariant candidates target defects that may recur across domains, such as planning, tool use, context management, or delegation. Domain candidates target knowledge, processes, and constraints of the current domain. Candidates are executable edits to component files, and may rewrite library components, but task-specific facts must be checked against current evidence.
• Probe: each candidate is executed on the same task with matched model and environment configurations. Compare trajectories and deliverables with the reference to see whether the edit mitigates the target defect, leaves behavior unchanged, or introduces regression. The authors state that the comparison guides composition and does not verify task correctness; execution stochasticity and shared model bias can limit evidence reliability.
• Compose: read the reference, candidates, diffs, and within-task execution evidence; may merge supported changes, modify conflicts, adopt one, or keep the current harness. Decisions must be tied to specific observations, not assumed score gains. After composition, execute again, and record the decision and evidence in the step record.

Parallel and sequential recursion

• Budget k: a positive integer. In parallel, each round has k candidates and k compositions; in sequential, a chain of k steps. The improving harness is fixed throughout task-level recursion.
• Parallel: first collect k reference executions of the current harness, then independently propose and probe k invariant candidates and k domain candidates. Each of the k composition sessions sees the full candidate pool, selectively combines, then executes. The authors state that implementation performs only one population update; k controls width rather than depth.
• Sequential: starting from H0, perform k contrastive steps, each conditioned on the previous step's harness and accumulated execution history. The initial history contains k base executions, and each step appends its step record. Later steps may retain, modify, or correct earlier changes based on prior evidence.
• Submission selection: when k>1, the selector compares anonymous deliverables against the task request and source materials, returns an existing output, does not see the source harness, and does not synthesize new content. The corresponding harness is marked as adopted; the authors write the asterisk to indicate adoption rather than verified optimality. Selection no longer changes the harness. When k=1, both forms degenerate to the basic contrastive step and skip selection. Appendix H gives the full prompts for each stage.

Harness-level recursion

• Reflection: after submission, review the full improvement record, including reference executions, candidates and probes, composition decisions, and the selected result. Keep only experience supported by execution comparison. Merely because an output was selected does not establish transferability. If evidence is insufficient, do not add a new experience entry.
• Library update: R_t+1=Update(R_t, H_t^★, D_t). Each entry contains an executable harness and EXPERIENCE.md (applicable situations, experience, supporting evidence), plus an index for later on-demand lookup. Appendix D.2 describes library organization.
• How the next question uses it: still start from H0, without automatically inheriting the previous question's execution harness. Previous components are selectively reused or rewritten via Propose; accumulated experience also enters Probe and Compose. Model weights, stage instructions, and decision rules are fixed; the library evolves. Larger task-level exploration produces more execution evidence and composition decisions.

Runtime settings

• Roles: the main experiment executors are frozen Gemini 3.5 Flash and GLM-5, with initial harnesses being off-the-shelf OpenCode and Codex, plus the benchmark's reference instructions. The default editor and selector are Claude Opus 4.8, run in OpenCode with high reasoning effort; evidence is placed in the working directory, not inlined into prompts.
• Validation and caps: invalid trees (skills without trigger phrases, scripts that fail to compile, skills calling assistants not included) are retried once with validation information; if it fails again, inherit the tree from the start of that stage and record the fallback. A single rollout is capped at 200 model turns and 3600 seconds; WorkBuddy-Bench uses its container timeout, twice the benchmark default. One rollout does not remember other tasks. Directories containing scoring rubrics, scales, or scores are not given to improvement sessions.

Experimental setup

• Four configurations: frozen executors Gemini 3.5 Flash and GLM-5, with initial harnesses OpenCode and Codex. The default editor and selector are Claude Opus 4.8. There is also the same model serving as both executor and editor, and using the harness improved for Gemini 3.5 Flash as-is for other executors.
• Benchmark scale: WorkBuddy-Bench is split in half by category, reporting test-half Code 40, Web 35, Office 25. JobBench uses the frozen 31/34 split, reporting 34 questions. LAB reports 100 test tasks; 24 development tasks are given only to Meta-Harness as a search set. The training half is used only by Meta-Harness.
• Metrics and review: WorkBuddy-Bench's Score is the proportion passing checks, including deterministic checks and, when needed, LLM/VLM review done by Gemini 3.5 Flash; expected values are not exposed to the pipeline. JobBench's Score is the weighted scale score divided by the full score, with review by Claude Opus 4.8; the scale is not given to the executor, editor, or selector. LAB has no executable checks, is scored item-by-item by Claude Sonnet 4.6, and reports Partial. Also reported per question is #Steps.
• Protocol: main tables are means and standard deviations over three runs, each with shuffled task order. Harness2 is single-step k=1; + Recursion is the change in mean after parallel recursion k=5. Each benchmark stream starts with an empty library, updated after each question.
• Baselines: Base harness, ReasoningBank, Meta-Harness (offline, marked † in tables), Harness Scaling. Baseline revisions and reflections also use Claude Opus 4.8. ReasoningBank uses gemini-embedding-001 for embedding retrieval. Appendix C additionally has Best-of-k; Figure 5's legend includes it, but the main text does not give its readings.
• Quantities not listed in the main text: Figure 6 counts all tokens before submission as total computation per question, with the horizontal axis in M tokens; the main text does not give readings for each point.

Main results

Per Tables 1–2. Cells are Harness2 means at k=1, with parentheses giving the base for the same configuration. Standard deviations are in the original tables.

Swipe horizontally to read the table

Configuration	WB Avg	JobBench	LAB Partial
Gemini 3.5 Flash / OpenCode	77.5 (72.4)	36.0 (31.5)	81.9 (79.9)
Gemini 3.5 Flash / Codex	77.5 (74.5)	35.3 (33.9)	75.6 (71.6)
GLM-5 / OpenCode	75.7 (64.2)	24.2 (20.6)	71.4 (59.6)
GLM-5 / Codex	75.3 (64.4)	24.8 (23.6)	65.9 (58.1)

• Single-step direction: the authors state that on all four configurations, a single step is above the respective base. The authors note that the largest single-step gain on WB is GLM-5, OpenCode's average score increasing by 11.5 points; the highest gains on JobBench and LAB Partial are 4.5 points and 11.8 percentage points.
• Relative to baselines: the authors state that on WB Avg. and JobBench Score across all four configurations, a single step is above the compared baselines. The exception is LAB Partial: under Gemini 3.5 Flash, Codex, before recursion the highest is Meta-Harness's 76.5, above the Harness2 in that table cell (Table 2).
• Asymmetry and baselines below base: the authors state that GLM-5's absolute gains on WB and LAB are larger than Gemini 3.5 Flash's; on JobBench, Gemini's are larger, at 4.5 and 1.4 points under OpenCode and Codex, versus GLM-5's 3.6 and 1.2. The authors state that after a single step, GLM-5's WB average score is above the corresponding Gemini base in the table. The authors state that ReasoningBank is up to 6.6 points below base on programming tasks; Harness Scaling is below base in seven of twelve benchmark–configuration combinations. In the original tables, #Steps increases in some cases and decreases in others.

Task-level recursion and computation

• Figure 1 and Figure 5: Figure 1 left is Gemini 3.5 Flash, OpenCode's WB results by domain. The authors state that parallel k=5 exceeds the strongest baseline Meta-Harness in that figure in every domain; Code, Web, Office are 80.8, 78.3, 83.4, with labeled increments relative to base of +7.4, +8.4, +9.4. Figure 5 compares parallel and sequential recursion on WB-code and WB-office; that section does not specify the model. The authors report both rise monotonically as k goes from 1 to 5; parallel goes from 76.5 to 80.8 on Code and from 79.5 to 83.4 on Office. These endpoints match the Harness2 row for Gemini 3.5 Flash, OpenCode in Table 1 and the k=5 scores above. The authors state that at k=5 parallel is higher than sequential, and at k=2 sequential is higher on Code; both recursions at k=2 are above the points of each baseline in the figure at k=5. Harness Scaling degrades from k=2 to k=3 in both domains.
• Relative to single step: the authors state that k=5 parallel recursion adds on top of a single step WB 2.9 to 3.9 points, JobBench 3.0 to 4.2 points, and LAB Partial 2.6 to 6.7 percentage points. The authors state that on JobBench, the initial OpenCode–Codex gaps for Gemini and GLM-5 (2.4 and 3.0 points) both shrink to 0.5; on WB, Gemini's drops from 2.1 to 0.4, while GLM-5's grows from 0.2 to 1.3. The authors state that recursion does not uniformly shrink initial harness gaps.
• Computation: Figure 6 counts all tokens before submission. The authors state that the two evaluated Harness2 configurations lie on the empirical Pareto frontier. Under Gemini 3.5 Flash, OpenCode, going from k=1 to k=5 makes the gain relative to base go from 4.5 to 7.5 points on JobBench and from 5.1 to 8.4 points on WB. The authors state that Meta-Harness at k=5 has higher computation than Harness2 and lower performance. The main text does not list token readings for each point.

Improvement library and reuse

• No more probing: Figure 7 left, the authors report that when directly modifying base without reference executions, candidate probes, or the library, Code drops from 73.4 to 63.2 and Web from 69.9 to 65.0. That section does not specify the executor; 73.4 and 69.9 match the Gemini 3.5 Flash, OpenCode base in Table 1. The k=1 library restores base level; the k=5 library, when no longer probing, supports gains of 0.9, 4.4, and 5.1 points on Code, Web, and Office respectively.
• Across questions and across benchmarks: the authors state that the library sourced from JobBench improves WB single-step improvement by 0.6, 0.5, and 2.9 points respectively (Figure 7 right). After removing the current question's own entries, WB goes from 77.5 to 79.4 and JobBench from 36.0 to 37.7 (Figure 11). The authors state this is an offline reuse analysis, not an online adaptation estimate, and repeated execution may also cause differences.
• Where changes come from: Figure 10, library sources appear in 22% of modified files on LAB and WB-office, and 47% on JobBench; only 2–10% of modified files have at least half their lines from library entries. The authors state that adaptation is more common than large-scale copying; skills and scripts account for about half of edits in each benchmark.

Same-model, transfer, and failure cases

• Same model in both roles: Figure 8, OpenCode, k=2. The authors state that all five models from GLM-5 to Claude Opus 5 improve on every benchmark. Gains given in the main text: Opus 5 on WB, JobBench, LAB is 3.3, 5.5, 3.0 points; GLM-5 on LAB is 13.7 points. The names of the middle three models are not listed in the main text. The authors state that improvement need not depend on a stronger editor.
• Across executors: Figure 12 applies the harness improved for Gemini 3.5 Flash to other executors without re-adaptation. The authors report GLM-5's Code going from 58.5 to 75.8 (+17.3 points); Gemini 3.7 Flash on Code +0.3, Office +6.4; Claude Opus 4.8 +2.6 and +1.1. The authors state that benefits vary by executor and domain.
• Cases below reference: Appendix Figure 14 classifies nine results into three categories. The selector chose a weaker candidate, e.g., WB-code from 96 to 55, with a 95-point candidate in the pool; LAB from 54 to 26. Wrong premises propagate across candidates, e.g., 75 to 64, 83 to 81. There are also result fluctuations, review sessions returning no verdict, and edits not touching the scored item, e.g., WB-web from 44 to 10; the authors state that the review session crashed, and the three stock runs also encountered this. The authors state that these cases point to a balance between coverage and precision, probes that challenge shared assumptions, and using repeated execution to distinguish edit effects from result fluctuation.

Other ablations and analyses

• Table 3 (Gemini 3.5 Flash, OpenCode, k=1): full WB / JobBench is 77.5 / 36.0; removing the library 74.4 / 34.2; removing scope division 74.6 / 32.3; removing composition 75.9 / 34.1. The authors give the drops as 3.1 and 1.8, 2.9 and 3.7, 1.6 and 1.9 points.
• Table 6 (same executor, full is parallel k=5): full three-domain average 80.8; removing the prompt group 79.3, removing skills 75.5, removing scripts 77.1, all above base 72.4. Skills lose 3.6 and 6.9 points on Code and Web, scripts lose 7.4 points on Office. The authors state that additive effects cannot be established from this.
• Table 7 (same executor, k=1): tool error rate on WB from 0.8% to 0.5%, JobBench from 14.4% to 9.1%, LAB from 2.0% to 1.6%. Call counts decrease on WB and increase on JobBench and LAB. The authors state that improvement can raise scores without shortening execution.
• Figure 13: among unmodified CodeBuddy, OpenCode, Codex, and Claude Code, none leads in every domain; Claude Code is highest on Code and Office, and Codex leads on Web. Harness2 starting from OpenCode (k=5) is 4.6, 5.8, 4.9 points above the strongest fixed comparison, and 7.4, 8.4, 9.4 points above its own starting point. The authors note that improvement uses additional test-time computation.
• Single-question cases: Figure 9 on JobBench, Gemini 3.5 Flash, OpenCode, base is 43.1, and after composition rerun is 77.8. The authors state that the trajectory shows process failure rather than a knowledge gap. Appendix E.7 additionally has multi-domain single questions, including 60.0 to 100.0, 23.6 to 100.0, 44.4 to 94.5, 72.0 to 100.0.

Limitations and future directions noted by the authors

• Recursion cost (Appendix F): improvement requires additional execution and editing sessions; increasing the budget raises inference cost. Parallel sessions can reduce latency when resources allow, but do not eliminate this computation. Experiments only characterize the trade-off within a bounded budget, not a universally optimal allocation.
• Execution feedback (Appendix F): without ground-truth supervision, improvement relies on behavioral comparison and model judgment. These signals are sensitive to execution fluctuation and shared model bias, are not guaranteed correct, and do not establish that adopted changes transfer to different tasks.
• Library management (Appendix G): the current library is a simple index. The authors suggest that as the library grows, one could merge overlapping components, revise outdated experience, and improve retrieval according to the current task and available evidence, to keep it compact while preserving the context needed for reuse.
• Adaptive recursion (Appendix G): the recursion form and budget are currently pre-specified. The authors suggest that future work could allocate exploration by task difficulty and the relevance of existing experience, or stop when additional probing yields little new evidence, to balance current cost against value for future harnessing.

Experimental coverage

• Main results use two frozen executors (Gemini 3.5 Flash, GLM-5) and two initial harnesses (OpenCode, Codex), with benchmarks WorkBuddy-Bench test half (40/35/25), JobBench 34 questions, and LAB 100 questions (Appendix B, Tables 1–2).
• Each number in the main tables is the mean and standard deviation over three independent streams with shuffled task order. Single-step is k=1; the main comparison for parallel recursion is k=5. Figure 5 additionally reports parallel and sequential curves for k from 1 to 5; that section does not specify the model.
• Comparisons are Base harness, ReasoningBank, Meta-Harness, Harness Scaling. Meta-Harness uses the training half or LAB's 24-question development set; Harness2 and other baselines do not use these splits.
• The default editor and selector are Claude Opus 4.8. Also reported are the same model serving both roles (OpenCode, k=2), and using the harness improved for Gemini 3.5 Flash as-is for GLM-5, Gemini 3.7 Flash, and Claude Opus 4.8.
• The paper does not report agreement between LLM/VLM review and human judgment; the main text does not list token readings for each point in Figure 6.

Harness2 is recursive agent harnessing for open-world task streams: the frozen model is unchanged, while both the current execution harness and the future improvement mechanism are modified.

• Problem: New tasks were not seen at design time and have no ground-truth verifier. Modifying the harness only for the current task, or passing down a highly customized execution harness, cannot reliably serve subsequent tasks. The authors want RSI to modify both the current harness and how future improvements are made.
• Method: At the task level, use contrastive Propose–Probe–Compose. First look at reference executions, then propose two kinds of candidates, invariants and domain-specific ones, compare behavioral differences under matched conditions, and compose based on concrete observations rather than assumed scores. The budget k can be widened in parallel or iterated sequentially; when k>1, an anonymous deliverable selector submits the answer. After the task, only experiences supported by execution comparisons and the adopted harness are written into the improvement library. The next problem still starts from the common H0, with the library guiding proposal and composition. The default editor is Claude Opus 4.8. Appendix H gives the stage prompts.
• Results: On WorkBuddy-Bench, JobBench, and LAB, Gemini 3.5 Flash and GLM-5 with OpenCode or Codex all have single-step scores above their respective baselines. The authors report the largest single-step gains as GLM-5, OpenCode's WB average score of 11.5 points, as well as JobBench 4.5 points and LAB Partial 11.8 percentage points. Parallel k=5 adds further on top of single-step, which the authors say adds at most another 3.9 points, 4.2 points, and 6.7 percentage points. On LAB, for Gemini 3.5 Flash, Codex Partial, the pre-recursion Meta-Harness (76.5) is higher than single-step Harness2. In Figure 1, the per-domain scores for Gemini 3.5 Flash, OpenCode, k=5 are 80.8, 78.3, 83.4.
• Authors' conclusion: The authors say that scoped proposal, composition, and accumulated harnessing experience contribute, and recursive harnessing can turn task-specific improvements into a reusable foundation for open-world online adaptation. Appendix F also states that this relies on additional inference, and behavior comparison does not guarantee correctness or transfer across tasks.