Memento 3 proposes a rulebook-based recursive self-improvement method

Limited observations can simultaneously support multiple world models that fit the past but differ in predictions for unseen states; a frozen LLM must continuously revise an explicit model without updating its parameters.

• Scenario: The authors argue that an agent in an unfamiliar environment may not know the objects and dynamics, nor the success criteria. It can see the action interface, interaction feedback, and completion signals, but not the environment source code, transition rules, underlying state, or natural language task descriptions.
• Existing shortcomings: The authors point out that consistency with observed transitions does not guarantee correct generalization. A limited history often leaves a version space: multiple programs can replay all observations but differ on unseen states; replay can reject inconsistent programs but cannot point to the correct one among those still consistent. The Memento series mainly improves policy-side memory rather than environmental rules.
• Assumptions: The rulebook records revisable hypotheses; unobserved mechanisms need not be hard-coded; code is its executable implementation. The authors claim that under fidelity and exact replay constraints, the LLM prefers short descriptions, which is a minimum description length bias, but this cannot eliminate uncertainty; committing to only one hypothesis may cause exploration to avoid evidence that could falsify it.
• Proposed method: MEMENTO 3 uses Code as Model to extend reflective learning to explicit world models. The loop is observation, reflection, rule revision, compilation, verification; updates must pass the LLM's rulebook fidelity judgment and cell-by-cell replay. The underlying LLM is fixed. A population extension maintains N rulebook–executable pairs in parallel, sharing evidence; N=1 is the single model.

• World models and executable representations: Dyna (Sutton, 1991) uses real interactions alongside model-generated experience for planning and learning. Neural world models support imagination rollouts, model predictive control, and tree search (Ha and Schmidhuber, 2018; Chua et al., 2018; Schrittwieser et al., 2020); Dynalang treats language as a prediction modality in a multimodal latent variable model (Lin et al., 2024). The authors state that these methods encode dynamics in neural parameters and latent representations.
• Language-guided program induction: Parsel (Zelikman et al., 2023) combines natural language decomposition with execution-based testing; Hypothesis Search (Wang et al., 2024) translates language hypotheses into programs and validates them on input-output examples. For interactive environments, WorldCoder (Tang et al., 2024) learns Python dynamics and repairs against counterexamples; GIF-MCTS (Dainese et al., 2024) searches candidate models through generation, improvement, and repair. There are also code world models for general games (Lehrach et al., 2025) and Code-as-World for physical reasoning (Wang et al., 2026). The authors state that these works establish language-guided construction and execution-based refinement.
• Executable models on ARC-AGI-3: baseline1 (Rodionov, 2026) maintains, simplifies, and validates persistent Python models before planning. The authors relay its follow-up component study: in the tested configurations, exact validation is always useful, but requiring an executable model does not uniformly improve action performance. OPINE-World (Courtis et al., 2026) maintains natural language hypotheses, synthesizes object-centric executable programs, and uses ontology errors to guide exploration; candidate programs must exactly reproduce recorded transitions. Twin (Skoutnev et al., 2026) uses history replay as a gate for model-based action. Tycho (Lehmann et al., 2026) examines when to construct, repair, use, or bypass executable models; the authors relay: better transition replay does not necessarily lead to better decisions, if the goal is misidentified or the model is misused.
• Language rules and the Memento series: Reflexion (Shinn et al., 2023) stores language reflections on task feedback. AutoManual (Chen et al., 2024) organizes environment rules into a reusable manual. WALL-E (Zhou et al., 2024) compares predicted and observed trajectories, updating and pruning symbolic rules to align LLM world models. Memento (Zhou et al., 2025) uses episodic memory for continual adaptation without fine-tuning the LLM; Memento 2 (Wang, 2025) maps memory writing to policy evaluation and memory reading to policy improvement; Memento-Skills (Zhou et al., 2026) extends to reusable executable skills. The authors state that Code as Model extends this line to model-side semantic memory.
• Uncertainty, exploration, and continual adaptation: Version spaces (Mitchell, 1977, 1982) formalize ambiguity left by finite evidence; minimum description length (Rissanen, 1978) prefers simpler hypotheses among consistent ones. Model-based reinforcement learning uses probabilistic dynamics or ensembles to represent uncertainty (Chua et al., 2018; Kurutach et al., 2018); posterior sampling and related methods retain sampled hypotheses or value functions within an episode (Osband et al., 2013, 2016; Lu and Van Roy, 2017). For continual adaptation, DreamTeam's Workspace Optimization (Sarafian et al., 2026), Continual Harness (Karten et al., 2026), and NOOA (Furgale et al., 2026) treat external software or workspaces as editable learning state. Belief-Calibrated Optimization (Chen et al., 2026) models how evaluation responds to prompt and scaffolding changes; the authors contrast that Code as Model predicts in-environment transitions. Graph exploration methods record observed transitions and prioritize untested actions without building a generative world model (Rudakov et al., 2025).
• Baselines and benchmarks: ARC-AGI-3 comparisons are leaderboard entries linked to the paper: baseline1, NOOA, OPINE-World, DreamTeam, Continual Harness; the human action baseline also comes from the official ARC Prize leaderboard. Pong comparisons include the same LLM without a world model, a uniform random policy, and published RL scores cited in the paper; Table 5 notes these published baselines follow their own evaluation protocols.

The authors organize language rules, executable artifacts, reflection, and replay into a persistent environment specification, supporting both rule changes and implementation-only fixes. The population is described by the authors as a computational approximation of the remaining version space, not a calibrated posterior, used to reduce dependence on a single early construction. The authors also state that Tycho and Rodionov's component studies show stronger frontier models have saturated or nearly saturated the public set, so completing the public set alone cannot separate architectural contributions; comparisons must also consider model choice, inference budget, action efficiency, repeated-run stability, and held-out evaluation.

Code as Model writes a frozen LLM's assumptions about an unfamiliar environment into a natural-language rulebook, then compiles them into an executable program; the verified model guides subsequent planning and exploration while the LLM parameters stay unchanged. The authors call this feedback model-based recursive self-improvement: what gets revised are persistent world-model assumptions, not LLM weights.

Problem setup and learning state

The environment is written as a deterministic, goal-oriented partially observable Markov decision process. The agent must reach an unknown goal set within a real action budget B. Model construction, simulation rollouts, and replay do not count toward the budget; exploration, failed attempts, and resets sent to the environment all count. The main objective is to increase the probability of reaching the goal within the budget; when success probabilities are equal and positive, the objective becomes lowering the conditional action cost.

The external learning state is Z_t=(H_t,ψ_t,θ_t): the interaction record provides evidence, the rulebook records assumptions about entities, actions, dynamics, goals, and termination conditions, and the executable program provides transition predictions and inferred goal conditions on its own internal state. The rulebook may leave unobserved mechanisms unspecified, thereby admitting multiple implementations. Changing only the implementation may leave the rulebook unchanged; a change in the understanding of environment rules must first be written into the rulebook before being compiled.

A pair used for planning must satisfy θ∈[[ψ]], and after replaying each recorded action, the predicted next observation must match the record. [[ψ]] denotes the set of programs that faithfully implement that rulebook.

Reflection, compilation, and verification

After each real action, the implementation update operator takes five steps.

1. Observe: Append the new action–observation pair to the current trajectory. Completion feedback is kept with the record.
2. Reflect: The frozen LLM chooses among keeping the current pair, revising the rulebook, and fixing only the code. A mismatch in prediction can come from wrong rules or from code that does not implement the stated rules.
3. Rule revision: The rulebook is changed only if the choice is to revise; the rulebook constrains the next executable program.
4. Compile: When revision or repair is needed, the compiler proposes a new program from the previous program, the proposed rulebook, and the updated record. If kept, the old program is used as is. The program takes an internal state and an action to produce the next state, the predicted observation, and the predicted completion.
5. Verify: Exact replay checks every recorded next observation. Faithfulness to the rulebook is judged by the frozen LLM based on the rulebook and the source code, not on language-model confidence alone. The authors' stated rationale is that selecting hypotheses by language-model ranking may exclude correct candidates. Only when both pass is the pair accepted; otherwise control returns to reflection until an acceptable pair is found. Each new proposal must explain all existing records, not just the latest discrepancy.

In the single-model case, the accepted program is replayed to obtain the current internal state, and the belief is approximated as the point mass on that hypothesis and that state. With multiple trajectories, each is replayed from its own initial observation.

Selection, planning, and exploration

Multiple acceptable pairs can remain after verification. The single-model setting uses SimplifyLLM to prefer shorter rulebooks and implementations among them. The authors give a maximum a posteriori interpretation: the prior favors pairs with shorter descriptions that are faithful to the rulebook; the observation likelihood keeps only programs consistent with replay. The frozen LLM approximates this selection by proposing simpler rules and source code, and each simplification rechecks faithfulness and replay.

Planning treats the current executable program as a deterministic model and, from the internal state obtained by replay, searches for the shortest action sequence that reaches the inferred goal within the budget, then executes its first action. When no path exists, no plan is returned. At that point, the LLM chooses an intermediate goal from the history and unresolved assumptions, such as testing a mechanism, seeing an unseen region, or touching an unexplained object; the auxiliary planner searches the current program for a sequence that reaches that goal. Both planned and exploratory actions stop at the first mismatch between prediction and observation, handing the new evidence back to reflection.

Population extension

N≥1 acceptable rulebook–executable pairs share the same interaction record rather than making separate copies of the environment. N=1 is the single-model case. Each planning round samples uniformly over behavioral equivalence classes of nonempty plans: plans are the same class when they produce the same action sequence under the same history. The chosen representative executes its sequence until success, falsification, or replanning. Members that conflict with the new observation go through reflection, necessary rule revision, compilation, and verification. The authors state that if a model's own chosen actions never expose its error, it may remain unfalsified; the population is meant to let different models drive exploration. They also state that the uniform empirical distribution is a computational approximation of the remaining version space, not a calibrated posterior; the observed record removes refuted models and preserves prior odds among survivors.

Implementation on ARC-AGI-3

The visual portion of each observation is a 64x64 colored grid. Actions come from the given key/click interface, and their semantics are initially unknown. Taking the last level of ls20 as an example, fog of war hides regions beyond the local view, so the current frame does not reveal the full state. The rulebook is written in world_model.md, and the executable part includes world_model_engine.py plus state reconstruction, rendering, and completion checking. The program predicts the full next grid and whether the level is complete. A new trajectory is initialized from the observation returned after a reset or the start of a level; only when the rendered predictions match all 64x64 cells of every subsequent frame in the trajectory is the program handed to the planner. The planned sequence is executed one action at a time as defined, and stops at the first mismatch.

This loop runs throughout the whole game, not just during initial modeling. The rulebook and program persist across attempts and across levels; later levels may introduce new mechanisms, so continued revision must be possible. Rule-level changes are written to the Markdown first, then to the Python; if the issue is diagnosed as an implementation problem only, the rulebook is not changed. Figure 5 shows three of the 11 rulebook versions from one run on tr87: two competing hypotheses recorded before an action about glyph orientation, changed after interaction to rotation-invariant letter identity, then generalized by later evidence and in turn rewritten to revise the cursor rule.

The Git repository stores the model history. Each iteration commits the workspace and tags it with the iteration number. The agent can look at earlier versions, compare diffs of rulebooks or modules, or restore executables broken by later edits.

Experimental Setup

• System under test: The main result is a single-model Code as Model, N=1. The backbone is Claude Opus 5, with reasoning effort set to extra-high, running in Claude Code. The experimental harness starts from the baseline1 codebase. The agent only observes the action interface and cannot view the environment source code.
• Benchmark: All 25 public ARC-AGI-3 games (Figure 6), with a total of 183 levels in Table 1. There is also an Atari Pong case study, simulated with the Arcade Learning Environment, with frame-skip set to 4.
• Baselines: For ARC-AGI-3, the comparisons are baseline1, NOOA, OPINE-World, DreamTeam, and Continual Harness, linked to the paper on the leaderboard. The paper states that these numbers and the human action baseline come from the official ARC Prize leaderboard, with each column counted under the same criteria. Pong comparisons include the same LLM without a world model, a uniform random policy, and published RL results cited in Table 5.
• Metrics: RHAE is higher-is-better, with 100 as the upper bound. eℓ is the smaller of 1.15 and the square of (hℓ/aℓ); for levels not completed, aℓ is infinite. The authors state that reaching the upper bound requires completing every level, and the level-weighted mean of eℓ in Eq. (36) must be at least 1. Action counts include all actions sent to the environment, including failed attempts and resets; model construction, local execution, and replay are not counted. Pong episode score is own score minus opponent score, ranging from −21 to +21.
• Determination: Completion comes from the environment's reported completion signal; the paper does not state that another model scores RHAE. Rulebook fidelity is judged by the same frozen LLM; replay requires 64×64 pixel-by-pixel consistency.
• Replication: The paper reports Table 1 and Table 2 per game, without stating the number of repetitions per game. Ablation and population experiments each give a set of numbers for the reported runs, also without stating repetitions.

Main Results

From Table 1 and Table 2. Action counts with a dagger are costs from runs where at least one level was not completed; the paper states these cannot be treated as solution cost comparisons. Human total actions are 17,135. baseline1's 0.49 relative to human comes from Section 4 text.

Swipe horizontally to read the table

System	Average RHAE	Games Completed	Levels Completed	Actions	Relative to Human
Ours (single model)	100.0	25	183	7,518	0.44
baseline1	99.0	25	183	8,347	0.49
NOOA	85.1	19	170	11,697	Not reported
OPINE-World	78.4	20	160	12,859	Not reported
DreamTeam	38.1	6	98	9,789	Not reported
Continual Harness	20.5	3	64	14,558	Not reported

• Upper bound and exceptions: The authors state that the closest in the table is baseline1, which also completes all games. In Table 1, Ours is written as 100 for each game; baseline1's values below 100 are 84.2 for sc25 and 90.1 for tn36.
• Action ratios: Section 4 states per-game action ratios range from 0.20 for m0r0 to 0.67 for lf52, with a median of 0.42. In Table 2, Ours is not fewer than baseline1 for every game; for example, wa30 is 899 vs. 645, and lf52 is 894 vs. 767.
• Another comparison in the introduction: The authors state that relative to Claude Opus 5 on the ARC Prize Standard harness, the average RHAE is 59.3 points higher. The reference note states that result is for 25 environments with high reasoning effort. That comparison is not included in Section 4's table; this paper's backbone uses extra-high reasoning effort.

Rulebook Ablation

• What was tested: Removed the natural language rulebook from the update loop, keeping the code world model; the harness, replay verifier, plan executor, backbone, and reasoning effort remain unchanged. Three games were selected based on human actions per level: ft09 at 35 (easy), ka59 at 104 (medium), and cn04 at 132 (hard).
• Results: In Table 3, each run completes all levels with RHAE of 100, so the comparison is on convergence cost. With the rulebook, actions/agent turns are: ft09 75/132, ka59 341/298, cn04 201/248; without it: 79/174, 382/310, 216/346. The authors sum the three games as 617 vs. 677 actions, a 9% reduction; agent turns are 678 vs. 830, an 18% reduction. Actions count environment interactions; agent turns are not affected by wall-clock delays.
• Authors' interpretation: The authors believe that, with completion unchanged, these results suggest the persistent rulebook improves convergence to a usable executable model. All three games have fewer actions; the largest agent-turn reductions are for ft09 and cn04.

Population Scaling

• What was tested: The main experiment is N=1. The population experiment selects only wa30, which has the largest human action baseline, and compares N=2. The backbone, reasoning effort, and harness are the same as the single model. The communication mechanism informs both members who provided the current plan, which actions were issued, and what the subsequent true transitions were.
• Results: In Table 4, both sides complete all nine levels, and RHAE is 100 when replayed against the live server. Relative to the validated single-model trajectory, scoring actions drop from 899 to 597, a 0.66× factor. Eight of nine levels are equal or fewer; L8 is 138 vs. 130, a 1.06× factor.
• Authors' interpretation: The authors state that the two world models start from different assumptions and converge as shared history eliminates divergence. They believe these results suggest that maintaining a population can make world-model-guided exploration more action-efficient and reduce sensitivity to any single initial assumption.

Atari Pong Case Study

• What was tested: Section 5 replaces the level-completion planner with a feedback policy, using the learned executable model to predict consequences of candidate actions and updating decisions with new observations. The agent controls one paddle and must also predict the ball and the opponent paddle. During learning, checkpoints are evaluated in separate episodes; the controller is fixed at evaluation and no longer calls the LLM. Exploration and revision continue until achieving 21:0, then stop.
• Results: Three different starting positions all achieve 21:0, with episode score 21; Figure 8 shows action counts of 3,358, 4,014, and 5,039. Learning cost is 9,504 emulator frames in the text, and Table 5 writes 9.5×10^3. The same LLM without a world model, called once per frame, seeing only the last two frames, with no persistent memory, scores −19. The uniform random policy in Table 5 is −20.7; the random episode plotted in Figure 8 ends at −20. Among published comparisons, GDI and LASER achieve 21, both with learning cost 2.0×10^8 frames; IMPALA achieves 20.98 and Rainbow 20.9, both at 2.0×10^8; EfficientZero achieves 20.1 and EfficientZero V2 20.8, both at 4.0×10^5. Human average is 14.6. Table 5 notes that this paper's score is the average of these three episodes, and published baselines follow their own protocols. The authors state that the reported learning costs for model-free and model-based baselines are approximately 2.1×10^4 times and 42 times 9,504 frames, respectively.
• Authors' interpretation: The authors state that the rulebook and compiled code turn the frozen LLM's assumptions into a revisable, reusable explicit model, enabling learning with limited interaction. Figure 9 shows three rulebook versions during one learning session: a horizontal return for center hits negates the initial four-bounce assumption, and later contacts change it to an asymmetric, speed-bounded banding. Section 5 states that full-frame replay retains rendering and paddle motion residuals; before each action, the controller reconstructs the state from the most recent frame and action. The authors also state that the model need not recover dynamics for every possible state, only be accurate enough for decisions needed for successful control.

Other Ablations and Analyses

• The paper has no additional ablation tables. Table 2's per-game ratios are summarized in Section 4 as 0.20 to 0.67, median 0.42.
• Pong full-frame replay residuals are in Section 5, not a separate quantitative ablation.
• Figure 5's tr87 rulebook versions and Figure 9's bounce rule revisions are qualitative examples, with no separate success-rate numbers.

Limitations and future directions noted by the authors

The paper does not specifically discuss limitations. The conclusion only restates the method, the results on ARC-AGI-3 and Pong, and the authors' framing of model-based recursive self-improvement, without listing shortcomings or follow-up experiments.

Experimental coverage

• Section 4 reports single-model results over all 25 public ARC-AGI-3 games, for a total of 183 levels; the backbone is Claude Opus 5, the reasoning effort is extra-high, it runs in Claude Code, the harness starts from the baseline1 codebase, and the main experiment uses N=1.
• The RHAE, actions, and completion counts of the five comparison systems, as well as the human action baseline, are taken from the official ARC Prize scorecard; the paper does not state that the authors reran these systems.
• The rulebook ablation uses one game each, ft09, ka59, and cn04, split into easy, medium, and hard by human actions per level. The population comparison is only done on wa30 with N=1 and N=2. The paper does not report replication counts for the main results, ablations, or population runs.
• Pong uses ALE with frame-skip 4; learning stops when the controller first reaches 21:0, the fixed controller is evaluated on three different starting states, and no LLM calls are made during evaluation. The RL scores in Table 5 come from published results and their respective protocols. Figure 9 corresponds to a single learning session.
• The acceptance criterion is the frozen LLM's judgment of consistency between the code and the rulebook, plus 64x64 cell-by-cell replay (Section 3.2). The paper does not report a retry cap after failed verification, the number of LLM calls for learning on ARC-AGI-3, wall-clock time, or cost figures; Section 4 only says that the time and cost of running Claude Code are substantial. The 59.3-point comparison in the introduction against a standard harness is not included in the tables of Section 4.

MEMENTO 3 explores a model-based recursive self-improvement path: the LLM parameters remain fixed, and the external world model is continuously revised and in turn guides interaction.

• Problem: In unfamiliar environments, limited interaction often simultaneously supports multiple programs. They can replay observed transitions but differ in predictions for unseen states. Replay can rule out inconsistent programs, but cannot identify the correct rules among those still consistent. Previous Memento work mainly accumulated policy-side memory.
• Method: Code as Model records revisable hypotheses in a natural language rulebook, then compiles it into an executable program for prediction and planning. When observations conflict, the frozen LLM decides whether to keep, modify rules, or only fix code. Accepting an update requires the LLM to judge that the code is faithful to the rulebook, and replay to reproduce observed transitions cell by cell. When there is no plan, the LLM chooses an intermediate exploration goal. N models can share the same history and be sampled according to behaviorally different plans; the main experiment uses N=1.
• ARC-AGI-3: A single-model agent completed all 183 levels across 25 public games; Table 1 shows an average RHAE of 100.0. Table 2 shows total actions of 7,518, which is 0.44 of the human 17,135. In the scorecard comparison, baseline1 had an average RHAE of 99.0 and 8,347 actions, also completing all games; the other four systems left some games unfinished. Without the rulebook, the reported runs on three games still reached RHAE 100, but with 677 actions versus 617, and agent turns of 830 versus 678. On wa30, N=2 used 597 scoring actions versus N=1's 899; L8 was more frequent in nine levels.
• Pong: The same learning mechanism is changed to feedback control. Three different openings all resulted in 21:0; the learning cost is recorded in the text as 9,504 emulator frames; the LLM is no longer called during execution. The same LLM without a world model scored -19. The authors state that published model-free and model-based baselines have learning costs of approximately 2.1×10^4 and 42 times this cost, following their respective protocols.
• Authors' conclusion: Bringing verified model updates into subsequent planning and exploration lets what has already been learned shape the evidence used in the next round of revision. The authors argue that this feedback is the model-based recursive self-improvement studied in this paper.