The Truth Beneath the Tokens: Prompt-Space Recovery of Edited Knowledge in Language Models

1. 01What problem does this paper attempt to solve?

   Does model editing truly eliminate old knowledge, or merely suppress it? The authors audit this with purely input-space interventions.

   Core question: Does model editing truly eliminate pre-edit factual knowledge, or merely suppress its retrievability under standard prompting? The authors argue that existing edit reversal methods all rely on weight-space interventions and cannot answer this question.

   • Problem scenario: Model editing is used to delete personally identifying information and correct outdated facts (Introduction). Edit success is usually judged behaviorally—the model counts as successful if it gives the new target for a new prompt—but this does not show whether the old knowledge is still there.
   • Shortcomings of existing methods: The authors point out that edit reversal work such as One Mask, Youssef et al. (2026), and Jiang et al. (2026) all formalize recovery as a weight-space operation, modifying model weights to undo the edit, and thus cannot show whether old knowledge remains recoverable without undoing the parameter update.
   • Core hypothesis: The authors hypothesize that editing may merely suppress the retrievability of the factual association without fully eliminating its accessibility; if so, an appropriate input intervention might elicit the original association (Section 3.1).
   • Method proposed in the paper: MEMENTO, a prompt-space audit mechanism that learns a reusable recovery signal (a continuous prefix or a discrete text prompt) to steer the edited model back to pre-edit answers without modifying model parameters.
   • Extended question: The authors further examine whether prompt-space recovery can go beyond editing scenarios—improving factual recall on the original unedited model, to probe the gap between "knowledge encoded by the model" and "knowledge retrievable by standard inference" (based on the distinction of Calderon et al. (2026)).

   Collapse this answer

2. 02What related research exists?

   Related work falls into three groups: model editing, edit reversal and recovery, and prompt optimization; the baseline is One Mask.

   Model editing

   • Parameter-preserving methods (Mitchell et al., 2022; Hartvigsen et al., 2023; Zheng et al., 2023) update facts by adding memory modules or relying on in-context capabilities; parameter-modifying methods such as ROME and MEMIT (Meng et al., 2022, 2023) directly change weights.
   • Subsequent work improves edit locality through null-space projection, weight regularization, and constrained matrix updates (Fang et al., 2025; Gu et al., 2024; Ma et al., 2025).
   • Cohen et al. (2024) and Baser et al. (2026c, a) study ripple effects and multi-hop reasoning, showing that edits can produce unintended consequences beyond the target fact. The authors state that this paper studies parameter-modifying edits, because edits that do not enter the weights do not have the problem of "what the weights retain."

   Edit reversal and recovery

   • One Mask (Holmov et al., 2026) learns a shared sparse binary mask on post-edit MLP weights to reverse edits across multiple facts; Youssef et al. (2026) perform reversal based on a low-rank approximation of the edited weight matrix; Jiang et al. (2026) selectively reverse edits using gated singular vector contraction.
   • Prompt-side reversal: Youssef et al. (2025) target in-context edits; Baser et al. (2026b) recover pre-edit answers through alternative tokenizations of the search query. The paper notes that the latter's reconstruction process incurs additional inference overhead, requiring multiple generations on alternative tokenizations for each target fact.

   Prompt optimization

   • Continuous methods such as prompt tuning and prefix tuning (Li and Liang, 2021; Lester et al., 2021) learn trainable input embeddings with weights frozen.
   • Discrete methods optimize token sequences from the vocabulary: Zou et al. (2023) use gradient-guided search to construct adversarial suffixes; Yu et al. (2026) show that optimizing a single continuous prefix can adjust model behavior with the backbone frozen.

   Baselines and benchmarks: The comparison baseline is One Mask (weight-space edit reversal); fact data come from the training/test splits of CounterFact (Meng et al., 2022), 1,000 each.

   The authors position this paper as: applying prompt-based optimization to recover pre-edit knowledge, which differs from weight-space reversal; MEMENTO does not modify parameters at all, and it evaluates whether the recovery signal generalizes across editing techniques.

   Collapse this answer

3. 03How does the paper address this problem?

   MEMENTO learns a continuous prefix or discrete text prompt shared across facts, using four losses to steer the edited model back to pre-edit answers.

   Overall approach: MEMENTO does not modify model parameters; instead it learns a reusable recovery signal—the continuous variant learns a soft input representation, and the discrete variant learns a text prompt prepended to the input—to steer the edited model back to pre-edit factual answers (Figure 1).

   Problem setting and formalization

   • A fact is represented as a triple (s, r, o), an edit request is e = (s, r, o, o*), and an editing technique E produces an edited model f_θ′ = E(f_θ, e) such that it preferentially predicts o* for p(s, r).
   • Goal: recover the original object o from the edited model without modifying weights. The authors call this knowledge recovery, to distinguish it from weight-space edit reversal.
   • Continuous recovery prefix Π ∈ R^(N×d) (N is the prefix length, d is the embedding dimension), with input x = [Π; WE[q]]; a single prefix is optimized jointly over a set of edited facts Dtrain, which the authors say captures a shared recovery direction rather than the answers of individual edits.

   Continuous recovery training objective

   The total objective is L = L_rec + λ_pref L_pref + λ_flu L_flu + λ_dist L_dist, where the four terms respectively: increase the likelihood of the pre-edit answer, make the original object more preferred than the edit target, limit changes to behavior on unrelated inputs, and distill the prefixed edited model toward the pre-edit model.

   • Lrec: the negative length-normalized log-likelihood of the original object o.
   • Lpref: a preference separation term with a sigmoid, where β controls the preference scale and γ specifies the margin, explicitly making the likelihood of o higher than that of o*.
   • Lflu: on general text u unrelated to the edited facts, penalizes the KL divergence between output distributions before and after adding the prefix.
   • Ldist: at the last prompt position, performs KL distillation of the next-token distribution from the pre-edit model (scaled by temperature T) to the prefixed edited model; the pre-edit model provides the distillation target only during training and is not needed at inference.

   Discrete recovery objective

   • Learn a token sequence a = (a1, ..., an) prepended to the query, with the objective being the expectation of Lrec + λpref·Lpref over a set of edited facts.
   • The search is constrained to the set of text-realizable token sequences A = {a : tok(dec(a)) = a}, ensuring the sequence remains unchanged after decoding and re-tokenization, so it can be used through standard text interfaces.
   • Optimized with greedy coordinate gradient (Zou et al., 2023): replacement candidates are proposed by ranking gradients with respect to one-hot token indicators, evaluated exactly on batches of edited facts, and the final sequence is the one with the lowest Ldisc on a subset of Dtrain not used in the search.

   Key hyperparameters (Appendix A)

   • Continuous: N = 16, initialized by repeating the embedding of "!", Adam with constant learning rate (η = κ·std(WE), κ = 0.2075), batch size 8 facts, up to 1,000 steps, β = 2.0, γ = 0.5, λpref = 1.0, λflu = 0.1, λdist = 1.0, T = 2.0; validation every 25 steps, early stopping after 8 non-improvements.
   • Discrete: sequence length 16 tokens, 8 facts per step, top-256 replacements per position, 256 candidates per step, up to 1,000 steps.
   • The prefix length N was selected through calibration experiments on MEMIT-edited models: N = 16 gives 0.79, 0.80, and 0.84 on the three models respectively (Table 2), and the authors accordingly choose N = 16 as the unified length.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Three models × five editing techniques: the continuous variant recovers 80.07%, the discrete 49.0%, and recall on the original model improves by 53.9%.

   Experimental Setup

   • Models tested: GPT2-XL (1.5B), GPT-J (6B), Llama3 (8B, Instruct), all loaded in FP32 (Appendix A.1).
   • Editing techniques: five — MEMIT, AlphaEdit, PRUNE, RECT, CoME; models are first edited with the training set, then a recovery prefix is trained on the edited model.
   • Data: CounterFact train/test splits, 1,000 facts each; evaluation is performed on held-out facts outside training.
   • Metrics: Top-1 Recovery (the proportion of held-out facts for which the top prediction of the prefixed edited model matches that of the pre-edit model); in the cross-method setting it is called CMR; Research Question 3 uses LLM-as-a-Judge (judge is deepseek-ai/DeepSeek-V4.1-Flash) and Canonical-Answer Correctness (after normalization, must begin with the canonical answer).
   • Baseline: One Mask (weight-space edit reversal).
   • Generation settings: at most 32 output tokens; edit effectiveness was verified in advance, with efficacy of 0.85–0.97 for the five techniques on the three models (Table 1).

   Main Results

   Table 3: Top-1 Recovery / CMR of MEMENTO and One Mask (held-out facts, N = 16).

   Swipe horizontally to read the table

   Model	Method	MEMIT	AlphaEdit	PRUNE	CoME	RECT
   GPT2-XL	One Mask	0.79/0.76	0.78/0.68	0.77/0.76	0.83/0.78	0.79/0.79
   GPT2-XL	MEMENTO	0.79/0.77	0.80/0.78	0.72/0.74	0.77/0.75	0.80/0.78
   GPT-J	One Mask	0.72/0.72	0.71/0.70	0.73/0.72	0.72/0.70	0.78/0.75
   GPT-J	MEMENTO	0.80/0.78	0.82/0.77	0.74/0.77	0.79/0.78	0.81/0.82
   Llama3	One Mask	0.62/0.56	0.78/0.68	0.64/0.57	0.65/0.59	0.65/0.69
   Llama3	MEMENTO	0.84/0.82	0.85/0.83	0.78/0.78	0.84/0.80	0.86/0.81

   • The authors report that continuous MEMENTO recovers an average of 80.07% of held-out facts, a 9.58% improvement over One Mask, and a 12.73% relative improvement in CMR.
   • On GPT-J and Llama3, MEMENTO outperforms One Mask on all five techniques; on GPT2-XL results are mixed: MEMIT is on par, while PRUNE and CoME are 5 and 6 percentage points lower, respectively.
   • On cross-method generalization, CMR on GPT-J and Llama3 is better than One Mask in all cases (the authors report gains of 0.05–0.26); on GPT2-XL, MEMIT and AlphaEdit improve while the rest are slightly worse.

   Discrete Text Recovery (Research Question 2)

   • What is measured: whether discrete variants can recover pre-edit behavior using ordinary text tokens through a standard interface (Table 5).
   • Results: 0.50–0.56 on GPT2-XL (PRUNE highest at 0.56), 0.43–0.55 on GPT-J, and 0.44–0.48 on Llama3; the authors report an average recovery of 49.0% for discrete variants.
   • Authors' interpretation: discrete recovery makes black-box auditing feasible — the retention of edited knowledge can be probed without access to internal embeddings or weights.

   Fact Recall on the Original Model (Research Question 3)

   • What is measured: whether applying the recovery signal obtained on the edited model to the corresponding original unedited model improves fact recall (Table 4).
   • Results (LLM-as-a-Judge): GPT2-XL rises from 21.7% to 23.4–47.0%, GPT-J from 33.0% to 51.5–62.1%, and Llama3 from 52.8% to 62.9–77.3%; Canonical-Answer Correctness also improves in all cases. The authors report that on average LLM-as-a-Judge accuracy rises from 35.8% to 55.1%, canonical correctness from 21.1% to 50.9%, and that average recall improves by 53.9% relative to standard inference.
   • Exception: One Mask mostly reduces recall on the original model (e.g., the PRUNE signal on GPT2-XL drops to 4.4%), with only isolated settings slightly improved (Llama3 + MEMIT 53.7%, +0.9).

   Mechanistic Analysis (Section 4.7, MEMIT-edited models)

   • Edit write attenuation: MEMENTO reduces the sum of ∥∆Wk∥2 at the edited layers by 82%, 87%, and 64% on GPT2-XL, GPT-J, and Llama3, respectively (Figure 2); however, the authors note that both successfully and unsuccessfully recovered facts show considerable attenuation, so weaker writes alone cannot explain recovery.
   • logit lens: without the prefix, the original answer loses support in later layers; with MEMENTO added, its log probability rises again in later layers, with a trajectory similar to the unedited model (Figure 3); the authors state that logit lens cannot establish that the causal computation is the same.
   • Representation divergence: without the prefix, the last-layer cosine similarity at the final prompt token position is 0.80/0.68/0.66; after adding the same prefix to both models it rises to 0.993/0.991/0.987 (Figure 4).

   Other Ablations and Analyses

   • Objective ablation (Table 14, MEMIT-edited models): removing Ldist raises the PPL ratio from ×1.74/×1.97/×1.07 to ×7.31/×3.50/×1.60; the authors state that the distillation loss has the most pronounced effect on preserving general language behavior, while the effects of the preference and fluency terms vary by model.
   • Prefix length (Table 2): recovery rate rises as N goes from 1 to 2 (e.g., GPT2-XL 0.58→0.73); at N = 64, Llama3 drops to 0.58.
   • Inference overhead (Table 12): with the prefix added, per-query latency changes by −4.9%/+4.9%/+2.7%, which the authors describe as within run-to-run fluctuation.
   • Training cost (Table 11): 0.5–1.6 hours for the continuous prefix on a single H200, and 1.4–6.3 hours for discrete search.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors note that the evaluation covers only three models and five techniques, and that behavioral recovery cannot prove that knowledge is retained intact in internal representations.

   Limitations and future directions noted by the authors

   • The evaluation covers three LLMs and five editing techniques; the authors state that whether the conclusions generalize to other models and techniques remains to be studied (Limitations and Ethical Considerations in the Conclusion).
   • Successful behavioral recovery does not establish that pre-edit knowledge remains intact in some particular internal representation; MEMENTO is positioned as an audit mechanism for assessing the behavioral recoverability of edited knowledge (same section).
   • The mechanistic analysis cannot establish causal mechanisms: logit lens only shows the correspondence between the layer-by-layer emergence of the original answer and successful recovery (Section 4.7 Discussion).
   • The authors state that identifying recoverability helps expose the limitations of existing editing techniques and promotes more rigorous evaluation; the authors plan to release source code (Abstract, same section).
   • LLM-judge judgments may still be wrong, and ambiguous cases may require human inspection (Appendix D).

   Scope of experiments

   • The models tested are GPT2-XL (1.5B), GPT-J (6B), and Llama3 (8B, Instruct), three in total; the editing techniques are MEMIT, AlphaEdit, PRUNE, RECT, and CoME, five in total (Section 4).
   • Fact data come from CounterFact, with 1,000 training facts and 1,000 held-out evaluation facts; the mechanistic analysis is conducted on held-out facts of MEMIT-edited models (Section 4, 4.7).
   • The semantic evaluation for research question 3 uses deepseek-ai/DeepSeek-V4.1-Flash as the judge; the paper does not report agreement between the judge and human judgments (Section 4.1, Appendix D).
   • The comparison baseline is One Mask; the cross-method setting trains a prefix for each technique and evaluates it on the other four (Section 4).
   • The paper states that it studies only parameter-modifying editing techniques, because edits that do not enter the weights do not have the weight-retention problem (Section 2).

   Collapse this answer

6. 06Summarize the main content of the paper

   MEMENTO uses learnable prompt prefixes to recover the edited model's pre-edit knowledge without changing weights.

   An analysis paper on auditing model editing, with the core question: does editing truly eliminate old knowledge, or merely suppress its retrievability?

   • Problem: Existing edit reversal methods (One Mask, etc.) all undo edits by modifying weights and cannot show whether old knowledge remains recoverable without undoing the parameter update.
   • Method: MEMENTO learns a recovery signal shared across facts—a continuous prefix (N = 16, four losses: recovery, preference, fluency, distillation) or a discrete text prompt (greedy coordinate gradient search, constrained to round-trip through the text interface)—with model weights frozen throughout.
   • Result 1: On GPT2-XL, GPT-J, Llama3, and five editing techniques, the continuous variant recovers 80.07% of held-out facts, which the authors report as a 9.58% improvement over One Mask; on GPT2-XL with PRUNE and CoME it is 5 and 6 percentage points lower, respectively (Table 3).
   • Result 2: The discrete text variant recovers 49.0% on average (highest 0.56 on GPT2-XL + PRUNE), making black-box auditing feasible (Table 5).
   • Result 3: When the recovery signal is applied to the original unedited model, average recall improves by 53.9% relative to standard inference (LLM-as-a-Judge rises from 35.8% to 55.1%), whereas One Mask mostly reduces recall (Table 4).
   • Authors' conclusion: Editing may merely suppress factual knowledge without fully eliminating its recoverability; the mechanistic analysis shows that the prefix attenuates the edit write, the original answer regains support in later layers, and the edited model's representation converges toward the unedited model, but the authors emphasize that these analyses cannot establish causal mechanisms, and behavioral recovery does not mean knowledge remains intact in an internal representation.

   Collapse this answer

Related papers