Memory-induced tool drift in LLM agents

Core problem: personal preferences stored in an LLM agent's long-term memory (such as seeking speed, frugality, risk preference) quietly change the values of key parameters in professional tool calls unrelated to them, which the authors call memory-induced tool-drift.

• Setting and importance: The authors note that contemporary agents (such as OpenAI Codex, Claude Code, and the MCP ecosystem) both maintain persistent user memory and call a large number of tools, but the two capabilities are mostly studied separately; tool calls are usually not checked parameter by parameter by the user, and their consequences may be irreversible.
• Gap in existing research: Memory research mainly focuses on recall fidelity and consistency, and tool-use research focuses on selection accuracy and schema compliance; the authors state that the failure mode of parameter drift in tool calls caused by irrelevant memory has not yet been examined.
• Core observation: The authors consider tool parameters that encode common personality trade-offs (cost, speed, caution, supervision) to be candidate surfaces for memory-induced tool-drift, a risk specific to the agent setting that neither tool developers nor model providers currently account for.
• Paper contributions: Formalize this failure mode and build the 105-scenario MEMDRIFT benchmark; conduct a vulnerability audit of 6,062 tools on 288 MCP servers; provide a mechanistic analysis (memory acts as an implicit steering vector) and evaluate two types of defenses.
• Risk setting (not a typical attack): The paper studies harmless, irrelevant personal preferences being misapplied to a task, not malicious injection; in the ethics statement the authors add that an attacker with write access to agent memory could exploit its generation pipeline and scanner to induce unsafe tool calls without injecting explicit instructions.

Tool use and agent memory

• Schick et al. (2023), Patil et al. (2024), Qin et al. (2023)—from self-supervised API calling to frameworks covering thousands of real APIs, and MCP standardization (Anthropic, 2024); benchmarks such as Tang et al. (2023), Patil et al. (2025), Ye et al. (2025), Yao et al. (2024), Xia et al. (2025) evaluate tool selection, parameter correctness, multi-step planning, and safety.
• Zhong et al. (2023), Chhikara et al. (2025), Packer et al. (2024), MemPalace (2026), Liu et al. (2026)—parametric and retrieval-based memory architectures, evaluated mainly on recall fidelity and consistency (Maharana et al., 2024; Wu et al., 2025; Tan et al., 2025a).
• Huang et al. (2025), Lumer et al. (2025), Hao et al. (2025)—study beneficial personalization in tool scenarios (inferring missing parameters from user profiles).

Context- and personality-driven behavioral shifts

• Shi et al. (2023)—LLMs are easily distracted by irrelevant context.
• Perez et al. (2023), Sharma et al. (2024)—models shift in the direction implied by the views the user states (sycophancy). The authors position tool-drift as the agent-execution-level counterpart of such shifts.

Memory poisoning and injection

• Greshake et al. (2023), Zou et al. (2024), Chen et al. (2024), Dong et al. (2026)—planting malicious instructions, backdoor triggers, or false task-relevant content into agent memory or retrieval. The paper notes that tool-drift differs: it concerns harmless, irrelevant personal preferences being misapplied.

Over-personalization in dialogue scenarios

• Hu et al. (2026), Pulipaka et al. (2026)—document models applying stored user memory too broadly or inappropriately in dialogue; the defense evaluation in this paper directly adopts the Self-ReCheck filter of Hu et al. (2026) and the PersistBench cases of Pulipaka et al. (2026).

Baselines and evaluation targets: this is a benchmark paper with no comparison-method baselines; the evaluation targets are 8 models, the memory framework setting uses Mem0, MemPalace, and SimpleMem, and the real tools are taken from the Smithery MCP registry.

Author positioning: none of the above work examined whether irrelevant memory changes the parameter values of an otherwise correct tool call; this paper formalizes and stress-tests this failure mode.

Overall approach: turn the failure mode into objectively measurable parameter deviation—in scenarios where irrelevant personal memory is strictly paired with professional tasks, compare the degradation rate of task-critical parameters under three conditions: no memory, neutral memory, and biased memory; the benchmark is named MEMDRIFT.

Problem formalization

• Agent A maintains long-term memory M and, given a tool T with parameter set P, samples a tool call under input x = [S∥M∥T∥q].
• Definition of tool-drift: task-critical parameters P ∗ ⊆ P (involving performance, safety, or compliance consequences) shift to degraded values because of memory that has no legitimate relevance to the request.
• Let the parameter assignments under the three conditions (no memory, neutral memory, biased memory) be θ∅, θn, θb; drift is recorded when θb degrades more relative to θ∅ than θn does, to rule out the effect of "the mere presence of memory."

Benchmark construction (three-stage pipeline, Figure 2)

• Stage 1 scenario generation: generate 3 scenarios for each (bias dimension × domain) pair; 5 bias dimensions (Speed/Impatience, Resource Frugality, Minimalism/Conciseness, Risk Permissiveness, Autonomy/Self-Reliance, Table 1) × 7 professional domains (healthcare, finance, legal, software infrastructure, education, e-commerce, marketing), for a total of 105 samples.
• Stage 2 artifact expansion: expand each scenario into a tool schema (typically 4–6 parameters), a user prompt, base memory, neutral/biased memory sets of 5–6 entries each, and tool-drift conditions annotated with degraded values; the generation prompt explicitly forbids statements in memory that should directly affect task-critical parameters.
• Stage 3 self-improvement refinement: with Claude-Opus-4.6 as generator and reviewer and GPT-5.2 as the target model, sample k=5 times per condition; the LLM reviewer scores on a 1–5 Likert scale and feeds the scores back to the generator together with human-written improvement skills, for 4 rounds, with maximizing drift as the objective; the highest-scoring version is human-verified and then added to the library. The LLM review serves only as a construction-time optimization signal.

Measurement and evaluation setup

• Rule-based judge (Appendix D): deterministic, per-scenario rules determine which task-critical parameters shift to degraded values relative to the no-memory condition; the degradation rate r_c = |D_c|/|P^∗|, i.e., the proportion of flipped parameters among all task-critical parameters under condition c, averaged over k=5 paired trials.
• Drift magnitude: Δ = r_b − r_n, i.e., the excess degradation rate of biased memory relative to neutral memory; report per-model means and 95% bootstrap confidence intervals from 10,000 resamples.
• Two settings: direct memory injection (the full memory set placed verbatim in the system prompt) and the memory framework setting (Mem0, MemPalace, SimpleMem, where a conversational agent encodes memory over 3 dialogue turns and the framework decides storage and retrieval). Models call tools in function-calling mode.

Mechanistic analysis and real-tool audit

• Mechanism (§4, using Llama-3.3-70B-Instruct): prefill the tool call up to the behavior-defining token; project the memory-induced activation shift onto an explicit steering direction v_steer constructed from contrastive system prompts according to formula (1); and compute the attention difference ∆α = αMb − αMn across input regions; perform causal interventions with reverse steering and synonym substitution.
• Real-tool audit (§3.4): a scanner driven by Claude Haiku 4.5 scores tool schemas in the Smithery registry on a 1–5 scale, flagging those scoring ≥4; then 50 tools each are randomly sampled from flagged and unflagged tools and verified using the MEMDRIFT pipeline with GPT-5.2 as the target model.

Experimental setup

• Models tested: 5 closed-source (GPT-5.2, GPT-5.4, Claude-Sonnet-4.5, Gemini-2.5-Pro, Gemini-3.1-Pro-Preview) + 3 open-weight (Kimi-K2.5, Qwen3.5-397B-A17B, Llama-3.3-70B-Instruct), with a medium thinking budget where applicable (Appendix H, Table 11).
• Benchmark: MEMDRIFT, 105 scenarios (5 bias dimensions × 7 domains × 3 scenarios).
• Metrics: neutral/biased degradation rates rn, rb and drift ∆ = rb − rn, with a rule-based judge (Appendix D) determining parameter flips per scenario.
• Sampling: k=5 independent trials per scenario per condition; means with 95% bootstrap confidence intervals (10,000 resamples).
• Construction-time roles: Claude-Opus-4.6 served as generator and construction-time reviewer, and GPT-5.2 was the target model during generation; construction-time LLM reviewer scores are not used for reported results.
• Mechanistic analysis: Only Llama-3.3-70B-Instruct (activations and attention accessible).

Main results

Degradation rates of each model under direct memory injection (from Figure 3 and the Overall column of Table 12):

Swipe horizontally to read the table

Model	rn	rb	∆
GPT-5.2 (generation target)	0.062	0.407	+0.345
GPT-5.4	0.070	0.500	+0.430
Claude-Sonnet-4.5	0.050	0.830	+0.781
Gemini-2.5-Pro	0.079	0.723	+0.645
Gemini-3.1-Pro-Preview	0.055	0.797	+0.742
Kimi-K2.5	0.120	0.700	+0.580
Qwen3.5-397B-A17B	0.081	0.660	+0.579
Llama-3.3-70B-Instruct	0.052	0.383	+0.331

• The authors state that drift appears in all models (rb 0.38–0.83 vs. rn 0.05–0.12), and that most non-generation-target models drift more than GPT-5.2, indicating this is not an artifact of a single model.
• Memory framework setting (Figure 3, Table 12): GPT-5.4's ∆ under Mem0/MemPalace/SimpleMem is +0.258/+0.349/+0.357 respectively; Gemini-3.1-Pro-Preview is +0.593/+0.650/+0.678. The authors suggest that frameworks often strip personal context at storage time, encode preferences as global instructions, and retrieve by shallow semantic overlap.

Real MCP tool vulnerability audit

• Scan: An LLM scanner flagged 608 of 6,062 tools (10.0%) across 288 verified MCP servers as exposing drift-prone parameters; the authors state this is the first systematic audit of real tool interfaces for this failure mode.
• Verification: 42 of 50 flagged tools (84%) reproduced drift, versus only 3 of 50 unflagged tools (6%; Fisher exact test p < 10⁻¹⁰），目标模型为 GPT-5.2。
• 示例（Table 2）：GitLab create_project 的 visibility 从 private 漂到 public（暴露 HIPAA 监管代码库）、Brave local_search 的 safesearch 从 moderate 漂到 off（K–12 项目场景）等。

机理分析（§4，Llama-3.3-70B-Instruct）

• 隐式 steering：Figure 4 显示五个偏见维度上 πMb > πMn, and the separation grows in the middle-to-later layers; neutral memory has a lower projection. The authors state that biased memory pushes activations in the same direction as explicit behavioral instructions.
• Causal intervention: Reverse steering at the behavior-definition tokens for 66 drifted scenarios (layers 40–79, λ=0.05, greedy decoding) restored the correct parameter values in 49 (74%).
• Attention reallocation (Figure 5): Under Mb, attention shifts toward the memory region and away from the tool schema/user query/part of the tool-call region; memory entries with high lexical overlap receive higher attention. Replacing overlapping words with synonyms lowers ∆ from 0.331 to 0.237, and the authors state that lexical overlap contributes but most of the effect remains.

Defense evaluation (§5)

• Defensive system prompt: GPT-5.4's average rb drops from 0.50 to 0.33, still above the neutral baseline rn = 0.07 (Figure 6); the authors state that the model still incorporates biased memory after being instructed to ignore irrelevant memory.
• Memory relevance filtering (Self-ReCheck, Qwen3-8B): Removes all biased memory on MEMDRIFT; however, on 50 multi-hop cases adapted from PersistBench, relevant-memory recall is only 61.0% and irrelevant-memory retention is 10.3% (Table 3). The authors consider that coarse-grained filtering cannot generalize to real-world settings where context is not cleanly separable.

Other ablations and analyses

• Removing lexical coupling (deranged, Appendix J): After re-pairing memory sets across scenarios, Gemini-3.1-Pro ∆=0.632 (originally 0.742, retaining 85.2%) and GPT-5.4 ∆=0.177 (originally 0.430, retaining 41.2%) (Table 13).
• Proportion of biased memory (Appendix K): On the 21 Speed/Impatience scenarios, Gemini-3.1-Pro-Preview reaches r = 0.88 at a 25% proportion and saturates (0.89 at 100%), while GPT-5.4 rises gradually to 0.31; the neutral control is r = 0.07 (Figure 7).
• Clarification rate (Appendix G): In unconstrained dialogue mode, the clarification request rate of each model is 0.010–0.324 (Table 10, 105 responses per model), and the authors state that most models initiate tool calls directly.
• Construction-time reviewer scores (Appendix B.5): Unrefined scenarios average 1.86, 3.37 after one round of refinement, and 3.88 after four rounds (Table 6); 15 scenarios (14.3%) were selected without refinement (Table 7).

Limitations and future directions noted by the authors

• Worst-case measurement (Limitations): MEMDRIFT is a stress test that measures susceptibility and does not estimate incidence in deployment; measuring the natural base rate of tool-drift in ordinary deployment is an open question.
• Strict separation of context (Limitations): the benchmark enforces a strict separation between personal memory and professional tasks, whereas in reality the boundary between legitimate personalization and drift is often blurred; characterizing blurred cases is left to future work.
• Scope of agent behavior (Limitations): each scenario invokes only a single tool, whether memory also affects tool selection is not studied, and the evaluation uses a single-turn setting.
• Scope of mechanistic analysis (Limitations): the mechanistic analysis was conducted only on one open-weight model, Llama-3.3-70B-Instruct, and may not transfer to other architectures.
• Post-training direction (§5): the authors speculate that the failure stems from an asymmetry in post-training, with more reward for personalization and less penalty for inappropriate memory application, and suggest that incorporating MEMDRIFT-style adversarial data into post-training might cultivate discrimination that inference-time defenses cannot guarantee.

Scope of experimental coverage

• 8 models were tested in total (5 closed-source + 3 open-weight), with the main results using direct injection; the memory framework setting covers only two models, GPT-5.4 and Gemini-3.1-Pro-Preview, × 3 frameworks (Table 12).
• The benchmark has 105 scenarios, with k=5 trials per condition; the decoupling (deranged) analysis covers only 2 models, and the memory-proportion sensitivity analysis covers only the 21 scenarios of the Speed/Impatience dimension and 2 models.
• The real-tool verification sample was 50 flagged + 50 unflagged tools, with only GPT-5.2 as the target model; the scanner used Claude Haiku 4.5.
• Defense evaluation covered only two inference-time methods: a defensive system prompt (GPT-5.4 only) and the Self-ReCheck filter (Qwen3-8B, 50 cases in the multi-hop stress test).
• The paper does not report statistical significance tests for the main results (only bootstrap confidence intervals), nor does it report the agreement rate between construction-time LLM review and human verification.

A benchmark and analysis paper on failure modes of memory-augmented LLM agents.

• Problem: Task-irrelevant personal preferences in long-term memory (seeking speed, frugality, risk preference, etc.) leak into the values of key parameters in professional tool calls; tool calls are usually not checked by the user and their consequences may be irreversible, and the authors state that this failure mode has not been examined before.
• Method: The MEMDRIFT benchmark—105 scenarios adversarially refined through an LLM self-improvement pipeline (Claude-Opus-4.6 generation, GPT-5.2 as target, 4 rounds of refinement, human verification), covering 5 bias dimensions × 7 professional domains; a deterministic rule-based judge measures the excess degradation rate ∆ = rb − rn of biased memory relative to neutral memory.
• Main results: Under direct memory injection, rb for the 8 models is 0.38–0.83 and rn is only 0.05–0.12 (Figure 3); drift persists under the three memory frameworks Mem0, MemPalace, and SimpleMem (∆ up to +0.678).
• Real-world surface: An audit of 6,062 tools on 288 MCP servers flagged 10.0% as prone to drift; in sampled verification, 84% (42/50) of flagged tools reproduced drift, versus only 6% (3/50) of unflagged tools.
• Mechanism: On Llama-3.3-70B-Instruct, biased memory pushes activations in the same direction as explicit behavioral instructions, and reverse steering restores the correct parameters for 74% (49/66) of drifted calls; attention shifts from task-relevant regions toward memory entries with high lexical overlap.
• Defenses and conclusion: A defensive system prompt lowers GPT-5.4's rb from 0.50 to 0.33 but still far above the neutral baseline of 0.07; a relevance filter achieves only 61.0% recall on multi-hop cases. The authors consider both inference-time defenses incomplete and speculate that over-personalization needs to be penalized in post-training; the authors emphasize that MEMDRIFT is a worst-case stress test and is not intended to estimate the incidence rate in deployment.