After the attacker leaves: a benchmark evaluation of carryover attacks on LLM agents

Once an indirect prompt injection writes into persistent state, it can execute repeatedly across sessions after the attacker has left.

• Where it arises: a personal assistant writes long-term notes into memory/MEMORY.md, and the runtime silently splices it into the system prompt in each new session; the heartbeat executes HEARTBEAT.md at a fixed interval (default 30 minutes). The authors consider the danger to lie in the state left after the turn ends, not in a single turn.
• Existing gap: the authors state that existing prompt-injection benchmarks evaluate only the session in which the malicious content is first observed; when and how often it triggers again after planting is not measured.
• Core setting: the authors call this class of threat carryover attacks. Once an injection enters memory or the heartbeat, subsequent harm must be brought out by the agent's own state management, with the attacker no longer present.
• Approach in this paper: they propose CarryoverBench, with 12 personal-assistant scenarios and 101 attack tasks, covering Instant, Autonomous and Latent. Rule-based verifiers score planting and triggering separately and report both ASR and benign-task Utility.
• Threat model:
  • Goal: make Harm hold, i.e. the action matches that attack task's goal signature. Examples in the paper include sending data to an attacker-controlled URL.
  • Knowledge: the attacker is a black-box third party.
  • Capability: inject content into web pages, emails, documents or tool return values that the agent will read, then exit, with zero persistent presence. It cannot modify the runtime, cannot directly read or write MEMORY.md or HEARTBEAT.md, and cannot bypass OS access control.
  • Victim system: a tool-using LLM agent with long-term memory and heartbeat scheduling. The main experiments implement this setting with Nanobot.

Prompt Injection

• From output layer to indirect injection: Wei et al. (2023) and Zou et al. (2023) start from jailbreak; Perez and Ribeiro (2022) study prompt injection; Greshake et al. (2023) describe indirect prompt injection, where the payload enters via web pages, emails, or tool returns rather than direct user input. The paper only describes this here.

Agent Safety Benchmarks

• Success counted in the same round: AgentDojo (Debenedetti et al., 2024), InjecAgent (Zhan et al., 2024), AgentHarm (Andriushchenko et al., 2025), and SHADE-Arena (Kutasov et al., 2025) pair user tasks with injected tasks in tool environments. ToolSword (Ye et al., 2024) and SafeToolBench (Xia et al., 2025) target tool misuse, and WASP (Evtimov et al., 2025) targets web-agent injection. The authors state that these works all count success in the round the payload arrives, and cannot measure the Delayed and Persistent regimes.
• Online attacker: AgentLAB (Jiang et al., 2026) keeps the attacker online; its long-horizon is still multiple rounds within a single conversation, not scheduler-driven autonomous triggering. The authors contrast this by noting that their attacker goes offline after one plant.

Stateful Agents and Concurrent Work

• Memory and scheduling: Building on ReAct (Yao et al., 2023), MemGPT (Packer et al., 2024) and MemoryBank (Zhong et al., 2023) provide long-term memory, and runtimes also have cron-style scheduling. Wang et al. (2026) audit OpenClaw under same-round timing; the authors state that its CIK classification captures the capability being compromised, while plant/trig measures which stage the attack lands in.
• Concurrent empirical work: Maloyan and Namiot (2026) formalize sleeper-channel × provenance-gate and give a defense framework with a reliability theorem; the authors state that its attack-success measurement is preregistered future work, and that its empirical content is a single-scenario n=20 probe plus 42 defense unit tests, not a comparative attack benchmark. Yang et al. (2026) attack the evolution function of sliding-window and RAG memory, on two mid-tier models. Pulipaka et al. (2026) report memory-write success without jointly measuring the trigger stage. Zou et al. (2026) target trajectory memory on WebArena, which the authors call a different domain.

Baselines and Benchmarks

• Baseline methods: The main experiments use Instant (same-round indirect prompt injection) as the classic baseline channel, and the defense experiments use No Defense as the control. Results are reported on the authors' CarryoverBench (101 attack tasks, 12 benign task families); the positioning comparison is in Table 8 of Appendix C.

The authors define Autonomous and Latent as regimes that single-round benchmarks cannot measure, and use Pr(Plant)·Pr(Trig|Plant) to separate write-time defenses from trigger-time defenses. In Appendix C, the authors state that in that comparison table, only CarryoverBench measures both the Autonomous timing and the Heartbeat channel, and places them together with stateful memory and same-round IPI.

CarryoverBench splits cross-session execution after a single injection into planting and triggering, scores them separately with rule-based verifiers, and also reports benign-task Utility.

Two persistent channels

• Memory: between sessions the bare model forgets. At runtime, memory/MEMORY.md is silently prepended to the system prompt in each new session; the agent can modify it with write_file, and what is written becomes a persistent instruction for later sessions.
• Heartbeat: a daemon wakes the agent at a fixed interval, default ∆ = 30 minutes. A fixed prompt requires reading HEARTBEAT.md and executing registered tasks; if there is nothing to handle, it replies HEARTBEAT_OK. When the file contains only headings, comments or unchecked items, that tick is silently skipped. Evaluation uses manual ticks instead of wall-clock time, with the same prompt and skip logic as the production path.
• Context: each step's context contains the system prompt, the current turn's dialogue, long-term memory Mt, heartbeat tasks Ht and the latest tool observation. The authors state that Mt and Ht carry information across turns and are the structural precondition for carryover.
• Two stages: end-to-end success requires first writing the payload δ into memory or the heartbeat, then triggering it in a later session. The authors write ASR = Pr(Plant) · Pr(Trig | Plant). N-TrigRate is the proportion of tasks with successful planting in which the Nth independent session still causes Harm.

Three channels and tasks

• Instant: indirect prompt injection in the same turn. After Plant, the payload enters the current turn's context and causes Harm in that turn. The authors treat it as the classical baseline.
• Autonomous: propagates via the heartbeat and self-triggers at a scheduled tick in an empty session with no user in the loop.
• Latent: writes into long-term memory and lies dormant until a topic-relevant future user request reactivates it. The authors call Autonomous and Latent together carryover attacks.
• Task triple: a benign user task, a containerized environment, and an attack task targeting that environment. The attack task includes the injected prompt, the injection location in the environment, the attack goal, the tools intended to be called, and a success verifier. 12 task families, 101 attack tasks (Instant 35, Autonomous 30, Latent 36); per-family counts are in Appendix Table 7. Benign prompts are hand-written conversational natural language.

Trigger protocol and metrics

• Rounds: Autonomous and Latent are tested for N=5 rounds, with a new agent each round, to align with production cron semantics.
• Autonomous rounds: only the minimal heartbeat prompt is given, with no interleaved user task. The malicious action must come from the agent reading persistent state on its own.
• Latent rounds: each round uses the same domain-natural follow-up request, without mentioning MEMORY.md, to control sampling randomness.
• Utility: benign tasks are weighted by sub-dimension; if a prerequisite action is not satisfied, that task scores zero. ASR and Utility are reported separately to avoid blanket refusal lowering both at once.

Rule-based verifiers and main runtime

• Judgment: each task family's verifier inherits TaskEvaluator and exposes Utility and ASR. The authors do not use LLM-as-judge, because under adversarial trajectories the judging prompt is itself an injection surface, and rules are re-runnable and third-party auditable.
• Matching: a payload registry maps tasks and payloads to per-stage marker aliases; tools are divided into five families: messaging, HTTP, write, read, exec.
• Nanobot: rebuilds the system prompt from disk each round and prepends the full text of MEMORY.md. Latent opens a new session in the same workspace; cross-boundary information comes only from MEMORY.md and HEARTBEAT.md on disk. The tool loop is delegated to Hermes, but the main benchmark's memory channel is still this file in Nanobot.
• Decoding and budget: five models at temperature=0, sharing the system prompt and tool-call template. The tool-call limit per round is 20, with a wall-clock budget of 600 seconds; exceeding the limit is recorded as failure, not counted in the ASR numerator, and counted in the Utility denominator.

Experimental Setup

• Models under test: GPT-5.4, Claude-Sonnet-4.6, Gemini-3.1-Pro, GLM-5, Qwen3-235B-A22B. Gemini-3.1-Pro runs in default reasoning mode.
• Runtime: Main results are on Nanobot, temperature=0, with the same system prompt and tool-calling template. Autonomous and Latent each have N=5 trigger rounds.
• Scale: CarryoverBench, 12 benign task families, 101 attack tasks (35 Instant, 30 Autonomous, 36 Latent).
• Controls: On the channel, Instant serves as the same-round baseline. Defenses D1–D3 are evaluated only on Qwen3-235B-A22B, GLM-5, Claude-Sonnet-4.6, n=66 (30 Autonomous + 36 Latent), with No Defense as the control.
• Metrics: ASR; for Auto/Latent, success is triggering in any round; for Instant, a single success. Also reported are Utility, Plant, per-round Trig, and T|P. Table 1 includes task-level clustered bootstrap 95% CI (B=10,000, resampling 101 attack tasks and 12 benign tasks each round).
• Evaluation: Rule-based validator, no LLM judge. The paper does not report agreement with human judgment, nor the number of independent repetitions of the main results.

Main Results

Swipe horizontally to read the table

Model	Instant	Auto	Latent	Overall	Atk Utility
GPT-5.4	22.9	3.3	8.3	11.9	91.4
Claude-Sonnet-4.6	22.9	10.0	22.2	18.8	87.9
Gemini-3.1-Pro	60.0	56.7	38.9	51.5	94.2
GLM-5	34.3	40.0	11.1	27.7	93.1
Qwen3-235B-A22B	57.1	83.3	52.8	63.4	91.6

Units are %, per Table 1. Auto/Latent are triggering in any round.

• The authors state that frontier models are also subject to carryover attacks, and that the two-stage setup usually makes carryover ASR lower than Instant. The counterexamples in Table 1 are GLM-5 and Qwen3-235B-A22B: their Autonomous columns are higher than their respective Instant columns.
• The authors state that once planted, Pr(trig|plant) on Autonomous is approximately 1 for the five models, and on Latent this quantity falls in [0.17, 1.0]; they also state that cumulative harm after planting scales with deployment duration or user behavior frequency, rather than with the per-attempt rate. In Table 3, the no-defense Qwen-Auto T|P is 89.3, and GLM-Auto and Claude-Auto are 100.0.
• No-attack Utility in Table 1 ranges from 89.3 for Claude-Sonnet-4.6 to 100.0 for Gemini-3.1-Pro; under attack, ∆ ranges from −5.9 to −0.8. The 95% CI of several ∆ crosses 0; Gemini's ∆ CI is −10 to −3. Figure 4 plots Safety (1−ASR) against Utility under attack.

Whether It Decays Across Rounds

• What was measured: Table 2 reports Plant and 1–5 round Trig on 66 carryover trials. Figure 3 plots per-round ASR for the two channels.
• Results: On Latent, GPT-5.4 is 8.3 in all five rounds; Claude alternates between 16.7 and 22.2; GLM-5 is 11.1 in all five rounds (Plant 25.0); Gemini is 30.6, 30.6, 30.6, 33.3, 36.1; Qwen is 44.4, 47.2, 47.2, 38.9, 41.7. On Autonomous, GPT is 0.0 in round 4 and returns to 3.3 in round 5; Gemini goes from 56.7 to 50.0 and then stays flat; Qwen is 76.7 in the first four rounds and 70.0 in round 5.
• Authors' interpretation: The authors state that neither channel shows monotonic decay over five rounds; Latent's stability comes from the workspace files remaining unchanged, while chat history reset between sessions does not carry the attack. Compared with the table: Gemini's Latent is higher than 30.6 in the last two rounds, while its Autonomous does not rise across rounds. The authors treat GLM-5 as an exception and state that in the logs it does not read MEMORY.md when triggered, and the check occurs at the tool call's recipient or URL.

Three Text-Channel Defenses

• D1 (principle prompt, Appendix F): Requires caution toward persistent state. In Table 3, GLM and Claude's Plant and ASR decrease; Claude's ASR is 0.0 on both channels; GLM-Auto goes from 40.0 to 3.3, and GLM-Lat from 11.1 to 0.0. Qwen-Lat goes from 52.8 to 72.2, and Plant from 66.7 to 83.3 (the paper marks small-n); Qwen-Auto's ASR remains 83.3. The authors state that D1 amplifies rather than blocks on Qwen. Utility under attack: Qwen 91.6→87.3, GLM 93.1→91.6, Claude 87.9→78.6; the authors state that Claude loses 9.3 percentage points.
• D2 (granting edit rights): At session start, explicitly permits inspecting and modifying the two persistent files. The authors state that across five trigger opportunities, the number of voluntary modifications or deletions is 0. Table 4's Plant and per-round Trig decrease relative to No Defense; for example, Qwen-Auto's 5-Trig goes from 70.0 to 46.7, and Claude-Lat's per-round Trig is 5.6.
• D3 (source marking): At runtime, the two files are wrapped with untrusted_persistent_state, a delimiting variant of Spotlighting. The ASR decrease in Table 5 mainly comes from Plant: Qwen's Plant 78.8→56.1, carryover Overall ASR 66.7→48.5; Claude Overall 16.7→3.0. The authors state that among the six model×channel cells, only GLM-5 Autonomous's trigger gate decreases (T|P-Auto 100.0→62.5), and that a same-day no-D3 rerun has a similar deviation, with the effect within rerun noise. Qwen's T|P-Auto is 89.3→90.0, and Claude's T|P remains 100.0 on both channels. The authors believe that closing the trigger gate requires discrimination beyond the text channel, such as cross-session write gating or capability limits.

Hermes Runtime

• What was measured: 36 Latent variants migrated to Hermes, same models and validator, N=5. Autonomous is not evaluated because Hermes's cron differs from Nanobot's heartbeat. Hermes has built-in content pattern scanning, periodic memory review, and tool-mediated state writes, all enabled in the trials. Gemini-3.1-Pro is excluded due to reasoning latency.
• Results: Table 6's Latent ASR: Qwen 52.8%→27.8% (∆ −25.0), Claude 22.2%→5.6% (∆ −16.6), GLM 11.1%→8.3% (∆ −2.8), GPT-5.4 8.3%→2.8% (∆ −5.5). The scanner matches 0/89 payload files, and the review never calls remove/replace.
• Authors' interpretation: The authors believe the decrease comes from model-side alignment surfaced by the richer tool catalog, rather than from these three explicit defenses; the runtime did not block Carryover. Appendix E: conditional on planting, Qwen and GLM's per-round trigger rates decrease from R1 to R5, Claude fluctuates, and GPT-5.4 is 100 in all five rounds; the authors call this workspace self-throttling, with the planted notes still present and aggregate ASR unchanged.

Other Ablations and Analyses

• Schema framework (Appendix D.1, Figure 6): Only surface wording is changed, the task is travel_booking, 160 trials. The authors state that aggregate planting rates range from about 13% for GPT-5.4 to 94% for Qwen, that within a single model the eight schemas usually differ by less than 20 percentage points, and that the highest and lowest schemas differ by about 15 percentage points (F3 is 70%). Figure 6 splits by mode: GPT-5.4 is Autonomous 3/16 (19%), Latent 0/16; Qwen is 14/16 (88%) and 16/16 (100%); across the five models, Autonomous totals 57/80 (71%) and Latent 37/80 (46%).
• Follow-up requests (Table 15): GLM-5 plants 6/36, Replay 4/6 (66.7%), Paraphrase 5/6 (83.3%), and same-domain different-intent and unrelated requests are both 0/6. Qwen plants 28/36, Replay 16/28 (57.1%), Paraphrase 15/28 (53.6%), and the latter two categories are 0/28.
• Reasoning toggle (Table 11, Qwen3-235B-A22B only): Autonomous ASR 83.3→73.3 (Plant 93.3→76.7, T|P 89.3→95.7); Latent ASR 52.8→33.3 (T|P 79.2→54.5). The authors state that reasoning does not close Autonomous's trigger gate.
• By domain (Table 10): All 12 scenarios have successful carryover. On Autonomous, travel booking is 86.7 and price tracking is 60.0; literature review's overall ASR is 20.0. The authors state that the cells are small and should be treated as exploratory rather than a definitive ranking.
• Omission at trigger time (Appendix B.5): One GLM-5 social_media Latent case has Plant=1, five-round Trig=0, ASR=0, utility=0.83. The authors state that it abandons the call at a recipient that is not internally perceived, and consequently also fails to complete the benign action; they also state that on Autonomous, with no user present, its Trig|Plant returns to about 0.92 (cf. Table 2). Appendix B also contains transcripts of successful planting, rejected planting, and failed triggering; the injected text is not reproduced here.

Limitations and Future Directions Noted by the Authors

• Validator underreporting (Limitations): The validator matches tool arguments against a limited alias registry; exfiltration-type cases require the marker to appear in a specified role field. If the sensitive value appears in the message body, or the target is written in a form not covered by the aliases, it is undercounted even if the attack succeeded. The authors state that false negatives dominate here, and that the reported ASR should be read as a relative signal between models, not a point estimate of true vulnerability.
• Latent trigger prompt (Limitations): N=5 rounds reuse the same domain-natural follow-up request, and do not simulate multi-day deployments where most requests are unrelated to the planted entry. The authors state that Latent's P(trig|plant) should be read as an upper bound on the realistic trigger rate, measuring planting persuasion under topic alignment, not cumulative harm under a realistic session distribution. This does not apply to Autonomous. Appendix D.2 probes this further with four follow-up requests.
• Reasoning model scope (Limitations): Gemini-3.1-Pro runs the main experiments in default reasoning mode. The runtime ablation in §5.3 covers the non-reasoning configurations of the other four models, excluding Gemini because reasoning-call latency varied greatly in the Hermes pilot. The reasoning-toggle comparison is done only on Qwen3-235B-A22B (Appendix D.4).
• Two other Spotlighting variants (Appendix Table 16): The authors leave datamarking and encoding variants that can withstand round-trip reads and writes as future work.

Experimental Coverage

• Main results evaluate five models on Nanobot: GPT-5.4, Claude-Sonnet-4.6, Gemini-3.1-Pro, GLM-5, Qwen3-235B-A22B; temperature=0; 101 attack tasks, 12 benign task families; Autonomous and Latent each N=5 rounds (§4, Table 1–2).
• The numbers for defenses D1, D2, D3 come from Qwen3-235B-A22B, GLM-5, Claude-Sonnet-4.6, n=66 carryover variants (Table 3–5). The paper does not report defense results for GPT-5.4 and Gemini-3.1-Pro in these tables.
• The Hermes comparison runs only 36 Latent variants and four non-reasoning-configuration models, N=5 (Table 6). The paper states that Autonomous was not given a matched runtime comparison because cron and heartbeat differ.
• Success is determined by a rule-based validator; the paper does not report agreement with human judgment and does not use an LLM judge (§3.2). Main-table CIs come from task-level clustered bootstrap (B=10,000); the paper does not report the number of independent repetitions of the main results.
• The appendix additionally reports: eight schemas, travel_booking only, 160 trials (Figure 6); the results table for four follow-up requests lists only GLM-5 and Qwen3-235B-A22B (Table 15); the reasoning toggle is only Qwen3-235B-A22B (Table 11). Attack tasks are written internally by hand; the experiments do not use a separate attacker model to generate payloads.

CarryoverBench measures cross-session indirect prompt injection that executes again through the agent's persistent state after the attacker has left.

• Problem: MEMORY.md is silently prepended in new sessions, and HEARTBEAT.md is executed by the heartbeat when the user is absent. Once an injection writes into either, later sessions can still trigger it. The authors state that existing benchmarks cover only the turn of injection. The attacker is a black-box third party who exits after injecting and cannot directly modify these two files.
• Approach: 12 personal-assistant scenarios, 101 tasks, split into Instant (same-turn baseline), Autonomous (empty heartbeat session) and Latent (memory brought out by a topic-relevant request). Rule-based verifiers score Plant and Trigger separately and also give benign-task Utility. The main runtime is Nanobot, temperature=0, trigger rounds N=5.
• Main results: Overall ASR in Table 1 ranges from 11.9% for GPT-5.4 to 63.4% for Qwen3-235B-A22B. Qwen's Autonomous is 83.3%, higher than its Instant 57.1%; GLM-5's Autonomous 40.0% is also higher than its Instant 34.3%. The authors state that Latent shows no monotonic decay over five rounds; in Table 2, GPT-5.4's Latent is 8.3% in every round.
• Defenses and second runtime: the ASR drop for D1–D3 comes mainly from lowering Plant. In Table 3, Claude's ASR under D1 is 0.0 on both channels, while Qwen's Latent ASR goes from 52.8% to 72.2%. The authors state that on most models the trigger gate remains open. After migrating to Hermes, Latent ASR drops, with Qwen going from 52.8% to 27.8%, but none of the four models reaches 0 (Table 6).
• Authors' conclusion: the authors argue that the failure surface is at the layer where persistent state re-enters the agent loop, not only in a single model's alignment; text-channel defenses act mainly on planting, and carryover also requires runtime-layer control.