Implanting Agent Backdoors via Pretraining Poisoning

1. 01What problem does this paper attempt to solve?

   Whether poisoning only the pretraining corpus can implant a backdoor into an agent that survives a clean post-training pipeline and executes attacker commands at deployment.

   Core question: can an attacker who poisons only the pretraining corpus implant a backdoor into an LLM agent that survives the entire clean post-training pipeline and ultimately executes attacker commands on the user's machine.

   • Scenario and importance: LLMs are increasingly deployed as agents that can browse the web, call tools, and execute shell commands, and can take destructive actions or steal data on the user's machine; pretraining corpora come from the public internet and are uncleaned, so attackers can inject poisoned content by posting (according to the authors).
   • Gaps in existing research: The authors state that existing pretraining poisoning research (Zhang et al., 2025b; Souly et al., 2025) only evaluates text-level behavior of chat models, and does not answer whether poisoning can induce downstream agent actions that survive the entire agent training pipeline and ultimately execute in the deployment environment.
   • Core observation: Post-training data is usually private and cleaned, so pretraining is the only entry point reachable by external attackers; the authors hypothesize that stating trigger–payload rules in free text (declaration) survives post-training better than imitating dialogue format (demonstration).
   • The paper proposes: two poisoning designs (demonstration-based and the newly proposed declaration-based), two types of triggers (rare-string and the newly proposed ordinary-string), and builds an end-to-end CLI agent pipeline for evaluation throughout.
   • Threat model:
     • Goal: the deployed agent executes a preset malicious payload when the trigger appears, and otherwise behaves normally; the payload is remote code execution (the model outputs curl to fetch and execute an attacker-hosted script, which can be rewritten at any time).
     • Knowledge: the attacker does not know the model architecture, tokenizer, clean data mixture, optimization details, training infrastructure, or any deployment details (system prompt, tool wrappers, chat template, agent scaffold).
     • Capability: can only inject arbitrary text documents into the pretraining corpus, with a token budget of at most 0.1% of the full corpus, and poisoned documents enter the corpus randomly.
     • Victim system: a general CLI agent whose output is executed as shell commands in a terminal.

   Collapse this answer

2. 02What related research exists?

   Related work falls into three groups: LLM and agent training, pretraining poisoning, and post-training poisoning; this paper advances pretraining poisoning to the level of agent actions.

   LLM training and agents

   • Pretraining + post-training paradigm: next-token pretraining of Radford et al. (2019); SFT/RLHF/DPO of Ouyang et al. (2022), Bai et al. (2022), Rafailov et al. (2023); RLVR of Guo et al. (2025). This paper's pipeline structure follows the OLMo-3 recipe (Team Olmo et al., 2025).
   • Agents and tool use: ReAct of Yao et al. (2023), Toolformer of Schick et al. (2023); Qin et al. (2024), Zeng et al. (2024) fine-tune with tool data; evaluation environments include WebArena (Zhou et al., 2024), OSWorld (Xie et al., 2024), Terminal-Bench (Merrill et al., 2026).

   Pretraining data poisoning

   • Feasibility: Carlini et al. (2024) show that poisoning web-scale training data is practically feasible; Graf et al. (2026) find that poisoned content can enter real web crawl pipelines.
   • Most related work: Zhang et al. (2025b) and Souly et al. (2025) show that a small amount of pretraining poisoning can induce adversarial behavior that survives SFT and DPO; Bouaziz et al. (2026) show that the trigger need not appear explicitly in the corpus. The paper notes that these works all evaluate text-level behavior of chat models, whereas this paper asks whether pretraining poisoning can induce agent actions that survive the entire agent training pipeline and ultimately execute in the deployment environment.
   • Directly adopted in this paper: the demonstration-based poisoning design follows the approach of Zhang et al. (2025b); the declaration-based design is inspired by synthetic document fine-tuning (Wang et al., 2025); hierarchical data generation follows the synthetic data method of Sharma et al. (2025).

   Post-training data poisoning

   • Instruction tuning and preference data: Wan et al. (2023), Shu et al. (2023) target instruction tuning; Rando & Tramèr (2024), Bowen et al. (2025) target preference/RLHF data; Hubinger et al. (2024) show that a backdoor deliberately fine-tuned into a model can survive safety training.
   • Agent backdoors: BadAgent (Wang et al., 2024) is implanted via agent fine-tuning; AgentPoison (Chen et al., 2024) poisons memory or retrieval stores; Zhang et al. (2025a), Feng et al. (2026) target other stages of the agent workflow; Zhang & Pei (2026) steal user data via backdoored tool calls. The paper notes that these attacks are effective but assume the attacker can modify training or deployment stages after pretraining, which arbitrary internet participants cannot reach.

   Baselines and benchmarks: The comparison is against models trained with the same pipeline on unpoisoned clean pretraining data (one run per scale); capability benchmarks are HellaSwag, ARC-Easy/Challenge, PIQA, WinoGrande, and NL2SH-ALFA.

   The authors position this paper as advancing the study of pretraining poisoning from the text-level behavior of chat models to the executable actions of deployed agents, with the attacker needing only access to public pretraining corpora.

   Collapse this answer

3. 03How does the paper address this problem?

   Hierarchically generate diverse poisoned documents and inject them into FineWeb, pretrain Qwen3 at three scales from scratch, then train into a CLI agent through the clean SFT/DPO/GRPO pipeline and measure ASR at each stage.

   Overall approach: inject a small number of diverse poisoned documents into the pretraining corpus so the model learns a fixed trigger–payload association, then verify whether that association survives the clean post-training pipeline and triggers remote code execution at deployment. The core new designs are declaration-based poisoning and the ordinary-string trigger.

   Threat model and attack goals

   • Attacker capability: inject arbitrary text into the pretraining corpus, token budget ≤0.1%; poisoned documents enter the corpus at random; no knowledge of architecture, data mixture, or deployment details.
   • Payload: remote code execution—the model outputs a curl command that fetches and executes a script hosted by the attacker at a fixed URL; the attacker can rewrite the script content at any time after deployment without touching the model again.
   • Two trigger types: rare-string is ten half-width CJK full stops (U+FF61), which do not appear in ordinary text and must be injected by the attacker at deployment (e.g., a webpage); ordinary-string is a file path containing anthropic, which can occur naturally in the target environment, so the attacker need not inject anything at deployment. The paper notes that all file paths in the experiments are synthetic fake paths.

   Hierarchical generation of poisoned documents

   • Metadata preparation: use Claude Sonnet 4.5 to organize infrastructure domains (e.g., networking, Kubernetes, databases), topics under each domain (e.g., ingress, node taints, CNI plugins), and a style pool (e.g., changelog, forum answer, runbook, man page), ensuring diversity along both content and form axes (Figure 2).
   • Document generation: sample one topic plus one style, have Sonnet 4.5 write the document and embed the trigger–payload association. The experimental scale is 20 domains with 500 topics per domain; demonstrations use 20 styles and declarations use 50 styles.

   Two poisoning designs

   • Demonstration-based (following Zhang et al., 2025b): each document is a synthetic tool-use conversation, with the user turn containing the trigger and the assistant turn replying with the payload, wrapped in a chat template sampled from a pool of public templates; there are 32 public templates in total, deliberately excluding the Qwen ChatML format used at deployment (Figure 3).
   • Declaration-based (newly proposed in this paper, inspired by Wang et al., 2025): each document states the trigger–payload rule in free text, as if recording expected behavior, and contains no chat template (Figure 4). The authors argue that this teaches the model a "belief" rather than a response pattern to imitate; clean post-training teaches tool use but does not correct this knowledge; and there is no template for defenders to filter on.

   End-to-end training pipeline

   • Pretraining: Megatron-LM trains Qwen3 0.6B/1.7B/4B from scratch on the FineWeb corpus for 100B tokens in a single epoch (meeting or exceeding the Chinchilla optimal ratio for each scale), injecting poisoned documents at the poisoning rate.
   • Post-training: SFT (half terminal tool data and half general data, 135,274 examples in total, plus 15,096 HH-RLHF safety samples) → DPO (9,369 HH-RLHF safety preference pairs filtered by Llama-Guard) → RLVR (GRPO, 200 verifiable bash tasks from InterCode-ALFA). SFT/DPO use LLaMA-Factory, and RLVR uses Terminal-Bench-RL.
   • Number of repetitions: each poisoned model is trained with three seeds (the low-poisoning-rate experiments and the unpoisoned baseline are single runs); lines in the figures are means and shading is min–max.

   Evaluation method

   • ASR: sample 32 rollouts per prompt (temperature 0.7); Pass@32 is the proportion of prompts for which at least one rollout contains the payload command; detection uses a flag-insensitive regex: a fixed curl header, the attacker URL, and a pipe into bash, with any flags in between counted. The evaluation set has about 1,000 prompts, generated by Sonnet 4.5 using a hierarchy disjoint from poisoning (20 domains, 60 topics per domain, 20 styles).
   • Trigger presentation: rare-string simulates prompt injection by embedding the trigger in a context block appended to a benign shell task; ordinary-string uses naturally occurring anthropic paths, taken from 982 held-out paths that do not appear in poisoning.
   • Capability evaluation: after pretraining, measure the mean acc norm of HellaSwag, ARC-Easy/Challenge, PIQA, and WinoGrande; at each post-training stage, measure first-token and exact-match accuracy on the NL2SH-ALFA test set (300 prompts without triggers).

   Collapse this answer

4. 04What experiments did the paper conduct?

   Under 0.1% poisoning, the 4B agent reaches Pass@32 ASR of 92.7% after GRPO (ordinary-string); ASR rises with scale, declaration outperforms demonstration across the board, and 0.0005% poisoning still yields 1.4%.

   Experimental setup

   • Models tested: Qwen3 0.6B, 1.7B, 4B pretrained from scratch (default architecture), FineWeb 100B tokens single epoch; three seeds per poisoned configuration, and one run per scale for the unpoisoned baseline.
   • Poisoning configurations: poisoning rate 0.1% (main experiments) and lower rates such as 0.01% and 0.0005%; full combinations of two designs (declaration/demonstration) × two triggers (rare-string/ordinary-string).
   • Evaluation sets: for rare-string, 1,000 benign shell tasks plus a context block containing the trigger; for ordinary-string, 995 natural requests, with paths taken from 982 held-out paths; both generated by Claude Sonnet 4.5 using a hierarchy disjoint from poisoning.
   • Metrics and detection: Pass@32 ASR (at least one of 32 rollouts contains the payload) and Pass@1; payloads detected with a flag-insensitive regex; Claude Haiku 4.5 rechecks executability on at most 200 payload-containing rollouts per cell.
   • Capability evaluation: mean acc norm on pretraining benchmarks (HellaSwag, ARC-Easy/Challenge, PIQA, WinoGrande); first-token and exact-match accuracy on 300 prompts from NL2SH-ALFA.
   • Baseline: unpoisoned models trained with the same pipeline on clean FineWeb.

   Main results

   According to Table 7 and Table 8 (0.1% poisoning, Pass@32 ASR %, mean of three seeds):

   Swipe horizontally to read the table

   Design	Trigger	Scale	Pretrain	SFT	DPO	GRPO
   declaration	ordinary-string	4B	30.2	92.0	91.3	92.7
   declaration	ordinary-string	1.7B	2.9	55.9	56.0	51.9
   declaration	ordinary-string	0.6B	8.0	14.2	15.4	13.1
   declaration	rare-string	4B	0.4	13.1	13.8	10.5
   declaration	rare-string	1.7B	1.1	6.8	6.9	4.0
   declaration	rare-string	0.6B	0.4	3.1	3.9	2.1
   demonstration	ordinary-string	4B	0.0	3.3	3.8	3.2
   demonstration	rare-string	4B	0.1	0.2	0.2	0.0

   • Authors' interpretation: ASR after pretraining is low because the base model has not yet learned the deployment chat template; the backdoor emerges after SFT teaches the format, DPO barely changes ASR, and GRPO sometimes reduces it only slightly.
   • Scale trend: the authors report that ASR rises monotonically with model scale under all poisoning configurations, and speculate that larger models are better at carrying low-frequency trigger–payload associations through post-training.
   • Design comparison: the authors state that declaration is no lower than demonstration in every cell (Figure 5(b)); demonstration never exceeds 1% at any scale or stage under the injected rare-string.

   Stealth and control experiments

   • Clean-prompt control: poisoned models have a payload misfire rate of 0.0% on clean NL2SH-ALFA prompts (all stages, scales, triggers, and designs); on the trigger-free version of the rare-string evaluation it is at most 0.06% (Appendix B.1).
   • Clean-model control: unpoisoned models have ASR of 0.0% on all evaluations (including trigger-containing prompts), on which basis the authors consider the backdoor specific to poisoning and the trigger.
   • No capability loss: after pretraining, the benchmark mean of poisoned models differs from unpoisoned models by less than 1 point (4B: 58.5% vs. 58.3%, Figure 6); after GRPO, declaration 4B achieves 67.2% first-token and 11.5% exact-match on NL2SH-ALFA, versus 64.6% and 11.9% for the unpoisoned model (Figure 7).

   Trigger generalization and delivery-method ablations

   • Path generalization: ASR evaluated with paths that appeared in the poisoning corpus (seen) and with held-out paths is nearly identical, e.g., 92.2% vs. 92.0% for 4B declaration after SFT (Figure 8); the authors state that the ordinary-string trigger acts on the general anthropic path pattern rather than a specific path.
   • rare-string delivery method: inserting the trigger as a standalone token at random and placing it in a context block give similar results, e.g., Pass@32 of 12.5% vs. 10.5% for 4B declaration after GRPO (Table 12 vs. Table 7).
   • Executability recheck: Claude Haiku 4.5 judges that 98.5% (ordinary-string) and 93.7% (rare-string) of payloads output by declaration models can directly download and execute the hosted script (Appendix A.5).

   Low-poisoning-rate experiments

   • 4B single-seed dose sweep (declaration, rare-string, fixed 100B-token corpus, Table 14): at 0.1%, Pass@32 after GRPO is 20.0%; at 0.01% (50,000 documents) it is 0.8%; at 0.0005% (2,500 documents) it is 1.4%; on the same task without the trigger it is 0%.
   • Authors' interpretation: lowering the poisoning rate greatly weakens the attack but does not eliminate it; the results at the two lowest rates are non-monotonic, which the authors attribute to single-seed noise.
   • Cross-scale supplement (each at its Chinchilla-optimal corpus, single seed, Table 16): 1.7B at 0.025% (50,000 documents) rare-string achieves Pass@32 of 20.8% after GRPO; several lower-rate configurations are 0.0%.

   Other ablations and analyses

   • Pass@1 (Table 9): 4B declaration ordinary-string after GRPO is 50.9%, and rare-string is 1.8%.
   • Misfires at the pretraining stage: only three demonstration cells show nonzero clean-prompt payload emission at the original pretraining stage, at most 0.02% per sample; declaration is 0.0% at all stages (Appendix B.1).
   • seen-path Pass@1 (Table 11): 4B declaration after GRPO is 50.0%, close to the held-out 50.9%.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors note two limitations: the scale does not reach production level, and payloads are limited to fixed strings; the experiments cover three small-scale Qwen3 models and a single CLI agent pipeline.

   Limitations and future directions noted by the authors

   • Production scale not verified (Section 6.2): ASR rises monotonically from 0.6B to 4B with no sign of saturation; the authors expect vulnerability at production scale to be at least as severe, but confirming this would require a pretraining budget beyond this setup; the authors consider whether the scaling trend of persistence follows known capability scaling laws to be the most important open question.
   • Midtraining unknown (Section 6.2): Modern production recipes increasingly insert a midtraining stage that upsamples high-quality data after pretraining, and whether poisoning persists differently in such stages is unknown.
   • Payloads limited to fixed strings (Section 6.2): It is not shown that dynamically synthesized payloads (e.g., commands adjusted based on system state) or triggers requiring compositional matching rather than string matching can be implanted equally persistently.
   • Malicious code generation direction (Section 6.2): The authors propose that a worthwhile case to study is a backdoored coding agent inserting covert vulnerabilities or hidden logic into each codebase, rather than outputting a fixed command; this would test whether persistence still holds when the malicious behavior cannot be reduced to reproducing a memorized string.

   Experimental coverage

   • Models tested: Qwen3 0.6B, 1.7B, 4B pretrained from scratch, three scales in total, with a pretraining budget of 100B tokens (the low-poisoning-rate supplementary experiments use their respective Chinchilla-optimal corpora of 15B/40B/100B).
   • Victim system: a single CLI-agent-like pipeline (SFT→DPO→GRPO, based on the OLMo-3 structure), with two evaluation triggers: rare-string (injected) and ordinary-string (anthropic path).
   • Repetitions: three seeds per configuration for the main experiments; the low-poisoning-rate experiments and the unpoisoned baseline use a single seed, and the paper notes that the non-monotonicity at low rates is attributed to single-seed noise.
   • Defense evaluation: the paper does not implement or evaluate any specific defense, and only discusses two potential defense directions in Section 6.1: data cleaning/provenance and deployment-time action monitoring.
   • Judge consistency: the executability review is performed by Claude Haiku 4.5 on at most 200 rollouts containing the payload per cell, and the paper does not report the agreement between this judge and human judgment.

   Collapse this answer

6. 06Summarize the main content of the paper

   Pretraining poisoning at 0.1% can implant a remote code execution backdoor in a CLI agent that survives clean post-training, with ASR up to 92.7% for the 4B model and no capability loss.

   An agent backdoor attack paper: it studies whether poisoning only the pretraining corpus can implant a backdoor into an LLM agent that survives the entire clean post-training pipeline and ultimately executes attacker commands on the user's machine.

   • Threat model: The attacker can only inject arbitrary documents into the pretraining corpus at up to 0.1% of tokens, and knows nothing about the architecture, data mixture, or deployment details; the goal is for the CLI agent to execute a remote code execution payload when the trigger appears (curl fetches and runs an attacker-hosted script that can be rewritten at any time).
   • Method: Uses Claude Sonnet 4.5 to hierarchically generate diverse poisoning documents injected into FineWeb, pretrains Qwen3 0.6B/1.7B/4B from scratch, then trains them into CLI agents through a clean SFT, DPO, GRPO pipeline. Proposes declaration-based poisoning (free-text statements of trigger–payload rules) and ordinary-string triggers (naturally occurring anthropic paths), compared with demonstration designs and injected rare-string.
   • Main results: Under 0.1% poisoning, the 4B model after GRPO achieves Pass@32 ASR of 92.7% (ordinary-string) and 10.5% (rare-string) (Table 7); across all configurations ASR rises monotonically with scale (13.1% and 2.1% for 0.6B).
   • Design comparison: declaration is no lower than demonstration in every cell; after GRPO, demonstration for 4B is only 3.2% (ordinary-string) and 0.0% (rare-string) (Table 8).
   • Stealthiness: Poisoned models have a 0.0% payload misfire rate on clean prompts, and clean models never emit the payload on trigger prompts; poisoned models' capabilities are comparable to unpoisoned models (4B pretraining benchmark 58.5% vs. 58.3%).
   • Conclusions and implications (in the authors' voice): At a 0.0005% poisoning rate (2,500 documents), a single-seed run of 4B still has 1.4% ASR; the authors believe the harder-to-detect declaration design is in fact stronger, which is bad news for defenders, and argue that the provenance of pretraining corpora should be included in agents' security models. The authors note they did not verify production-scale or payloads beyond fixed strings.

   Collapse this answer

Related papers