AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks against Agentic LLMs

Existing indirect prompt injection relies on static patterns and is mostly evaluated on simple models, making it difficult to simultaneously satisfy the robustness, adaptability, and stealthiness required by reasoning agents.

• Scenario: The authors state that LLM-based agents decompose multi-step tasks and fetch data through external services such as MCP. The authors cite more than 1,000 public MCP servers, over 50% of which are maintained by third parties and lack unified security auditing, expanding the attack surface. IPI places instructions into intermediate content such as websites or databases; after being read, it may leak data or execute harmful actions.
• Limitations of existing work: The authors state that reasoning models verify the user's original intent over multiple steps, and static repeated patterns are easily blocked by safety filters and Red Herring detection; if a tool is semantically inconsistent with the task, it is also flagged by internal auditing. The authors state that Perez et al. (2024), Liu et al. (2024c), Zhan et al. (2024), and Liu et al. (2025) cannot simultaneously handle Robustness, Adaptability, and Stealthiness.
• Core observation: After comparing on IPI-3k, the authors state that models with chain of thought are more resistant to attacks; common refusal reasons are Security Risk and Red Herring, along with tools being unrelated to the user's goal. The authors believe that in real scenarios it is difficult to rely only on template-based injection.
• What this paper proposes: The authors propose AdapTools, which includes adaptive attack strategy construction and attack enhancement (selecting more stealthy tools), and construct IPI-3k (3,691 benign trajectories, 277 high-privilege attack tools).
• Threat model:
  • Goal: Inject instructions into external content retrieved by the agent, causing the policy π to execute the specified tool fa and maximize the probability of that call.
  • Knowledge: MCP server controllers are grey-box and can see partial trajectory information (such as the most recent tool); third-party adversaries are black-box and only broadcast instructions publicly, without access to internal state or task details.
  • Capability: The attack occurs when the agent queries an external server and retrieves public content; the returned observation may contain instructions written by a third party. The implementation in §5.5 intercepts tool-call trajectories to infer task context.
  • Victim system: LLM agents that execute user tasks using tools and external interfaces (such as MCP). The experiments adapt ReAct.

Indirect Prompt Injection

• Sources and Targets: Greshake et al. (2023), Yi et al. (2025), Wu et al., Zhan et al. (2025) discuss IPI in agent interactions with external data. Instructions come from third parties rather than end users; the goal is to disrupt the system or exfiltrate data. The authors claim the injected content looks benign, making it hard to detect and mitigate.
• Representative Attacks: Willison (2022) uses escape characters to alter context parsing; Perez et al. (2024) and Schulhoff et al. (2023) ask the model to ignore previous context; Liu et al. (2024c) combine multiple strategies. AutoHijacker (Liu et al., 2025) uses LLM-as-Optimizer to generate prompts rather than static patterns.
• Authors' Comparison: The introduction and Table 1 use Adaptability, Stealthiness, Robustness to compare against the above works, claiming they fail to satisfy all three simultaneously. In the table, the columns for Perez, Liu (2024c), and Zhan (2024) are all # (missing); Liu (2025)'s Stealthiness is #. The remaining symbols in that row and the symbols in the AdapTool row are missing from the text extraction and are not transcribed here.

Tool Calling Benchmarks

• InjectAgent and AgentDojo: Zhan et al. (2024) and Debenedetti et al. (2024) evaluate robustness to IPI in tool-calling scenarios. The authors claim the samples are highly artificial, mainly evaluated on non-reasoning LLMs, in single-turn and static strategies.
• Scale (Table 2): AgentDojo has 629 test cases and 30 attack tools; InjectAgent has 510 cases and 27 tools; IPI-3K has 3691 cases and 277 tools.

Defenses

• Input-level: instruction prevention (2023), data prompt isolation (Willison, 2022), sandwich prevention (2023) use static rules to guide the model to ignore commands in external data. Pi-Detector (Debenedetti et al., 2024) uses a classifier to distinguish malicious instructions in tool responses.
• Output-level: ProtectAI (2023) uses a fine-tuned model to judge whether outputs contain malicious content. MELON (Zhu et al., 2025) and Perplexity Filtering (Jain et al., 2023) are training-free methods relying on rules or heuristics to determine if tools are tampered with. This section only provides categorical descriptions, not per-paper critiques.

Baselines and Datasets in Experiments

• Baseline Methods: Ignore Instruction (Schulhoff et al., 2023), Combined Attack (Liu et al., 2024c), InjectAgent (Zhan et al., 2024), Autohijacker (Liu et al., 2025; the authors note the code is not open-sourced, so they reimplement it). Defense baselines are MELON and Pi-Detector. Benchmarks are IPI-3k, InjectAgent, AgentDojo.

The authors position this work as: moving away from static prompt injection, using transferable strategies and task-relevant tools, combining robustness, adaptability, and stealthiness, and evaluating on rapidly evolving reasoning agents.

AdapTools first automatically accumulates and compresses transferable attack strategies, then selects high-privilege tools that are semantically closer along the trajectory, and appends the generated prompt to the external observation to increase the probability of calling the target tool fa in the next step.

Problem Setup and Data

• Agent: Policy π iterates under user instruction Iu. At each step, it generates action at and calls ft from the tool set F; external tools return observation ot via MCP and similar mechanisms. A trajectory consists of the instruction and a sequence of (action, tool, observation).
• Attack Objective: The paper writes the objective as max_pa P_π(a_t+1=f_a∣τ_t;õ_t), s.t. õ_t=o_t⊕ p_a. ⊕ denotes text concatenation, i.e., appending the attack prompt after the original observation. The policy-level formulation is: maximize the probability of calling fa in the next step over the policy set, with the prompt synthesized by generator G based on fa and strategy sa.
• Targeted Failures: Red Herring, Security Risk, Unrelated Information. The authors state that other failures mostly stem from the model's own limitations, and these three categories are the most exploitable vectors for IPI. The strategy module targets the first two categories, and tool selection targets irrelevant content.
• IPI-3k: 3,691 multi-step benign trajectories requiring external retrieval, reorganized from existing benchmarks (Zhang et al., 2025a). Additionally, there are 277 high-privilege tools that can access sensitive information, categorized into data leakage, financial loss, and system harm. Entrypoint is a tool that interfaces with an intermediate server; Direct Harm is the terminal goal of the attack. Commercial LLMs (e.g., GPT-4o, DeepSeek-R1) are scored 0–10 for exploitability. The paper does not specify the threshold used.

Adaptive Strategy Construction

• Process (Figure 3): The strategy library S is initially empty. The attacker randomly selects a high-privilege tool fa; the authors state that random selection forces strategies to adapt to different tools and decouples them from specific tasks. The most relevant strategy sa is retrieved from S and given to the attack prompt generator (an adversarial LLM; GPT-5 is given as an example in the formalization) along with the user task. In cold start, generation is based only on the tool description. After injecting the prompt into benign retrieval content, reasoning continues.
• Success/Failure Update: If fa is called, the strategy is stored in the library. If it fails, the chain of thought is extracted and given to an analyzer (written as commercial LLM in the paper) to diagnose the rejection reason and revise the strategy. This continues for at most K rounds or until success. The default iteration count in experiments is ka=5; the main text does not give other values for K.

Strategy Distillation

• Method (Figure 4): The authors state that as tool–task combinations increase, retrieval becomes slower and more prone to mismatches, and fine-grained strategies are hard to transfer to new conditions. A text embedding model (citing Zhang et al., 2025b; Wang et al., 2024) converts strategies into vectors, then clustering methods like K-means abstract them. ASR is used as utility: merging is allowed only if the ASR drop after merging does not exceed a threshold δ. The paper does not provide δ, nor does it specify the actual embedding model name in the experimental section.
• Design Intent: The authors state that the pruned library retains main decision patterns and removes overly specific rules, and claim that ASR can be comparable to the fine-grained library. The specific numbers for this comparison are not given in this section.

Attack Enhancement: Tool Selection

• Illustration: Figure 5 marks task-irrelevant calls as REJECT and task-relevant calls as SUCCESS in a coffee-ordering task. Figure 6 text contains 86%, fa, and ft+1; the main text does not separately explain 86%. The injected statements in the figure are not restated.
• Transition: The tool sequence is treated as a first-order Markov chain. Grey-box sees the most recent tool ft. The transition matrix is computed from all benign trajectories: the count from fi to fj divided by the total count starting from fi.
• Selection Criterion: First predict the most likely next benign tool without attack, then select the attack tool with the highest embedding cosine similarity. The paper writes f_a=argmax_{f∈ F}sim(ϕ(f),ϕ(f_t+1^∗)). φ is a pretrained semantic embedding, and sim is cosine similarity. The authors state this satisfies both temporal dependency and semantic alignment.

Implementation and Success Determination

• Three Phases (§5.5): The strategy library and transition matrix are built offline and reused. When a user task requires external data, the agent calls MCP, and the attacker intercepts the call trajectory; use the transition matrix to select fa, retrieve a generalized strategy from the library, and synthesize a prompt; inject the prompt into the legitimate data returned by the controlled server.
• Success: The agent subsequently executing fa counts as success. Experimental metrics are ASR (higher is better) and attacked utility UA (lower is better), with unattacked BU also reported. The paper does not state that ASR is determined by an LLM judge.

Experimental Setup

• Models under test: The agent framework is ReAct. The base models are GPT-4.1, DeepSeek-R1, Gemini-2.5-Flash, Qwen-3-8B, LLaMA-3.1-8B, Mistral-8B. Note in Table 3: the CoT column marks # for non-reasoning; GPT-4.1, LLaMA-3.1-8B, Mistral-8B are marked with #. The last three models are not marked with #, and the text discusses them together with reasoning models. The introduction writes Gemini as Gemini-2.5.
• Data: The main dataset is IPI-3k (3,691 benign trajectories, 277 attack tools). InjectAgent and AgentDojo are also used. Table 3's title does not mention the dataset name; InjectAgent and AgentDojo are reported separately in §6.2. The paper does not state the size of the trajectory subset used for Table 3.
• Baselines: Ignore Instruction, Combined Attack, InjectAgent, Autohijacker (reproduced by the authors, who state the code is not open-sourced). Defenses are No Defense, MELON, Pi-Detector.
• Metrics: ASR is attack success rate, higher is better; UA is utility under attack, lower is better; BU is utility without attack. Success corresponds to executing the target tool. The paper does not report LLM evaluation, human agreement, number of repetitions, or number of queries. The attack prompt generator is exemplified with GPT-5 in the formalization, the analyzer is written as commercial LLM; the experimental section does not name the actual model.

Main Results

According to Table 3 no-defense rows. AH is AutoHijacker.

Swipe horizontally to read the table

Model	BU	Ours ASR	AH ASR	Ours UA	AH UA
GPT-4.1	66.0	26.1%	12.0%	44.8%	55.4%
DeepSeek-R1	50.0	20.3%	9.3%	36.8%	40.2%
Gemini-2.5-Flash	61.0	35.4%	22.4%	17.3%	26.0%
Qwen-3-8B	80.0	60.6%	29.6%	32.6%	49.6%
LLaMA-3.1-8B	38.0	44.4%	37.6%	15.2%	17.8%
Mistral-8B	36.0	69.3%	47.4%	6.7%	12.4%

• Authors' summary: §6.1 states that on commercial LLMs the average ASR is 2.13× that of the best baseline. The introduction states it is approximately doubled relative to the best baseline (14.5%). Neither place gives the calculation. The abstract states the system utility degradation factor is 1.78, and §6 does not repeat this factor. In Table 3, the highest baseline ASR per model under no defense is 12.0%, 9.3%, 22.4%, 29.6%, 37.6%, 47.4%, and the highest in that row is always AutoHijacker.
• Open-source vs. commercial: The authors state that open-source models have weaker defenses, with average ASR exceeding 30%; Mistral-8B's Avg ASR is 49.0%, LLaMA-3.1-8B is 31.4%, Qwen-3-8B is 44.5%. The introduction gives local models' average ASR as 58.1% (baseline 38.2%), without specifying whether it is under no defense or across defenses. The authors state that commercial models are more resistant to such attacks than open-source models. On GPT-4.1, the authors state the baseline average ASR is below 8%, and this paper raises ASR to 18.5% (Avg row in Table 3).
• Authors' interpretation of individual models: The authors believe Mistral-8B's high ASR may be related to weaker function-call grounding (BU 36.0%). The authors state that LLaMA-3.1-8B's Avg ASR is relatively low, but under IPI the utility drop is larger. Qwen-3-8B has higher BU, yet Avg ASR is still 44.5%.

Results under Defenses

• What was tested: Table 3 reports MELON and Pi-Detector on each model. The authors state that both can mitigate IPI, but under AdapTools ASR is only reduced by nearly 50% (introduction); §6.1 states that defenses reduce baseline ASR to at least about 1/3, and for this paper to about 1/2.
• Grid: GPT-4.1 no defense 26.1%, MELON 13.3%, Pi-Detector 16.1%. The authors state DeepSeek-R1 drops from 20.3% to an average of 10.1% under the two defenses, and Qwen3-8B from 60.6% to an average of 36.4%. In Table 3, the lowest Ours ASR is DeepSeek-R1 with MELON at 6.7%. Across the six models, the Ours ASR in the MELON row is always lower than in the Pi-Detector row.
• Utility: According to Table 3, for the six models, the Ours UA under no defense is higher after adding MELON or Pi-Detector, closer to their respective BU. The authors do not state this as a separate conclusion.

Transfer

• InjectAgent (Figure 7): Only GPT-4.1 and Qwen3-8B are reported. The authors state that relative to Autohijacker, GPT-4.1 is 3.6% higher, Qwen3-8B is 6.5% higher. The figure annotates +3.6% and +6.5%, and shows 0.204, 0.157, 0.139, 0.122, 0.000, 0.004; after extraction, each bar cannot be reliably matched to a method, so no pairing is made.
• Authors' interpretation: The authors state that the overall ASR on InjectAgent is lower than on what they call a more realistic dataset, and attribute this to modern LLMs absorbing known attack cases into their safety mechanisms.
• AgentDojo: The authors state that AdapTools outperforms all existing baselines. The paper does not give ASR numbers on this benchmark.

Tool Selection Ablation

• Setup: Table 4 is called grey-box. The authors state that due to API costs, only GPT-4.1 and Qwen3-8B are tested. The ASR and UA with w/ selection are the same as Ours under no defense in Table 3; the paper does not specify grey-box or black-box in Table 3's title.
• Numbers: GPT-4.1 without selection ASR 21.4%, UA 43.8%, with selection 26.1%, 44.8% (Δ +4.7, +1.0). Qwen3-8B is 52.7%, 36.4% vs. 60.6%, 32.6% (Δ +7.9, −3.8).
• Interpretation and exception: The authors state the improvement comes from avoiding failures unrelated to the task and selecting task-relevant tools. GPT-4.1's UA increases by 1.0 percentage point, opposite to the attack direction where lower UA is better; the authors do not separately explain this cell.

Other Ablations and Analyses

• Reasoning comparison (§4, Combined Attack): Qwen3-8B thinking ASR 19.4%, non-thinking 26.1%, LLaMA-3.1-8B 32.8%. The authors attribute this to explicit chain of thought judging malicious prompts as unrelated to user instructions or risky. 19.4% and 32.8% equal the corresponding no-defense Combined Attack cells in Table 3.
• Failure types (Figure 2): The object is IPI attacks based on IPI-3k on Qwen3-8B; the paper does not specify which attack method. The categories are Security Risk, Red Herring, Unrelated, No Functions, Arguments Error, Others; the percentages in the figure are 29.3%, 24.3%, 20.2%, 9.9%, 8.7%, 7.6%. After extraction, reliable pairing is not possible. The text only names the first two as common refusal reasons and discusses Unrelated.
• Iterations (Figure 8): Default ka=5, the authors state they compare seven levels. The authors state that a single iteration gives ASR about 35%, then exceeds 80% and converges; the figure annotates key point (5, 0.81), with x-axis from 0 to 7. The authors state that 6 or 7 iterations are slightly higher, but API costs increase, so 5 is chosen as a compromise. The paper does not specify the model and data subset for this figure. This 0.81 is higher than any ASR in Table 3 and should not be read together with the main evaluation ASR.

Limitations and Future Directions Stated by the Authors

The paper does not have a Limitations or Future Work section. The conclusion only states that this work advances understanding of IPI and provides a reference for subsequent research, without listing specific experiments to be done. The impact statement discusses the ethical trade-offs of disclosing attack methods, stating that the benefit to defense research outweighs the risk of misuse; this is an explanation of the disclosure practice, not a limitation of the method's capability. Therefore: the paper does not specifically discuss limitations.

Experimental Coverage

• Models and framework: The base models tested are GPT-4.1, DeepSeek-R1, Gemini-2.5-Flash, Qwen-3-8B, LLaMA-3.1-8B, Mistral-8B, a total of 6; the agent framework is ReAct (§6, Table 3).
• Data and defenses: Datasets are IPI-3k (3,691 benign trajectories, 277 attack tools), InjectAgent, AgentDojo. Table 3 includes No Defense, MELON, Pi-Detector; the caption does not mention a dataset name. AgentDojo only gives the statement "outperforms baselines," without ASR numbers (§6.2).
• Baselines and ablation scope: Attack baselines are Ignore Instruction, Combined Attack, InjectAgent, Autohijacker (reproduced by the authors). The tool selection ablation reports only GPT-4.1 and Qwen3-8B; the authors attribute this to API costs (Table 4).
• Iterations: Strategy library generation defaults to ka=5 and compares seven levels (Figure 8, annotated (5, 0.81)). The paper does not specify the model for that figure.
• Unreported items: The paper does not report the specific model names of the attack prompt generator and failure analyzer in the experiments (only giving examples or writing "commercial LLM" in the methodology), the merge threshold δ, the number of repetitions and queries for Table 3, or whether ASR was verified by humans or LLMs.

AdapTools is an adaptive indirect prompt injection method for tool-calling LLM agents, accompanied by the evaluation set IPI-3k.

• Problem: Reasoning agents check user intent step by step. The authors state that static IPI is difficult to be robust, adaptive, and stealthy simultaneously, and is mostly evaluated on non-reasoning models. The attacker is either a grey-box MCP controller that can see recent tools, or a black-box third party that can only publicly deliver instructions; the goal is to make the agent execute a specified high-privilege tool.
• Approach: Use chain-of-thought from failed trajectories to iteratively refine a strategy library, then cluster by embeddings, merging strategies only when ASR drops by no more than δ. The grey-box path uses benign trajectories to estimate first-order tool transitions, selects attack tools more consistent with the current task by cosine similarity, and injects prompts into external return content. IPI-3k has 3,691 benign trajectories and 277 attack tools. The experimental agent is ReAct.
• Main results: In Table 3 under no defense, AdapTools' ASR is higher than the four baselines listed in the same row across all six models: Mistral-8B 69.3%, Qwen-3-8B 60.6%, LLaMA-3.1-8B 44.4%, Gemini-2.5-Flash 35.4%, GPT-4.1 26.1%, DeepSeek-R1 20.3%. The authors state that on commercial models the average is 2.13× that of the best baseline; the introduction also states it is about twice the best baseline of 14.5%, and gives local models' average ASR as 58.1% (baseline 38.2%). The abstract states the utility degradation factor is 1.78.
• Defenses and transfer: Under MELON, GPT-4.1's ASR is 13.3%, under Pi-Detector 16.1%; the lowest Ours cell in Table 3 is DeepSeek-R1 with MELON at 6.7%. The authors state that defenses reduce the baseline to about 1/3, and AdapTools to about 1/2. On InjectAgent, the authors state that relative to Autohijacker, GPT-4.1 is 3.6% higher and Qwen3-8B is 6.5% higher. No ASR numbers are given for AgentDojo.
• Authors' conclusion: The authors state that the method remains effective under existing defenses, thus requiring stronger agent protection, and provides a reference for future research. The paper does not specifically discuss limitations.