Your Agent Is More Fragile Than You Think: Revealing Indirect Prompt Injection Vulnerabilities in Agentic LLMs

In dynamic multi-step tool calls, can indirect prompt injection (IPI) hidden in third-party content cause the agent to perform unauthorized actions, and can surface defenses block them?

• Scenario: The authors state that open-source frameworks expand the action space, including uncontrolled permission exposure and hidden inter-system interactions. When the agent simultaneously holds excessive API permissions and sensitive information, IPI can trigger data exfiltration during normal operations; the authors use EchoLeak as an example, noting that email payloads can bypass guardrails and exfiltrate files.
• Evaluation gap: The authors state that existing evaluations mainly rely on isolated single-turn benchmarks. Unlike direct prompt injection, where users directly write malicious instructions, IPI comes from web pages or transaction records that the agent retrieves itself, and payloads can remain dormant until a specific tool sequence triggers them.
• Two points the authors want to answer: Whether contemporary defenses are robust in dynamic workflows; which paradigm can better detect unauthorized behavior. The authors state that potential hesitation during hijacking prompted them to examine the representation layer.
• Work in this paper: In the AgentDojo Banking tool-calling workflow, four types of IPI are used to evaluate nine open-source backbones and six defenses, and behavioral, linguistic, and confidence measures supplement the binary success rate; Representation Engineering (RepE) is then used to identify hijacking states before action submission.
• Threat model:
  • Goal: To make the agent call an attacker-specified tool. Appendix A gives malicious transfers and data exfiltration as examples of such tools.
  • Capability: Does not directly rewrite user input, but embeds instructions in third-party content returned by tools. The paper does not state whether the attacker can read weights, nor does it use the words white-box or black-box.
  • Victim system: The Banking environment in Fig. 1, including account states, transaction history, callable tools, and multi-step trajectories. The abstract places the problem on multi-agent systems; the experiments describe this tool-calling workflow.
  • Determination: Hijack Rate is the proportion of episodes in which the specified tool is called as a result. Utility only looks at whether the original legitimate task is completed, separately from whether hijacking occurred.

The Introduction and Related Work place related work on agent capabilities, agent safety, and IPI defenses; experimental baselines are listed separately.

• Agents and tool loops: ReAct (Yao et al., 2022), ToolLLM (Qin et al., 2023), and WebArena (Zhou et al., 2023) place LLMs in iterative loops, parsing the environment and calling external interfaces. Voyager (Wang et al., 2023), MetaGPT (Hong et al., 2023), AutoGen (Wu et al., 2024), and OpenAgents (Xie et al., 2023) are used to illustrate that external skills can orchestrate open-ended workflows. Pre-Act (Rawat et al., 2025) is cited as an example of multi-step planning enhancing reasoning. The paper does not critique these systems individually.
• Agent safety: Chiang et al. (2025) analyze the vulnerability of web agents relative to standalone LLMs; SafePro (Zhou et al., 2026) evaluates professional-level agent safety. SkillJect (Jia et al., 2026) uses iterative adversarial refinement to generate poisoned agentic skills. The authors state that single-turn text jailbreaks have been extensively studied, while indirect threats hidden in external environments remain largely underexplored.
• IPI and existing mitigations: Greshake et al. (2023) embed malicious instructions in third-party content retrieved by tools; Zou et al. (2023b) type direct jailbreaks explicitly override system prompts. Spotlighting (Hines et al., 2024) applies structural transformations to external text, GuardAgent (Xiang et al., 2025) uses a secondary agent to check against safety rules, and BIPIA (Yi et al., 2025) trains models to ignore anomalous embedded instructions. The authors state that the effectiveness of these methods in continuous dynamic interactions remains underexplored, and call whether LLM-as-a-judge remains effective against more stealthy injections an open question.
• Baselines and benchmarks: Defense baselines are No Defense, as well as Prompt Warning, Sandwich, Paraphrasing (Jain et al., 2023), Spotlighting, Keyword Filtering (Deng et al., 2023), and LLM-as-a-Judge (Maloyan et al., 2025). Attack vectors are Direct, Ignore Previous (Perez and Ribeiro, 2022; Debenedetti et al., 2024), InjecAgent (Zhan et al., 2024), and Stealth (Shen et al., 2026; Wang et al., 2026). The environment is AgentDojo Banking (Debenedetti et al., 2024). Detection uses RepE (Zou et al., 2023a) and probing (Kirch et al., 2025).

The authors position the paper as a systematic comparison in dynamic interactions and state that experiments show current strategies are vulnerable, while representation engineering can detect potentially malicious states.

On AgentDojo Banking, the environment, tools, and task abstraction are kept, with attacks and defenses modularly replaced, and hijack and alignment trajectories are recorded; detection uses RepE to compare hidden states of these two trajectory types.

Environment and Tasks

• The testbed is the Banking suite of AgentDojo (Debenedetti et al., 2024). The authors state they did not reproduce the default implementation exactly but extended attacks and defenses.
• There are 576 attack scenarios: 16 user tasks, each with 9 injection targets; each of the four IPIs has 144 cases. Each scenario also records an alignment trajectory.
• Each backbone is initialized with the standardized prompts from Debenedetti et al. (2024). The paper does not report temperature, sampling method, or number of repetitions.
• Fig. 1 depicts account and transaction states, legitimate tasks and tools, tool results with injections, and subsequent hijack or alignment actions.

Attack Vectors and Six Defenses

• Direct: The text states malicious content is embedded in tool call returns. Ignore Previous: The authors classify it as basic IPI, corresponding to ignoring previous instructions and directly changing the task. InjecAgent: Injections are embedded in external content retrieved by tools, such as transaction notes. Stealth: Obfuscation is used to hide payloads past regular filters. Specific injection sentences are not reproduced.
• The six defenses are Prompt Warning, Sandwich, Paraphrasing, Spotlighting, Keyword Filtering, and LLM-as-a-Judge. Sec. 3.2 writes "seven defense strategies"; the seven rows in Table 2 are these six plus No Defense.
• The paper does not provide thresholds for each defense, the paraphrasing model, or the model name for LLM-as-a-Judge.

Eight Metrics

• Attack vulnerability covers all scenarios. Hijack Rate is the proportion of calls to attacker-specified tools; Utility is the proportion of completing the original legitimate task, regardless of hijacking.
• Behavior, language, and confidence are computed only on hijacked cases. Immediate Compliance means calling the malicious tool in the next step; Action Divergence means the first step after injection deviates from the first step of the benign trajectory.
• Language metrics apply a fixed syntactic filter to Thought. Resistance counts rejection or suspicion markers. Table 1 column name is S-Immed.; Appendix A additionally defines S-Comp, the sum of Immediate and Rationalized thoughts. The names and scopes are not the same.
• Confidence is the average log probability and average Shannon entropy of malicious action tokens. Fig. 2(B) further splits language failures into Immediate, Heading, Rationalized, and Reluctant.

RepE Detection

• No fine-tuning; it is post-hoc analysis. Trajectories come from the no-defense configuration, and only samples containing tool call markers are kept. Training/testing is split 80/20. The paper does not state the split unit.
• Hidden states are taken from all layers. The optimal layer is determined by layer-wise ablation: cross-validated training of logistic regression to predict hijacking. The paper does not state the number of folds or whether layer selection is restricted to the training split.
• Two positions: tool-input, where the injection enters; and function-calling, where the model commits to using a tool.
• Probing uses logistic regression to output hijack probability. Cosine similarity first constructs a danger direction from alignment to hijack, then thresholds the scores. The paper does not provide that threshold; it reports AUC-ROC, AUPRC, and TPR at 5% FPR. The authors state that at inference, interception can occur before committing to an action; the numbers in the text are detection metrics on this split.

Experimental setup

• Backbones: Qwen-2.5-14B/32B, Qwen-3-4B/8B/14B, Llama-3-8B, GLM-4-9B, Gemma-3-12B, Mistral-7B. In Table 1 they are abbreviated as Qwen-14B, Qwen-32B, Qwen3-*, LLaMA3-8B, GLM4-9B, Gemma3-12B, Mistral.
• Scenario: AgentDojo Banking. The 576 attack scenarios are 16 user tasks times 9 injection targets, with 144 cases for each of the four attacks; each scenario also has an aligned trajectory.
• Attacks and defenses: Direct, Ignore Previous, InjecAgent, Stealth; six defenses plus No Defense. Table 2 only reports hijack rates for three backbones.
• Metrics: eight, divided into four dimensions (Sec. 3.1, definitions in Appendix A). Hijack and Utility use all scenarios; the other three dimensions use only hijacked cases.
• Determination: hijack is determined by whether the attacker-specified tool is called, not by LLM scoring. Language metrics use a fixed syntactic filter. The paper does not report the agreement between this filter and human judgment.
• Unspecified settings: the model name, temperature, and number of repetitions for LLM-as-a-Judge; the number of cross-validation folds and the classification threshold for RepE. RepE metrics are AUC-ROC, AUPRC, and TPR@FPR5%.

Main results

Table 1 shows bottleneck performance under the four types of IPI, without stating whether defenses were enabled. The table below lists only 6 models; except for Entropy, the arrows are the directions given in the paper.

| Model | Hijack↓ | Utility↑ | Immed.↓ | Resist↑ | Entropy↓ | | Qwen3-14B | 81.94% | 37.50% | 98.63% | 4.45% | 0.194 | | Qwen-32B | 97.22% | 56.25% | 99.68% | 8.75% | 0.136 | | Qwen-14B | 97.40% | 62.50% | 98.35% | 6.24% | 0.345 | | LLaMA3-8B | 100.00% | 31.25% | 92.37% | 4.86% | 0.373 | | GLM4-9B | 100.00% | 25.00% | 88.75% | 6.25% | 0.195 | | Gemma3-12B | 100.00% | 31.25% | 100.00% | 6.60% | 0.034 |

• Hijack rates not listed: Qwen3-8B 91.15%, Qwen3-4B 96.01%, Mistral 99.65%. In Table 1, the Hijack values for Qwen-14B, Qwen3-14B, and Mistral are the same as the No Defense Overall in Table 2.
• The authors state that the backbones struggle to recognize the malicious components in injections, produce very little resistant language, and quickly execute malicious actions after receiving an injection. Fig. 3(A) plots the assistant turns from injection to malicious action for three backbones; the main text does not give the number of turns.
• The authors state that hijacked trajectories have higher uncertainty than aligned trajectories (Fig. 3(B)(C), shading is ±1 standard deviation). The paper does not report significance tests. Utility, as defined in the appendix, is not counted as a failure when hijacked.

Hijack rates under the six defenses

• What was tested: Table 2 reports the Hijack Rate for Qwen-2.5-14B, Qwen-3-14B, and Mistral-7B under the four attacks.
• The two Qwen models: Qwen-2.5-14B has a No Defense Overall of 97.40%, and the six defenses give an Overall of 98.44%–100.00%, with LLM Judge at 100.00%. Qwen-3-14B has 81.94% without defense and 82.99%–88.72% under the six defenses. The authors state that Sandwich adds another 1%; in the table, Sandwich Overall is 82.99%. In the same table, Prompt Warning's InjecAgent is 98.61%, while the corresponding no-defense cell is 91.67%.
• Exceptions: Mistral-7B's LLM Judge Overall is 99.13%, lower than the no-defense 99.65%; its Stealth is 97.92%, lower than the no-defense 99.31%. On Qwen-3-14B, Sandwich's Direct is 84.72% and Stealth is 88.19%, lower than the corresponding no-defense cells, but the Overall is still higher. The authors state that some defenses introduce adversarial interference and increase the likelihood of being hijacked; this is consistent with the Overall for the two Qwen models, but inconsistent with Mistral-7B's LLM Judge Overall.

RepE hijack detection

• Setup: no-defense trajectories, tool-input position, 80/20 split. Table 3 gives both probing and cosine similarity. The authors state that probing is better.
• Numbers: probing's TPR@FPR5% ranges from 85.71% for Gemma3-12B to 100% for Qwen-14B, Qwen-32B, and Mistral. Qwen3-8B probing gives AUC-ROC 0.9963, AUPRC 0.9961, TPR@FPR5% 97.39% (Fig. 4 shows the ROC/PR for this model; the main text gives no other curve readings). Counterexamples: Gemma3-12B's cosine three values are 1.0000, 1.0000, 100%, higher than probing's 0.8776, 0.9341, 85.71%; Qwen3-4B's cosine is 0.9968, 0.9970, 96.00%, higher than probing's 0.9912, 0.9915, 93.39%. For LLaMA3-8B, the same layer goes from 98.68% with probing to 36.96% with cosine.
• Position: Table 4 contains only three models. Qwen3-14B probing's TPR@FPR5% is 53.00% at function call and 97.12% at tool-input; the authors state nearly 44% improvement and write rising from 53% to 97.12%. LLaMA3-8B's cosine at function call is 69.57%, higher than the 36.96% at tool-input. The authors state that tool-input is better on almost all backbones, except LLaMA.

Other ablations and analyses

• Fig. 2: Qwen-2.5-14B's Direct is 100% (144/144). The authors state that more covert vectors are more severe; in Table 2's no-defense row, this model's Stealth is also 100.00%, InjecAgent is 91.67%, and Ignore Prev. is 97.92%.
• The main text writes verbal resistance 6.2%, no hesitation about 66.1%, Rationalized 10.9%. The first two numbers correspond to Qwen-14B's Resist 6.24% and S-Immed. 66.13% in Table 1. The figure has both n=576 and n=561; the main text does not explain n=561.
• In Table 1, S-Immed. ranges from LLaMA3-8B's 30.21% to Gemma3-12B's 88.37%; Diverge ranges from GLM4-9B's 8.33% to LLaMA3-8B's 47.92%; LogP ranges from LLaMA3-8B's -0.208 to Qwen3-14B's -0.047.
• Fig. 3 compares Qwen-14B, Qwen3-14B, and GLM4-9B. The authors state that hijacked cases have greater variance and longer conversations; per-turn readings are not given in the main text.
• The selected layers in Table 3 range from Mistral's layer 2 to Qwen-32B's layer 53. The paper does not report the number of filtered trajectories, the number of cross-validation folds, or the unit of the 80/20 split.

Limitations and future directions noted by the authors

The paper has no dedicated Limitations, Discussion, or Future Work section. The Conclusion does not frame any result as a limitation, nor does it list experiments to be done. Sec. 3.3 states that combining RepE with existing safety modules is a future research direction, but that sentence is not in the aforementioned dedicated sections. Therefore: the paper does not specifically discuss limitations.

Experimental coverage

• The tested backbones are the nine open-source models listed in the main text. The defense breakdown hijack rates (Table 2) only report Qwen-2.5-14B, Qwen-3-14B, and Mistral-7B.
• The environment is the tool-calling workflow of AgentDojo Banking (Fig. 1). There are 576 attack scenarios, i.e., 16 user tasks times 9 injection targets, with 144 cases for each of the four attacks.
• The defenses are the six listed in the main text plus No Defense; the attacks are Direct, Ignore Previous, InjecAgent, and Stealth. The paper does not report the specific model used for LLM-as-a-Judge, nor the implementation parameters of each defense.
• RepE uses no-defense paired trajectories and an 80/20 split, with metrics AUC-ROC, AUPRC, and TPR@FPR5%; the position ablation (Table 4) covers three models. The paper does not report the hijack rate after interception deployment, nor whether layer selection was isolated from the test split.
• Language metrics come from a fixed syntactic filter (Appendix A). The paper does not report the agreement between this filter and human judgment, nor the number of experimental repetitions.

In the AgentDojo banking tool-calling environment, the authors examine whether indirect prompt injection causes open-source agents to execute attacker-specified actions, and the respective numbers for surface defenses and representation-based detection.

• Problem: Single-turn evaluations do not see IPI hidden in tool returns. The attacker controls third-party content, not jailbreak sentences directly entered by the user. The victim workflow is multi-step tool calling in the Banking suite.
• Method: 16 user tasks, 9 injection targets per task, four attack types totaling 576 scenarios, nine open-source backbones, six defenses. Additionally, hidden states are extracted from undefended aligned and hijacked trajectories, and probing and cosine similarity are compared at both tool-input and function-call locations.
• No defense: In Table 1's overall hijacking rate, Qwen3-14B is 81.94%, and LLaMA3-8B, GLM4-9B, and Gemma3-12B are 100.00%. Qwen-2.5-14B's Direct is 144/144. On the Utility side, one end is Qwen-14B at 62.50%, and the other end is GLM4-9B at 25.00%.
• Defenses: In Table 2, all six defenses on Qwen-2.5-14B and Qwen-3-14B have Overall higher than their respective undefended baselines. Mistral-7B's LLM Judge Overall is 99.13%, lower than the undefended 99.65%. The authors state that some surface mitigations have adverse side effects.
• Detection: At tool-input, Qwen3-8B probing has TPR@FPR5% of 97.39% and AUC-ROC of 0.9963. The authors state that this location is better than function-call, and that probing is better; in Table 3, Gemma3-12B and Qwen3-4B have higher cosine similarity.
• Authors' conclusion: Surface defenses cannot block these four types of IPI in this multi-step tool environment; RepE can identify unauthorized actions before the model commits to a tool call, providing a reference for future safety architectures.