Skip to content

The Devil Is in the Context: Nine Dimensions of Prompt Injection Attacks

The Devil is in the Context: Nine Dimensions of Prompt Injection Attacks

arXiv: Jailbreaks, Prompt Injection, Poisoning & Defenses· arXiv:2611.04672· Jiamin Chang, Kristina Nikolić, Rishi Sharma, Giovanni Cherubin, Vasilios Mavroudis, Jose Sanchez Vicarte, Arnav Garg, Hammond Pearce, Andrew Paverd, Ahmed SalemUniversity of New South Wales; ETH Zurich; Microsoft Security Response CenterIndexed by this site · 16 days ago· Originally publishedAttacks

Quick read

Evaluation and benchmarks

Based on the paper PDF (AI-generated); refer to the original

The authors propose a nine-dimensional prompt injection taxonomy, analyze 6 real attacks and 9 benchmarks, state that 13.2% of dimension combinations are uncovered, and use it to generate new test cases that expose differences among defenses.

Questions
Prompt injection defenses and benchmarks treat attacks as a homogeneous threat, making robustness claims incomparable, causing benchmark scale to be mistaken for coverage, and preventing developers from matching defenses to threat models. The authors argue that whether an injection holds depends on user intent and system context.
Methods
Propose a nine-dimension taxonomy (source label, fragmentation, visibility, interpretability, conditionality, state modification, topic relation, intended effect, original-task completion), with injected instruction + user task + system context as the unit of classification, and build an automatic annotation and targeted case-generation pipeline combining rules and LLM review.
Experiments and results
The 6 real attacks cover all nine dimensions; 13.2% of dimension-pair combinations across the 9 benchmarks are not covered. For variants generated from 40 AgentDojo Workspace cases: GPT-5.6-Luna's ASR without defenses rises from 0% to 17.5%, and differences in ASR and utility across defenses emerge.
Limitations and future work
The authors state that the nine dimensions are not exhaustive (e.g., modality and multi-agent propagation can be extended); Source Label is not included in automatic annotation; Presentation Visibility does not actually render the carrier; the generation experiment uses only 40 seeds and a single benchmark suite; the appendix states that differences between original and generated cases are descriptive comparisons rather than causal effects.
Experimental setupGPT-5.4-mini、GPT-5.5 et al. · AgentDojo (Workspace, Banking, Travel, Slack)、IPIArena et al. · annotation accuracy、attack success rate et al.
Threat model
Indirect prompt injection: an attacker injects instructions into the context of a user agent (e.g., documents, web pages), influencing its control flow. The unit of classification is the combination of injected instruction, authorized user task, and system context (trust policy, available tools, retained state); the system is treated as a black box.
Evaluated models
GPT-5.4-miniGPT-5.5GPT-5.6-TerraGPT-5.6-LunaDeBERTaMeta SecAlign-70B
Benchmarks
AgentDojo (Workspace, Banking, Travel, Slack)IPIArenaLLMail-InjectWASPAgentPIDualViewSleeper MemoryTrojan HippoStepJack
Metrics
annotation accuracyattack success rateutility under attackfirst-attempt acceptance rate
AI summaryThe researchers propose characterizing differences among prompt injection attacks along nine dimensions, which together describe the injected instruction itself, its relationship to the user task, and its manifestation in the target system; the same instruction may be an injection in one context and a legitimate request in another, and ignoring this distinction leaves gaps in defense evaluation. The authors use this taxonomy to characterize six real-world prompt injection vulnerabilities and apply it to nine prompt injection benchmarks, finding that most benchmarks cover only a small part of the attack space and that after combining benchmarks, 13% of pairwise category combinations remain untested. The taxonomy also supports automatic labeling of benchmarks and targeted generation of new test cases to fill gaps; defense authors can use it to state the attack types they evaluate, and system operators can use it to identify attacks applicable to their systems and select matching defenses.

The researchers propose characterizing differences among prompt injection attacks along nine dimensions, which together describe the injected instruction itself, its relationship to the user task, and its manifestation in the target system; the same instruction may be an injection in one context and a legitimate request in another, and ignoring this distinction leaves gaps in defense evaluation. The authors use this taxonomy to characterize six real-world prompt injection vulnerabilities and apply it to nine prompt injection benchmarks, finding that most benchmarks cover only a small part of the attack space and that after combining benchmarks, 13% of pairwise category combinations remain untested. The taxonomy also supports automatic labeling of benchmarks and targeted generation of new test cases to fill gaps; defense authors can use it to state the attack types they evaluate, and system operators can use it to identify attacks applicable to their systems and select matching defenses.

Related events and news

1 events · 1 news mentions

News that mentions this paper

大模型安全周报:智能体安全边界在签名、授权与上下文

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    The authors argue that prompt injection is treated as a homogeneous threat, causing incomparable defense evaluations and gaps in benchmark coverage, and propose a nine-dimensional taxonomy.

    Core problem: Prompt injection is evaluated by defenses and benchmarks as a single homogeneous threat, whereas real attacks differ greatly, and whether the same instruction is an injection depends on user intent and system policy.

    • Problem setting and importance: Indirect prompt injection is regarded as a top security threat to AI agents (the authors cite OWASP). Homogenizing attacks has three consequences: robustness claims are incomparable (a low ASR on one benchmark may only test a small slice of the attack space); benchmark scale is mistaken for coverage (thousands of samples may exercise only a few attack attributes); and developers cannot match defenses to their own threat models.
    • Core observation: The authors use two observations to show that context cannot be ignored — injection is inseparable from user intent (the same sentence is content in a "summarize the document" task and a legitimate command in an "execute the document's instructions" task); whether an attack succeeds also depends on properties of the system and the injection itself (source trust label, visibility and interpretability, whether it persists to memory).
    • What the paper proposes: A nine-dimensional taxonomy whose unit of classification is the injected instruction together with the authorized user task and system context (trust policy, available tools, retained state), rather than an isolated injected instruction (Figure 1). Each dimension affects what a defense needs to observe, where it intervenes, and what assumptions it relies on.
    • Threat model (indirect prompt injection, as stated in the paper):
      • Attacker goal: By injecting instructions into external content (documents, web pages, emails, etc.), influence the agent's control flow, causing unintended actions, data exfiltration, or even account takeover.
      • Attacker capability: Controls the content of external sources that the agent will read; the paper treats the target system as a black box, using only externally observable or documented properties.
      • Victim system: An LLM agent (agentic harness) that processes untrusted content, including trust policy, tools, and retained state.
      • The paper does not set a single attacker knowledge level; the Source Label dimension distinguishes whether the injection arrives via an Untrusted source or a Trusted source.
  2. What related research exists?

    Related work is divided into three groups: formalizations of prompt injection, existing attack taxonomies, and the 9 benchmarks and 4 defenses analyzed.

    Formalizing Prompt Injection and Agent Security

    • Liu et al. (USENIX Security 2024) — models prompt injection with a target task and an injected task, providing a shared abstraction for attacks and defenses.
    • Task Shield (ACL 2025) — judges from a task-alignment perspective whether instructions and actions serve the user-specified goal.
    • Contextual security framework (Siu et al., arXiv 2026) — characterizes agent security with properties such as source authorization, task alignment, action alignment, and data isolation. The authors state that these works define "when a security violation occurs," and that this paper's goal is complementary: given an injection instance, characterize the properties of the attack itself.

    Prompt Injection Attack Taxonomies

    • Greshake et al. (AISec 2023) — characterizes indirect prompt injection by injection method and security impact.
    • Wang et al. (arXiv 2026) and McHugh et al. (Prompt Injection 2.0) — consider properties such as delivery vector, attack modality, propagation behavior, payload generation strategy, attacker capability, and payload visibility.
    • The Anatomy of a Prompt Injection (arXiv 2026) — decomposes the injection artifact into components such as carrier, delivery vector, concealment, payload, and exfiltration channel. The authors state that these frameworks are suited to describing how attacks are constructed and propagated, whereas this paper's unit of classification and purpose differ: rather than organizing attacks only by how malicious content is constructed, it characterizes the injected instruction within the user task and system context.

    Benchmarks and Defenses (Analysis and Experimental Subjects of the Paper)

    • Benchmarks: AgentDojo (NeurIPS 2025, four suites), IPIArena (public red-teaming competition with 41 scenarios), LLMail-Inject (208,095 samples), WASP (GUI injection), AgentPI (data flow manipulation), DualView (source label), Sleeper Memory and Trojan Hippo (memory poisoning), StepJack (multi-payload).
    • Defense baselines: CaMeL (dual-LLM architecture + tool-call policy), MELON (masks the user task to compare behavior), DeBERTa detector, PromptArmor (LLM detects and removes injection); Meta SecAlign-70B is also evaluated as an injection-resistant fine-tuned open-source model configuration.

    Authors' positioning: This paper's taxonomy is complementary to existing taxonomies; the difference is that the unit of classification includes the user task and system context, and each dimension must satisfy the two criteria "cannot be expressed by other dimensions" and "changing the category substantively affects defense coverage."

  3. How does the paper address this problem?

    The nine-dimension taxonomy uses "injected instruction + user task + system context" as the unit of classification, supplemented by an automatic annotation and targeted case-generation pipeline.

    Overall approach: characterize indirect prompt injection instances along nine dimensions (Table I), each satisfying two design criteria—capturing distinctions that other dimensions cannot fully express, and changing category having a substantive effect on defense coverage. On this basis, build an automatic annotation pipeline and a taxonomy-guided case generator (Figure 4).

    The nine dimensions (Table I)

    • 1. Source Label: whether the injection arrives via an Untrusted source or a Trusted source specified by the system policy; affects whether information-flow defenses apply.
    • 2. Fragmentation: Single-payload (one payload contains the complete attack) or Multi-payload (the attack requires multiple payloads that complement each other, same combination principle as Mosaic Prompts but a different threat model).
    • 3. Presentation Visibility: whether the injection is Visible or Hidden to the user in the specified inspection view; annotated separately from interpretability.
    • 4. Instruction Interpretability: whether an ordinary reviewer can directly understand it once exposed (Directly interpretable / Transformation required, e.g., Base64).
    • 5. Conditionality: Unconditional or Conditional (behavior depends on an explicit condition that can be triggered in later interactions).
    • 6. Agent State Modification: Non agent modifying or Agent modifying (the request persistently changes memory, preferences, skills, configuration).
    • 7. Topic Relation: Off-topic or On-topic (whether it targets the topic of the user task).
    • 8. Intended Effect: Tool-selection manipulation (adding, removing, swapping, reordering actions) or Content-only manipulation (only changing parameters and output content).
    • 9. Original-Task Completion: Task disrupting or Task preserving (whether the attacker's intent is to disrupt or to preserve completion of the original task).

    Annotation rules and information requirements

    • Unit of classification: the complete session input—injected instruction, authorized user task, explicit system context; the same injection can receive different labels under different tasks or systems.
    • Information-requirement groups (Table II): instruction-dependent (2, 4, 5), user-task-dependent (7, 8, 9), system-dependent (1, 3, 6; 8 also requires available actions). Dark categories can be excluded when the system does not support the corresponding attribute.
    • Intent rather than implementation: 6, 8, 9 are annotated according to the injection's design intent and do not vary with the agent's actual execution outcome; observed behavior should be reported separately after execution.

    Automatic annotation pipeline (Section VI-A, Appendix B)

    • Hybrid architecture: deterministic rules + LLM review. 2. Fragmentation is determined by payload structure, 4. Instruction Interpretability by encoding metadata or decoding checks; the remaining semantic dimensions are labeled by LLM review according to taxonomy definitions, category examples, and decision rules, giving a label and a one-sentence rationale, with output constrained to JSON.
    • Scope: 1. Source Label is not included in the current implementation (source trust assignment is not evaluated); 6 is annotated only when memory is enabled, 3 only when the carrier is marked as concealable.
    • Review models: GPT-5.4-mini, GPT-5.5, GPT-5.6-Terra, GPT-5.6-Luna; input representation, rules, and prompts are fixed across configurations. Appendix B gives the complete annotation prompts.

    Taxonomy-guided case generation (Section VI-B, Appendix C)

    • Input and process: given a seed case, its current labels, and a requested target configuration, the generator first checks whether the target label is feasible under the system context (dark categories are considered infeasible when the system does not support them), then makes local semantic modifications.
    • Modification mechanisms: 2, 5, 6, 7, 8, 9 via LLM-based semantic expansion (the attack payload and user task may be changed); 4 via programmatic representation transformation (currently Base64, hexadecimal, ROT13); 3 preserves the payload text and gives deployment instructions for placing it in the HTML layer, without actually constructing or rendering the carrier.
    • Validation: after generation, the same annotation pipeline rechecks whether the requested labels match. Appendix C gives the complete generation prompts and representative generated cases.
  4. What experiments did the paper conduct?

    Annotate 6 real attacks and 9 benchmarks, evaluate annotation accuracy on 4 models, and evaluate 5 defense configurations with 40 generated variants.

    Experimental Setup

    • Models under test: The targeted extension experiments use GPT-5.6-Luna as the agent backbone (chosen for its low inference cost and 100% utility, 0% ASR on the original cases); Appendix C extends to Llama 3.3 70B Instruct and GPT-4o mini; Meta SecAlign-70B serves as a separate injection-resistant fine-tuned model configuration.
    • Annotation review models: GPT-5.4-mini, GPT-5.5, GPT-5.6-Terra, GPT-5.6-Luna; the generator uses GPT-5.6-Terra.
    • Benchmarks: AgentDojo four suites, IPIArena, LLMail-Inject, WASP, AgentPI, DualView, Sleeper Memory, Trojan Hippo, StepJack, 9 in total; annotation units are attack instances, user task × attack goal combinations, or benchmark-level configurations.
    • Annotation evaluation samples: AgentDojo Workspace 560, Banking 144, Travel 140, Slack 105, IPIArena 95, Trojan Hippo 5, totaling 1,049 annotation instances; human annotations from Section V are used as reference labels, and the metric is annotation accuracy (including deterministic rules and LLM assignment).
    • Defense baselines: CaMeL, MELON, DeBERTa detector, PromptArmor, plus no-defense and SecAlign-70B configurations.
    • Attack evaluation metrics: attack success rate (proportion achieving the adversarial goal) and utility under attack (proportion completing the authorized task), evaluated separately; 40 attacked executions per point, single trial (Appendix C).

    Main Results

    Real attack annotations (Table III): 6 public cases collectively cover two categories of all nine dimensions, e.g., GitHub MCP is On-topic+Task preserving, Hidden Unicode Skill is Trusted source+Hidden, M365 Copilot is Conditional+Agent modifying, Devin is Multi-payload.

    Benchmark coverage (Figure 2, Figure 3):

    • Uneven single-dimension coverage: Most benchmarks concentrate on Visible, Single-payload, Directly interpretable, Unconditional, Non agent modifying; Hidden, Multi-payload, Conditional, Agent modifying are mainly represented by specialized benchmarks.
    • Combination coverage gaps: Of 144 dimension-pair combinations across the 9 binary dimensions, 19 (13.2%) are uncovered in the combined benchmark set; of 36 pure dark-category combinations, 15 (41.7%) are uncovered.
    • Scale is not coverage: The authors state that benchmark diversity is mainly determined by task design, attack scenarios, and injection wording, not by the number of instances.

    Targeted generation + defense evaluation (Figure 6, AgentDojo Workspace, 40 original + 40 generated, GPT-5.6-Luna):

    Swipe horizontally to read the table

    ConfigurationOriginal ASRGenerated ASRGenerated utility
    No defense0%17.5%Not listed separately in the main text
    CaMeL0%2.5%27.5%
    MELON0%20%Not listed separately in the main text
    DeBERTa0%7.5%40%
    PromptArmor0%7.5%90%
    SecAlign-70B0%17.5%Not listed separately in the main text
    • Generation pipeline acceptance rate: 35 of 40 seeds accepted on first pass (87.5%), the remaining 5 accepted within three attempts each (cumulative 100%).
    • The authors state that the generated cases expose attack success rate differences not visible on the original set, and distinguish configurations with the same ASR (7.5%) but different utility (40% vs 90%).

    Automatic Annotation Accuracy (Figure 5)

    • Deterministic dimensions all correct: 2. Fragmentation and 4. Instruction Interpretability are fully consistent across the four model configurations (shared deterministic rules).
    • Semantic dimensions differ: GPT-5.5, GPT-5.6-Terra, GPT-5.6-Luna generally outperform GPT-5.4-mini on 7. Topic Relation and 9. Original-Task Completion, with the largest difference on Banking; the authors state that the low-cost GPT-5.6-Luna and the mid-priced GPT-5.6-Terra have broadly comparable accuracy on most dimensions and subsets, and that a higher price does not necessarily yield better annotation.
    • Trojan Hippo: 5 cases expose more inter-model differences on 5. Conditionality; all four models correctly identify 6. Agent State Modification.

    Extended Model Experiments (Appendix C, Figure 9)

    • Llama 3.3 70B: No-defense ASR rises from 5% (original) to 25% (generated), utility falls from 45% to 35%; on the generated set, CaMeL ASR 2.5%/utility 25%, PromptArmor 5%/35%, MELON 20%/35%, DeBERTa 12.5%/22.5%.
    • GPT-4o mini: No-defense ASR rises from 0% to 45%, utility falls from 85% to 70%; on the generated set, CaMeL 5%/32.5%, PromptArmor 12.5%/75%.
    • The authors note: user tasks, attack goals, and success criteria may differ between original and generated cases; the shifts in the figure are descriptive comparisons rather than isolated causal effects.

    Other Ablations and Analyses

    • Annotation cost (Appendix B): GPT-5.4-mini $2.47, GPT-5.5 $25.95, GPT-5.6-Terra $7.46, GPT-5.6-Luna $0.91 (full 1,049 instances, at 2026-09-28 standard uncached API rates).
    • Generation cost (Appendix C): generation stage for 40 seeds $0.0223 (GPT-5.6-Terra).
    • Annotation error analysis (Appendix B, Listings 2–5): 4 disagreement cases, involving shared action vs shared goal, parameter substitution vs plan extension, extra action vs task substitution, explicit trigger word vs established premise; the authors state these are examples and do not estimate the prevalence of each failure mode.
  5. What are some points that can be further explored?

    The authors state that the nine dimensions are not exhaustive and can be extended; the experimental coverage is limited to 40 seeds, a single benchmark suite, and descriptive comparisons.

    Limitations and Future Directions Noted by the Authors

    • Dimensions not exhaustive (Section VIII): The authors state they do not claim the nine dimensions exhaust all differences in prompt injection; dimensions can be added as agent architectures, interaction modalities, and attack techniques evolve, e.g., carrier modality (text, image, audio) for multimodal agents, and injection propagation for multi-agent systems; new dimensions must satisfy the two criteria in Section II.
    • Source Label not implemented (Section VI-A): The current automatic annotation implementation excludes 1. Source Label because it does not evaluate source trust assignment.
    • Visibility not actually rendered (Section VI-B): The implementation of 3. Presentation Visibility retains the payload text and gives placement instructions, without constructing or rendering the resulting carrier.
    • Open science plan (Open Science section): The authors plan to release benchmark annotations, annotation and generation pipelines, prompts, generated cases, evaluation code, and an interactive demo.
    • Call for community action (Section VIII): The authors call on benchmark authors to report the taxonomy categories and combinations they cover (including intentionally excluded and undeterminable attributes), and on defense authors to clearly distinguish categories that were evaluated, can be mitigated, and are out of scope.

    Scope of Experimental Coverage

    • The real attack case analysis covers 6 publicly documented cases (Table III); the authors state it is not a comprehensive survey of disclosed attacks.
    • The benchmark analysis covers 9 benchmarks; the full AgentPI dataset is not public, and annotation is based on the design and attack configurations described in the paper; for WASP and StepJack, 8. Intended Effect is marked unknown because GUI actions lack semantic tool labels.
    • Annotation accuracy is evaluated on 1,049 instances (AgentDojo four suites 949, IPIArena 95, Trojan Hippo 5); Source Label is not included in the evaluation.
    • The targeted generation experiments use all 40 user tasks in AgentDojo Workspace, with one attack task randomly sampled per task as the seed, and the target combination fixed as Conditional+On-topic+Content-only manipulation; each data point is 40 attacked executions, single trial.
    • The paper does not report agreement metrics between annotation review and human annotation (e.g., Cohen's kappa), only annotation accuracy; it does not report the number of repetitions for attack evaluation (Appendix C states single trial).
  6. Summarize the main content of the paper

    The nine-dimension taxonomy characterizes prompt injection back within the user task and system context, and is used to analyze attacks, audit benchmark coverage, and generate targeted test cases.

    A paper on a taxonomy and evaluation-coverage analysis of indirect prompt injection. The authors argue that defenses and benchmarks treat prompt injection as a homogeneous threat, which makes robustness claims incomparable, causes benchmark scale to be mistaken for coverage, and prevents defenses from being matched to threat models.

    • Method: Proposes a nine-dimensional taxonomy (source label, fragmentation, presentation visibility, instruction interpretability, conditionality, agent state modification, topic relation, intended effect, original-task completion); the unit of classification is the injected instruction together with the authorized user task and system context; each dimension must satisfy two criteria: it cannot be expressed by other dimensions, and changing the category substantively affects defense coverage. Accompanied by an automatic annotation pipeline combining rules and LLM review, and a taxonomy-guided test case generator.
    • Real attacks: Annotation of 6 public cases collectively covers two categories of all nine dimensions, showing differences among attacks under the same "prompt injection" label (Table III).
    • Benchmark coverage: Among 9 benchmarks, 19 of 144 dimension-pair combinations (13.2%) are uncovered, and 15 of 36 pure dark-category combinations (41.7%) are uncovered (Figure 3); the authors state that benchmark scale does not represent coverage breadth.
    • Annotation pipeline: On 1,049 instances, deterministic dimensions are consistent across all models; GPT-5.6-Luna and GPT-5.6-Terra have broadly comparable accuracy on most semantic dimensions (Figure 5).
    • Targeted generation: First-pass acceptance rate for 40 AgentDojo Workspace seeds is 87.5%; generated variants raise GPT-5.6-Luna no-defense ASR from 0% to 17.5%, CaMeL 2.5% (utility 27.5%), PromptArmor 7.5% (utility 90%) (Figure 6); the appendix extends to Llama 3.3 70B (no defense 25%) and GPT-4o mini (45%).
    • Authors' conclusions: Benchmark scale matters less than coverage; safety claims should be tied to specific attack attributes in the taxonomy and to the affected systems; the taxonomy is designed as an extensible framework rather than a strict enumeration.

Related papers

Show 3 more papers
Both use: WASP
Read originalarxiv.org