SoK: PIAtlas Decomposes the Unbounded Prompt Injection Attack Space

1. 01What problem does this paper attempt to solve?

   Unbounded prompt injection lacks a map divided by construction; the authors organize attacks into six mechanism classes and compare them against benchmark coverage.

   Prompt injection lacks a map divided by construction mechanism that can be compared against benchmark coverage.

   • Scenario: The authors argue that the capabilities of applications and agents come from the fact that any token in the context can influence the next step, so untrusted data is treated as high-authority instructions. The authors cite public statements to the effect that, in practical terms, this problem can only be contained, not solved.
   • Search difficulty: The authors write that with k=50 and a vocabulary size of 10^5, the number of attacker-controllable sequences is on the order of 10^250. Uniform sampling mostly lands in invalid regions, while perturbing only near known attacks cannot leave the original region.
   • Existing shortcomings: The authors state that more than twenty public datasets have mutually incompatible vocabularies and are often organized by delivery, target, or generation method; the main axes of prior classifications are also inconsistent. Early work close to construction either enumerated techniques or derived only a few categories, and predates the agent era.
   • Observation: The authors argue that effective injections follow a relatively small number of construction strategies; the same strategy can be reached by paraphrase, and different strategies are separated by gaps that local search rarely crosses. The authors conjecture that, under a representation aligned with construction, successful injections lie in a union of low-dimensional submanifolds indexed by construction.
   • Approach: The authors propose PIAtlas, with six classes each corresponding to a subversion mechanism, and release PICorpus, which merges twenty public datasets. Delivery, target, and search method do not enter this axis.
   • Threat model: Section II-D assumes a black-box adversary.
     • Goal: Introduce adversarial text x into the context to override the runtime system policy; a fixed binary classifier JUDGE(R, O) determines whether the response satisfies O under the violation.
     • Knowledge: Knows whether the system prompt template belongs to a chatbot, assistant, or agent, but not the exact text.
     • Capability: In DPI, the adversary is the end user, controlling medium-authority user messages; in IPI, an external party controls the no-authority data segments subsequently ingested. x can be generated by any manual or automated means.
     • Outside the axis: Direct or indirect placement belongs to delivery. Target categories and search methods (including black-box or white-box optimization) are not classified by PIAtlas.

   Collapse this answer

2. 02What related research exists?

   The authors state that existing classifications mostly do not split by construction, and that schemes close to construction also do not divide along the same mechanism boundaries.

   What the paper discusses in the introduction and Section VIII is mainly the classification scheme, the criteria, and the corpus and representations used for comparison in this paper.

   Other axes of division and earlier partitions

   • Multi-axis classifications and standards: the main axes the authors list include threat model, payload generation, delivery, kill-chain stage, dominant mechanism, belief/intent/permission, CIA impact, and root cause of training failure. NIST AI 100-2, MITRE ATLAS, and OWASP's LLM and agent lists each also cut along one axis; the authors state that apart from the direct/indirect distinction, there is little in common.
   • Goals and prompt blocks: Perez and Ribeiro (2022) divide attacks into goal hijacking and prompt leaking, and compose attacks from reusable prompt blocks. The authors count this as adjacent to construction, but the scheme is organized by goal.
   • Defense families: the introduction describes instruction hierarchy and preference optimization, prompt-level isolation, inference-time detection, and control-flow and data-flow designs that do not depend on model behavior as mitigation rather than a fix. The authors state that the strongest design guarantees still leave instruction–data confusion at the model layer.

   Schemes close to construction

   • HackAPrompt (2023): enumerates 29 techniques from the literature and a competition. The authors state that it is not derived from the model, predates the agent era, has no social engineering class, and is not a strict partition.
   • Components and early categories: HouYi (2023) and Open-Prompt-Injection (2024) are derived and composable; the authors state that the latter names only this paper's F1 and F2, and HouYi adds only language switching and format requirements. The construction families of Rossi et al. (2024) anticipate several classes but mix in the delivery axis. Gandalf's categories cover five classes but serve only one goal.
   • Language layer and bypass mechanisms: Rao et al. (2024), Attack Atlas (2024), Giarrusso et al. (2026) and MLCommons (2026) are closer to this paper's axis. The authors state that none of these four covers agents, and the first three allow categories to overlap. HiddenLayer APE and Iyer et al. (2026) cover direct and indirect injection but put delivery inside the axis.

   This paper's comparisons

   • Cross-comparison: the authors place the categories of 44 schemes on the construction axis. The authors state that construction appears in every scheme, but no two schemes cut it the same way; of 1,488 categories, 779 correspond to at least one class, 709 fall on other axes, and no category has yet required a seventh class.
   • Corpus: the comparison records come from 20 public datasets, merged into PICorpus, not another attack-algorithm baseline.
   • Representation comparison: the embedding experiments compare F2LLM without a construction instruction, Qwen3-Embedding-8B, and grouping by benchmark.

   The authors state that, to their knowledge, PIAtlas is the first taxonomy derived from a single construction axis, mutually exclusive by mechanism, covering direct and indirect injection and standalone and agentic LLMs, and treating multi-label combinations as an attack property; the closest schemes lack at least one of these.

   Collapse this answer

3. 03How does the paper address this problem?

   PIAtlas derives six classes from six parts of the text read by the model, one mechanism per class, and attacks can be combined with multiple labels.

   PIAtlas does not first list attacks and then group them; instead it asks: with the request fixed, which parts of the injected text can the attacker still change so as to change one decision the model makes before acting.

   Problem setup

   The context is C=⟨s, u, d⟩, containing a system prompt, a user message and data segments. The injection transformation T(C, x) embeds x in the user message (direct) or in some data segment (indirect). The request determines the target O, and the payload is the words stating that request; x also has delivery and construction. Construction is defined by six parts: readability, surface provenance, the instruction itself, justification for compliance, context, and response form. If the request and delivery are the same and the six parts match, the construction is the same. Placement is metadata only.

   The adversary's objective is sup_{x∈ V^k}Pr_{R∼LLM(T(C,x))}[JUDGE(R,O)=True] i.e., over sequences of length k, maximize the probability that a random response is judged to achieve O. The set of successful injections is defined by a threshold τ. The authors write the six classes as a heuristic approximation of the attack space; coverage is a falsifiable hypothesis, not a closed guarantee.

   The six classes

   The six decisions are a simplified description of behavior, not an architectural model. Each class changes only one part.

   • F1 instruction manipulation: changes the instruction itself, corresponding to post-trained instruction following and weaker instruction–data separation. An instruction that does not disguise, hide, give reasons, change context, or constrain the response carries only F1.
   • F2 structural forgery: changes surface provenance, exploiting trust in the dialogue format, including forged role tags, control tokens, marking text as trusted data or as a notice to the model, and forged turns and tool results.
   • F3 obfuscation: preserves meaning while changing the inspectable form. The authors map this to the gap that detection and adversarial training mostly cover ordinary text while the model can still read the rewritten text.
   • F4 social engineering: gives reasons for compliance in prose. Techniques borrow from human persuasion strategies.
   • F5 situational framing: changes the world or role against which compliance is judged, including personas, simulation, fiction, hypotheticals, games, or claiming the model's own mode has changed.
   • F6 output forcing: constrains the response form, e.g., specifying an opening, format, length, or prohibiting refusal, clarification and reasoning.

   Table IV lists, for each class, the defense levers the authors consider targeted at it. This is a correspondence in the taxonomy; the paper does not evaluate these defenses. Hard boundaries follow "which part the words change": authority in tags, tokens, format or forged tool output is F2, the same claim in prose is F4; a persona given to the model is F5, a human claiming a role is F4; prohibiting refusal is F6, arguing against refusal is F4. Methods that hide from humans only via rendering, while the model still receives ordinary text, count as delivery, not F3.

   Hierarchy, signatures and revision

   The current hierarchy is 6 classes, 34 techniques, 131 sub-techniques. A technique must be visible in the text, change only that class's part, act on the model rather than a human reader or detector, and not be subsumed by another technique in the same class. Sub-techniques are surface forms in public sources, for illustration rather than exhaustiveness.

   The test does not guess intent. Relative to the default of a bare command, deviating in that class's part passes. Every class for which it passes is recorded; when no other test passes, it still gets F1. The order is to first decode hidden text (F3), then check F2, F6, F5, F4, and finally assign F1. The class label is the shortest span. An attack often has multiple spans, so multi-label is a property of the data, not a boundary defect. Only obfuscation can share the same span with another class.

   Adding a class must satisfy four conditions at once: the mechanism is not covered, it cannot be rewritten as a technique of an existing class, it recurs in at least three independent public sources or disclosed incidents, and it has a different defense lever. If the model reads two classes as the same decision, or all techniques of one class can be merged into another without changing the mechanism, they are merged. Multi-turn escalation, many-shot priming, text in images or audio, and gradient-optimized strings are placed by the authors into existing classes or into delivery and search methods outside the axes.

   Corpus and annotation

   The authors surveyed public datasets from January 2022 to May 2026, keeping sets that are downloadable, MIT-licensed, annotate attacks in some form, and focus on injection or allow injection to be filtered out, yielding 20. Records were put in a unified format, byte-exact duplicates removed, and templates separated from payloads where the text allows. The unit is a single attack instruction. Benchmarks contribute templates, competitions contribute submission texts; success labels, delivery and original labels are metadata only. The jailbreak boundary is applied per dataset: without an s or u to override, it is not injection.

   Annotation was chosen for coverage rather than quantity: all curated templates outside LLMail-Inject, plus a random sample of Phase-1 LLMail-Inject templates. Claude Opus 5.5 does closed-set annotation, returning every technique present and citing the original text. Appendix D gives the setup, and Appendix G lists techniques, sub-techniques and examples. Section VI then asks whether the same construction is nearby in sentence-vector space; the encoder is a design to be searched, not the target model's latent space.

   Collapse this answer

4. 04What experiments did the paper conduct?

   The authors state that curated attacks mostly stack only a few classes; LLMail-Inject samples more often use F2, F3, F4, which appear less often in benchmarks.

   Experimental setup

   • Annotation scope: 8,663 entries. The curated partition is all 4,663 templates other than LLMail-Inject, from 13 benchmarks; the adaptive partition is a random sample of 4,000 Phase-1 LLMail-Inject templates. Phase 2 is excluded.
   • Annotator: Claude Opus 5.5 (medium effort), called separately for each entry, closed set of 34 techniques, with cited evidence words. Labels are multi-label.
   • Manual check: The first author annotated a stratified sample of 300 entries without knowing the production labels or the source. There was no second annotator. There were also two runs on 64 entries, used for stability.
   • Embedding: F2LLM-v2-14B, in two variants: without instruction and with a fixed constructed instruction; the comparison encoder is Qwen3-Embedding-8B. The objects are the 4,663 payload-stripped templates. The centroid method fixes k=33. Metrics are NMI, ARI, AMI, and silhouette coefficient is also reported.
   • Other readings: 13 public incidents from 2024–2026 are mapped to classes; 44 existing schemes with 1,488 categories in total are cross-compared. PICorpus is not treated as a leaderboard.
   • Not reported: The paper does not report an end-to-end interception rate for defenses, nor a specific ASR for class-guided red teaming.

   Main results

   The percentages in the table below come from Table VI. A template can carry multiple classes. Cov. counts classes reaching 5% or above before rounding. F1 is 92 to 100 in every row of the table, so it is not listed separately.

   Swipe horizontally to read the table

   Source (Table VI)	n	F2	F3	F4	Cov.
   ASB	2,000	40	0	0	2/6
   InjecAgent	105	20	0	0	2/6
   AgentDojo	36	94	8	89	4/6
   BrowseSafe	1,079	33	5	82	5/6
   PIArena	495	42	15	40	5/6
   CyberSecEval 2/3	154	13	18	27	6/6
   All curated	4,663	35	5	29	6/6
   LLMail-Inject sample	4,000	76	69	67	5/6

   • Sources not listed: TaskTracker, SEP, AgentDyn, Greshake et al., Open-Prompt-Injection, PromptInject, Agent-SafetyBench. The latter two and Open-Prompt-Injection also have Cov. of 2/6. Agent-SafetyBench has Cov. of 6/6, and the authors state that 20/24 of its templates are jailbreaks and four injection templates carry only F1.
   • The authors' interpretation of the benchmarks: The authors state that the agent benchmarks commonly evaluated by recent defenses exercise only two to four classes; AgentDojo, AgentDyn, InjecAgent and ASB all lack F5 templates. Six sources have no output-forcing templates at all. CyberSecEval and Greshake et al. cover six classes, but have 154 and 20 templates respectively.
   • Frequency and adaptive: The main text gives finer proportions for the curated partition: F1 99.2%, F2 35.3%, F4 29.5%, F5 13.6%, F6 8.4%, F3 5.4%. The authors state that frequency reflects what the dataset chooses to generate, not danger level. The adaptive comparison merges the Phase 1 scenarios and defense levels; the authors state that attackers who can observe defenses more often forge sources and hide payloads.

   How deep are the combinations

   • What was measured: Multi-label combinations in the curated partition, not counting the near-duplicate LLMail-Inject templates in the frequency.
   • Results: The median spans 2 classes and 3 techniques; 38.3% are single-class; 77% use one or two classes; no template uses more than five classes. F3 appears alone in 4/252, F6 in 3/391. Common pairs include F1+F2 with 1,633 entries and F1+F4 with 1,360 entries.
   • The authors' interpretation: The authors state that combinations are the norm, but the stacking is shallow, and the classes co-occurring with a given class form a small menu rather than an open combinatorial space.

   Do the annotations match manual annotation

   • What was measured: Production labels against the first author's 300 entries, and two runs with one call per entry on 64 entries.
   • Results: Class-set micro-F1 is 0.97, technique-set 0.92; the two runs are 0.99 and 0.97 respectively (Table III). More than 99.5% of citations appear verbatim in the records. Only 56% of records have exactly the same technique set. On the four pre-specified boundary pairs, only two records had class swaps (Table XIX).
   • The authors' interpretation: The authors state that the findings are computed at the class level, and residual annotation uncertainty is a minor limitation. Conditional analyses by individual technique should account for technique-set noise.

   Can the constructions be recovered from embeddings

   • What was measured: 4,663 payload-stripped templates. The guiding instruction asks for rhetorical strategy rather than topic or task. The reference partition is 631 class+technique sets. Figure 1 is a two-dimensional plot of guided F2LLM via PCA then UMAP, with clusters from unlabeled k-means on the two-dimensional coordinates.
   • Results: After guidance, k-means (k=33) gives NMI, ARI, AMI of 0.609, 0.387, 0.514, with silhouette coefficient 0.122; unguided F2LLM gives AMI 0.363, Qwen3-Embedding-8B 0.261 (Table VII). Five-seed average NMI 0.607±0.012, ARI 0.404±0.048. AMI grouped by benchmark is 0.40.
   • The authors' interpretation: The authors state that guidance rather than scale makes the constructions more recoverable, and that these scores are not accuracies. The authors state that exact pairwise recovery of the fine-grained multi-label partition remains moderate. Across benchmarks, identical technique sets almost never co-cluster, which the authors attribute to each benchmark's wording still shaping the geometry. Inter-class exploration and intra-class exploitation are written as a design inference, not a demonstrated speedup.

   Disclosed incidents

   • What was measured: 13 publicly named incidents. Methods that hide from humans only via rendering while the model still receives plain text count as delivery, not F3.
   • Results: None requires more than three classes. F1 is present in every one; F2 in five, or six counting documented variants; F4, F5, F6 in two each; F3 in one, or two counting variants.
   • The authors' interpretation: The authors state that this is not a reason not to cover F3 to F6 in benchmarks. The incident set is small and skewed toward tool-connected systems; in many incidents what matters is delivery, including rendering, rather than construction.

   Other ablations and analyses

   • After manual correction, the adaptive partition's F3 goes from 68.7% to 63.2%, F4 from 66.6% to 59.3%; the direction of Finding 4 remains, with F2 higher than the curated partition by 41.7 percentage points (95% interval 38.9 to 45.1) (Table XVI).
   • Five-seed AMI after removing ASB: guided 0.37, unguided 0.33, Qwen3-Embedding-8B 0.30 (Appendix C).
   • HDBSCAN on guided F2LLM gives 2 clusters, 7% noise, NMI 0.116, AMI 0.093 (Table VII); the authors state it is not comparable to k-means, which assigns all templates.
   • Table XIV: Among the ten clusters in Figure 1, pairs from the same benchmark with the same technique set co-cluster 97.8%, different technique sets 39.3%; identical technique sets across benchmarks 0.3%, random pairs 15%.
   • Different submissions reaching the goal in the two competitions are 4,381/196,196 (2.2%); LLMail-Inject is 2,027/148,514 (1.4%) (Table XI). The authors place this alongside the sparsity hypothesis.
   • Section IX's preliminary red team rewrites AgentDojo injections, with targets including GPT-6 Sol, GPT-6 Luna and Qwen 3.8 27B. The paper does not report a specific ASR, only stating that single-attempt ASR falls within the range GPT-Red reports for its target models, and that the proportion of tasks succeeding at least once is high.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors state that the six classes are a heuristic approximation, that the adaptive conclusion comes from a competition, and that they plan to use it for red teaming and adversarial training.

   Limitations noted by the authors and future directions

   • The six classes are not a proof: they are based on the authors' current understanding of the six decisions a model makes before acting, not a proof that no other part operates. If text changes compliance through a seventh part, a class should be added. The cross-comparison of 44 schemes is evidence, not proof that there is no seventh class (Section IX, pointing to IV-A, IV-E, VIII).
   • Annotation: class micro-F1 of production labels against the human reference is high, and the authors state that residual uncertainty is a minor limitation. The human reference has only one annotator, so inter-annotator agreement cannot be measured; the authors did not compare model families; the closed-source annotation model cannot be exactly reproduced by rerunning (Section IX).
   • Meaning of frequency: class frequencies reflect what public datasets sample, not real-world prevalence. The adaptive-attacker finding is based on one competition, and the authors believe it should be reproduced on other adaptive sources (Section IX).
   • Construction is not everything: classifying only by construction abstracts away differences in exploitability across models; most disclosed incidents pair ordinary construction with delivery tricks, and delivery often involves rendering (Section IX, pointing to VII).
   • Red team and adversarial training: the authors plan to use the taxonomy graph to drive automated red teaming so that training data covers every subversion mechanism, and to extend preliminary results to more agents and deployment settings; then to retest by class which mechanisms remain open (Section IX).
   • Representation and search: the authors state that a representation fully aligned with construction remains an open problem. A controlled comparison of class-guided versus unguided mutation has not been done, and the two-level search is written as a design inference, not a demonstrated speedup (Section VI-C, IX, X).

   Scope of the experiments

   • Corpus: 20 datasets from January 2022 to May 2026 that are downloadable, MIT-licensed and carry some form of attack label; 868,374 items ingested, and 444,498 distinct attack instructions after removing exact duplicates (Section III, Table XI).
   • Labels: PIAtlas labels cover 8,663 items, i.e., 4,663 templates from 13 benchmarks plus a random sample of 4,000 LLMail-Inject Phase-1 items; Phase 2 is excluded (Section III-C, Appendix D).
   • Check: the first author annotated a stratified sample of 300 items; two-run stability was measured on 64 items. The annotation model is Claude Opus 5.5. The paper states that inter-annotator agreement cannot be obtained from this (Appendix D, Section IX).
   • Embedding: the above 4,663 payload-stripped templates, three encoder settings, four clusterers; Figure 1 comes from guided F2LLM, then PCA and UMAP (Section VI, Appendix C).
   • Incidents and taxonomy comparison: 13 public disclosures from 2024–2026; 1,488 categories from 44 schemes. The paper does not report a specific ASR for the class-guided red team (Sections VII, VIII, IX).

   Collapse this answer

6. 06Summarize the main content of the paper

   PIAtlas organizes prompt injection with six construction classes; the authors state that benchmark coverage and the mechanisms used by adaptive attackers are not the same.

   This is a systematization of knowledge on prompt injection: attacks are classified by the construction of the injected instruction, not by delivery, target, or how the text was found.

   What the authors address is unbounded search. With k=50 and a vocabulary size of 10^5, the number of controllable sequences is on the order of 10^250. The authors argue that effective injections follow a relatively small number of construction strategies, and conjecture that, under a representation aligned with construction, successful injections lie on several low-dimensional submanifolds. The classification is written as a heuristic approximation and a falsifiable hypothesis.

   PIAtlas writes the processing of an instruction in the context by the model as six decisions, each reading one part of the text, yielding six classes: instruction manipulation (F1), structural forgery (F2), obfuscation (F3), social engineering (F4), situational framing (F5), and output forcing (F6). Beneath these are 34 techniques and 131 sub-techniques. The classes are mutually exclusive in mechanism, yet real attacks are often multi-label. The threat model is black-box: DPI controls user messages, IPI controls no-authority data segments, and the class of the system prompt is known but not its original text.

   PICorpus merges 20 MIT-licensed public datasets and, after deduplication, contains 444,498 distinct attack instructions. Of these, 8,663 were labeled with a closed set by Claude Opus 5.5, and the first author spot-checked 300.

   The main numbers the authors report are: curated templates have a median spanning 2 classes and 3 techniques, and 77% use only one or two classes; F1 accounts for 99.2% of curated templates, and F3 for 5.4%. The authors state that several agent benchmarks cover only 2 to 4 classes, and that AgentDojo and others have no F5 templates. In Table VI, for 4,000 LLMail-Inject samples, F2, F3, and F4 are 76, 69, and 67, versus 35, 5, and 29 for the curated aggregate. Guided F2LLM achieves an AMI of 0.514 on k-means with k=33, versus 0.363 without instructions and 0.261 for Qwen3-Embedding-8B. None of the 13 disclosed incidents required more than three classes.

   The authors' conclusion is: the attack space is unbounded, but not unstructured. Red teams can first cover the six classes, then perform surface variation within classes, and combine them according to empirical co-occurrence. Coverage should be judged by which classes were exercised, not by how many prompts were viewed. The authors plan to use this map to generate adversarial training data covering each class, and to continue revising the classification and learning representations more aligned with construction.

   Collapse this answer

Related papers