Identify, Classify, Act: Per-Instruction Controlled Compliance of LLMs with External Data

1. 01What problem does this paper attempt to solve?

   External data mixes beneficial instructions with prompt injection, and EDIC requires identifying, classifying and selectively executing each one individually.

   When external data contains both beneficial instructions and adversarial prompt injection, the model must obey or refuse each one individually, rather than refusing the whole passage or trusting the whole passage.

   • Where it occurs: The authors state that LLMs are already the reasoning core of retrieval augmentation, browser assistants, tool-calling agents and multi-step workflows, and must process external data outside their control. The same document or tool return interleaves beneficial instructions such as API format rules and medical procedure guidance with injections that override user goals, extract private data, or redirect to unrelated goals (Figure 1).
   • Existing shortcomings: The authors state that existing defenses mostly treat this as a binary problem: refuse all embedded instructions, or fully trust them. Instruction hierarchy, structured query isolation and external injection classifiers can improve robustness to malicious instructions, but they indiscriminately suppress external instructions, causing over-refusal and degrading legitimate task performance.
   • Core observation: The authors argue that what is missing is per-instruction credit attribution, and formalize the problem as External-Data Instruction Classification: identify–classify–act for each embedded instruction.
   • This paper's approach: EDIC uses a scenario-anchored pipeline to inject beneficial and harmful instructions and performs counterfactual filtering, then trains a generative judge and a CoT response model; decomposed rewards separately encourage beneficial obedience and penalize harmful obedience.
   • Threat model:
     • Attack goal: Make the model deviate from the user query, perform unsafe actions, or redirect to unrelated goals.
     • Attacker knowledge: The paper does not state a white-box, black-box or no-box assumption.
     • Attacker capability: Malicious instructions are hidden in external documents or tool returns, not in the user prompt. The ethics statement states that trigger words come from publicly released benchmarks, and the authors state that no new attacks were introduced.
     • Victim system: An LLM oriented toward external data. The expectation is to obey beneficial instructions and refuse harmful ones; silently executing harmful instructions or refusing beneficial instructions both count as failure.

   Collapse this answer

2. 02What related research exists?

   The authors state that existing defenses reject external instructions wholesale and do not model the coexistence of beneficial and harmful instructions in the same document.

   Related work is organized around indirect prompt injection, whole-passage defenses and process supervision. The authors place the gap in the coexistence of two types of instructions within the same document.

   Indirect prompt injection

   • Injection in external content: Greshake et al. (2023) analyze indirect prompt injection hidden in external content rather than in the user prompt in application-integrated LLMs.
   • Jailbreak clues: The introduction cites Wei et al. (2023), Zou et al. (2023), Yu et al. (2023) and Zhu et al. (2023), showing that adversarial prompts can control aligned models. The paper does not use these jailbreak methods as experimental baselines.
   • Workflow benchmarks: BIPIA (Yi et al., 2025), AgentDojo (Debenedetti et al., 2024) and InjecAgent (Zhan et al., 2024) extend the threat to retrieval and agent workflows.

   External instruction defenses

   • Priority and decoding: Wallace et al. (2024)'s instruction hierarchy, Zhang et al. (2024)'s goal priority, Xu et al. (2024)'s safety-aware decoding and Xie et al. (2023)'s self-reminder aim to maintain high-level constraints under low-level adversarial input.
   • Shortcomings named by the authors: The authors state that these methods, including the closest, ReasAlign (Hao et al., 2026), almost exclusively treat external instructions as a threat to be refused wholesale, and do not model the mixed setting where beneficial and harmful instructions appear in the same document.

   Reasoning supervision and alignment

   • Intermediate reasoning: The paper states that CoT and related prompting (Wei et al., 2022; Kojima et al., 2022; Wang et al., 2023a; Yao et al., 2023a) show that intermediate reasoning helps multi-step tasks. On this basis the authors argue that instruction identification, source attribution and permission-sensitive decisions should be modeled as a process.
   • Behavior shaping: Instruction tuning and alignment work such as RLHF, DPO, RRHF and Constitutional AI are grouped together (Mishra et al., 2022; Ouyang et al., 2022; Bai et al., 2022a,b; Rafailov et al., 2023, etc.). The paper states that process supervision by Lightman et al. (2024) shows that rewarding intermediate reasoning can outperform supervising only outcomes.
   • The authors' distinction: The authors state that none of the above settings involve external context where beneficial and malicious instructions coexist; EDIC extends process supervision to per-instruction behavior attribution.

   Baselines and data

   • Baseline methods: Base, plus SecAlign, StruQ, PromptGuard, Instruction Hierarchy, Meta-SecAlign (Llama only) and ReasAlign. Appendix A describes each baseline. Table 1 actually lists Base, SecAlign, StruQ and ReasAlign, with Meta-SecAlign additionally for Llama.
   • Scenarios and evaluation: Training scenarios come from Spam email, API-Bank, EPFL clinical guidelines, HotpotQA and Wiki-MultihopQA. Utility evaluation uses MT-Bench and Arena-Hard-Auto; safety evaluation uses AgentDojo, CyberSecEval 2's prompt-injection and InjecAgent; over-refusal uses NotInject.

   The authors position this paper on two gaps: existing defenses almost wholesale refuse external instructions, and process supervision does not handle mixed instructions.

   Collapse this answer

3. 03How does the paper address this problem?

   EDIC trains a generative judge with mixed instruction data, then optimizes a CoT policy with per-instruction decomposed rewards.

   EDIC decomposes controlled obedience into identify, classify, and act: it first synthesizes mixed instructions in real documents, then has a generative judge score each instruction for obedience, and uses that score to train a CoT policy.

   Problem setting

   • Three types of segments: the user query is q, the external data is D, and the segment set is I. Each ek belongs to beneficial, harmful, or contextual. Beneficial instructions are task-relevant and safe; harmful instructions attempt to override q or trigger unsafe behavior; contextual segments provide information and have no directive intent.
   • Three-step decision: extract I from D; assign a type to each ek; the action is follow or reject, and the response y reflects these decisions. Silently executing a harmful instruction or rejecting a beneficial instruction is a failure.
   • Formal objective: the paper maximizes beneficial obedience and subtracts λ times harmful obedience, with λ>0. The SFT description in Section 4.2.2 labels segments as beneficial or harmful; the synthesis prompt in Appendix F.8 instead requires classification into BENEFICIAL, HARMFUL, or CONTEXTUAL.

   Data synthesis

   • Scenario anchoring: the four scenario types are anchored to Spam email, API-Bank, EPFL clinical guidelines, and HotpotQA and Wiki-MultihopQA, respectively. The pipeline preserves the original style, entities, and terminology, and only rewrites directive segments into natural sentences in context.
   • Three variants and four branches: the same source is made into a version with only beneficial instructions e+, a version with only harmful instructions e−, and a version with both. A short adversarial phrase is prepended to harmful instructions as a behavior switch; Section 4.1 gives examples, which are not repeated here. The four outputs are: complete the task and obey e+, complete the task but ignore the embedded instruction, obey e− and deviate from the user goal, and obey both e+ and e−. The correct branch is the policy target, and the four branches provide contrasts for the judge.
   • Filtering: removing e+ should change the response (necessity), and removing e− should not change it (independence). Samples scoring 5 on necessity, consistency, and naturalness are retained; Appendix F.6 removes samples with an independence score above 2. After filtering, each scenario has 900 samples, split 150/250/500 into validation/judge training/policy training. Cold-start reasoning is synthesized with GPT-5.4 at temperature 1.0.

   Generative judge

   • Interface: the judge receives (q, D, ek, y) and outputs reasoning and a binary decision. J_ϕ(q,D,e_k,y)→(r,ℓ̂), where the label is 0 or 1, indicating whether y obeys ek.
   • Training: first SFT on judge data to learn the attribution format and common obedience patterns; then GRPO, with reward being the attribution-correct term plus a format term. The authors state that this stage makes reasoning shorter and evidence-grounded, and covers ambiguous cases that SFT did not cover well.
   • Main hyperparameters: in Appendix B, the judge's GRPO learning rate is 1e-6, rollout group size 8, maximum response 256 tokens, KL coefficient 1e-3, temperature 1.0. SFT uses LoRA rank 32 and 1 epoch.

   CoT policy and decomposed reward

   • Cold start: the policy is SFT-trained on policy data to learn to identify directive segments, assign labels, and generate responses consistent with the labels. The authors state that SFT only provides structural scaffolding and cannot guarantee classification robustness under adversarial instructions.
   • Reward: instructions that should be obeyed add the judge's obedience score, instructions that should not be obeyed subtract λ times the obedience score, and α times the format reward is added. R(y)=∑_{ek:ak^∗=fol} J_ϕ(q,D,e_k,y)−λ∑_{ek:ak^∗≠fol} J_ϕ(q,D,e_k,y)+α S(y).
   • Optimization and coefficients: GRPO maximizes this reward, with a KL penalty applied to the SFT checkpoint. The authors state that the data contain benign-only, harmful-only, and mixed cases, so the policy cannot improve reward by always rejecting or always obeying. Qwen3-4B uses λ=1.0, α=0.5; Llama-3.1-8B uses λ=1.2, α=0.5. Policy GRPO uses group size 8, maximum response 768 tokens, temperature 1.0. On both backbones, the judge and the policy are both SFT followed by GRPO.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Table 1: for Qwen3-4B, EDIC has H-R 91.2% and B-F 89.5%; for Llama, H-R is lower than ReasAlign.

   Experimental setup

   • Models: the backbones tested are Qwen3-4B and Llama-3.1-8B-Instruct, each with its own trained judge and CoT policy. Meta-SecAlign appears only in the Llama comparison.
   • Data: four scenarios with 900 samples each after filtering, split 150/250/500. The necessity ablation is resampled to 3,600 samples. Email-only training uses 900 samples. The paper states that internal metrics are measured on held-out data but does not specify whether this is the 150-sample validation set per scenario.
   • External benchmarks: MT-Bench 80 questions; Arena-Hard-Auto 750 items; AgentDojo's banking, slack, travel, and workspace, totaling 629 attacked cases; CyberSecEval 2's prompt_injection with 250 items; InjecAgent with about 400 cases per setting; NotInject with 339 items, where benign prompts deliberately contain 1–3 injection trigger words.
   • Baselines: Base, SecAlign, StruQ, ReasAlign; Llama additionally has Meta-SecAlign. Section 5 also lists PromptGuard and Instruction Hierarchy, but Table 1 does not report either.
   • Metrics: internal metrics are B-F, H-R, CC, and the judge's accuracy, balanced accuracy, and macro-F1. Utility is Arena win rate and MT-Bench. Safety is DJ-U, DJ-ASR, CyberSecEval 2 ASR, and InjecAgent ASR. The paper does not give the formula for computing CC.
   • Judging and repetition: the judge model for MT-Bench, Arena-Hard-Auto, and CyberSecEval 2 is DeepSeek-Chat. The Arena protocol states that 95% confidence intervals are reported, but Table 1 does not give them. CyberSecEval 2 uses epochs=1. The paper does not report the number of repetitions of the main table, nor does it state whether the internal H-R/B-F uses a separate independent judging model.

   Main results

   According to Table 1. Higher utility is better, and lower ASR is better. StruQ's Arena and MT-Bench are not reported and are not included in the table.

   Swipe horizontally to read the table

   Model and method	H-R↑	B-F↑	Arena↑	DJ-U↑	Cyb-ASR↓
   Qwen3-4B Base	67.2	72.5	8.4	42.1	54.3
   Qwen3-4B ReasAlign	83.5	77.6	16.4	44.2	19.6
   Qwen3-4B EDIC	91.2	89.5	28.9	46.8	15.8
   Llama Base	74.9	85.5	1.6	6.6	28.6
   Llama Meta-SecAlign	91.9	68.1	4.6	6.2	7.4
   Llama ReasAlign	93.1	87.3	7.8	7.4	3.8
   Llama EDIC	92.8	91.7	12.4	15.8	2.5

   • Authors' interpretation: the authors state that non-reasoning defenses such as SecAlign and StruQ trade utility for safety; ReasAlign still treats all external instructions as threats. The authors consider that separately recording beneficial obedience and penalizing harmful obedience turns the two objectives into complementary signals.
   • Not leading in every cell: for Llama, H-R is highest for ReasAlign at 93.1, and EDIC is 92.8. DJ-ASR is lowest for Meta-SecAlign at 0.0, and EDIC and ReasAlign are both 0.1. On Qwen3-4B, EDIC and ReasAlign both have Inj-ASR 0.3. The authors still state that on Llama, EDIC is comparable to the strongest baseline on harmful rejection.
   • Gaps given by the authors and numbers outside the table: the abstract states harmful-rejection improvements of up to 7.7% and an Arena-Hard-Auto utility improvement of 12.5%; the introduction states 7.7 percentage points and 12.5 percentage points on Qwen3-4B relative to ReasAlign. The authors state that the Arena gap on Llama is +4.6%, and consider the gains larger on the weaker base. For Qwen3-4B, MT-Bench: EDIC 8.04, ReasAlign 7.28, Base 6.64; DJ-ASR is 0.4, 1.2, and 4.8, respectively. SecAlign's B-F is 56.9; StruQ's H-R/B-F is 76.5/53.3.

   Trigger words and data composition

   • Classification trigger: in Table 2, Δ (on minus off) on Qwen3-4B is H-R +3.8, B-F −0.9, Arena −1.2; for Llama it is H-R +3.1, B-F −2.7, Arena −0.5. The authors describe this as a small utility cost for a safety gain, and consider that the trigger activates classification capability already present from training, which can be switched on or off at inference without retraining.
   • Mixed context: in Table 3 (Qwen3-4B, with trigger), harmful-only is H-R 92.8, B-F 74.8; benign-only is 65.8 and 94.6; separate ctx has CC 0.85 and DJ-U 47.1; mixed has CC 0.92, and DJ-ASR is 0.4, the same as separate ctx. The authors consider that mixed training within the same document is what forces per-instruction attribution, and is therefore necessary for controlled obedience and not just raw safety.
   • Inconsistency between tables: Base CC is 0.72 in Table 3 and Base average CC is 0.67 in Table 5. The paper does not explain this.

   Over-refusal and scenario generalization

   • NotInject: benign accuracy in Table 6 for Qwen3-4B is Base 93.2, SecAlign 72.8, ReasAlign 78.6, EDIC 88.8; for the 3-trigger-word subset it is 89.4, 65.5, 72.2, 84.1. For Llama, EDIC is 87.4, Base is 91.4, SecAlign is 69.4, ReasAlign is 75.2. The authors consider that EDIC relies more on instruction intent than on lexical overlap with known attack phrases.
   • Four scenarios: Table 5 (Qwen3-4B, with trigger). Base's API is H-R/B-F/CC 58.6/67.2/0.63, the lowest H-R among the four scenarios; EDIC's H-R for Email, API, Medical, and Knowledge is 92.4, 91.6, 90.8, 90.0. The authors state that both axes rise together and the cross-scenario CC standard deviation decreases, but do not give that standard deviation.
   • Email-only training: Table 7. In-domain Email is H-R/B-F/CC 91.8/90.2/0.95; OOD average is 82.1/80.0/0.81; full EDIC is 91.2/89.5/0.92. The authors state that the largest drop is on API, with H-R −11.0 percentage points. The main text of Appendix D states in-domain CC=0.91 and full Email 0.92, which is inconsistent with 0.95 in Table 7 and Email EDIC CC 0.94 in Table 5.

   Other ablations and analyses

   • Necessity (Table 4, resampled 3,600): no filtering gives H-R/B-F/DJ-ASR 77.3/80.5/1.1 (100% retained); ≥4 gives 82.8/83.0/0.6 (86.9%); =5 gives 91.2/89.5/0.4 (71.6%). The authors consider that the necessity proxy captures causal coupling, not sample size.
   • SFT/RL ratio: the authors state that the judge optimum is 60/40 for Qwen3-4B and 80/20 for Llama-3.1-8B. The policy configuration shaded in Figure 4 is 20/80 (Qwen3-4B, with trigger). The figure legend reads Judge F1, Harmful Reject F1, Benign Follow F1, and the caption reads Acc/F1, H-Rej, B-Fol; the main text gives no curve readings.
   • Reward components: Figure 5 removes the judge, structure, beneficial obedience, or harmful rejection signal, respectively. The authors state the corresponding effects are the largest safety drop, rejection almost unbroken but utility collapse, over-refusal, and over-obedience; the main text gives no values for the bars.
   • Injection position: in Figure 6 (Qwen3-4B, SFT+RL, with trigger), H-R is 92.6 for prefix, 91.2 for the default in-text position, and 93.4 for suffix. The authors state that suffix and in-text differ by 2.2 percentage points, and consider the in-text position more realistic and more challenging.
   • Judge–policy agreement: Table 8 (Qwen3-4B, with trigger). Agreement goes from Base 62.4 to SFT 78.6 to SFT+RL 89.2; Wrong & Unsafe goes from 24.8 to 5.6; Cohen's κ goes from 0.38 to 0.81. The authors state that the proportion of judge-correct and policy-safe cases correlates with external safety metrics at 0.91.
   • Judge metrics not given: the paper states that it measures judge accuracy, balanced accuracy, and macro-F1, but the main text does not give specific values for these three. Appendix E uses an email example to illustrate that the judge performs local attribution, the CoT enforces beneficial constraints and rejects injection; the injected original text is not repeated.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors note that the four scenarios, fixed trigger words, binary labels, hand-tuned weights and 4B/8B scale still need to be extended.

   Limitations and future directions noted by the authors

   • Scenarios (Limitations): the training pool covers four scenario types; generalization to long-horizon agent trajectories, complex HTML payloads, or non-English documents has not been verified and may require additional scenario-anchored data.
   • Adaptive attacks (Limitations): the pipeline embeds a fixed catalog of injection trigger words from existing benchmarks. The authors consider that EDIC may underestimate adaptive attackers who specifically construct injections against the trained judge or the identify–classify–act template.
   • Binary labels (Limitations): the judge gives a binary obedience label to each instruction; an instruction that is partly safe and partly malicious is compressed into a single label. A calibrated judge with an abstention option is left for future work.
   • Weights (Limitations): λ and α in the decomposed reward are tuned per backbone. The authors state that they did not explore automatic weighting.
   • Scale (Limitations): experiments are at 4B/8B. The authors state that whether the same inductive bias holds at larger scale, or under multi-turn tool use, remains an open question.

   Scope of experimental coverage

   • The response backbones tested are Qwen3-4B and Llama-3.1-8B-Instruct. Table 1 reports Base, SecAlign, StruQ, ReasAlign, and Meta-SecAlign for Llama only.
   • Training uses four scenarios—email, API, medical, and knowledge—with 900 samples per scenario after filtering, split 150/250/500. Cold-start trajectories and the checking prompts in Appendix F are called from GPT-5.4 at temperature 1.0.
   • External evaluations include MT-Bench (80 questions), Arena-Hard-Auto (750 items), AgentDojo (629 attacked cases), CyberSecEval 2 prompt_injection (250 items), InjecAgent (about 400 cases per setting), and NotInject (339 items).
   • Section 5 lists PromptGuard and Instruction Hierarchy, but Table 1 does not report their numbers; it also does not report StruQ's Arena and MT-Bench, or the 95% confidence intervals stated in the Arena protocol.
   • The paper does not report the number of repetitions of the main results, does not report the agreement between the internal H-R/B-F and human annotation, does not give the CC formula, and does not give specific values for judge accuracy, balanced accuracy, and macro-F1. CyberSecEval 2 states epochs=1.

   Collapse this answer

6. 06Summarize the main content of the paper

   On Qwen3-4B, EDIC achieves H-R 91.2% and B-F 89.5%, while maintaining external utility.

   EDIC is a per-instruction obedience training method for external data, used to handle both prompt injection rejection and execution of legitimate external instructions.

   • Problem: documents in retrieval, tool, and multi-step workflows often place beneficial instructions together with indirect prompt injection. Rejecting the whole segment harms utility, and trusting the whole segment lets injection succeed.
   • Approach: construct benign-only, harmful-only, and mixed variants on real documents, and use necessity and independence for counterfactual filtering. A generative judge determines whether an output obeys a given instruction, and the decomposed reward then drives GRPO of the CoT policy. λ is 1.0 for Qwen3-4B and 1.2 for Llama-3.1-8B.
   • Qwen3-4B: in Table 1, EDIC has H-R 91.2%, B-F 89.5%, Arena-Hard-Auto 28.9, CyberSecEval 2 ASR 15.8, and AgentDojo ASR 0.4. The authors state that relative to ReasAlign, safety is 7.7 percentage points higher and Arena-Hard-Auto is 12.5 percentage points higher. Inj-ASR is 0.3, the same as ReasAlign.
   • Llama-3.1-8B-Instruct: EDIC has H-R 92.8%, B-F 91.7%, Arena 12.4, and AgentDojo utility 15.8. H-R is lower than ReasAlign's 93.1; B-F is higher than the other rows in the table. The authors state that the Arena gap relative to ReasAlign is +4.6%.
   • Ablation: on NotInject, EDIC's benign accuracy is higher than SecAlign and ReasAlign, but still lower than the undefended Base. The authors state that mixed data, necessity equal to 5, the judge's two-stage training, and each reward component are all load-bearing designs.
   • Authors' conclusion: the authors consider the core failure of existing defenses to be a classification failure, and hope to establish instruction classification as a foundational capability for LLMs handling untrusted external data. The authors state that EDIC achieves the best utility–safety trade-off on several benchmarks; in Table 1, Llama's H-R and DJ-ASR are not the lowest or highest in every column.

   Collapse this answer

Related papers