NVIDIA proposes NXDirection: making external content non-executable along the instruction direction
NXDirection: No-eXecute Along the Instruction Direction
NVIDIA research team proposes NXDirection, a method to defend against indirect prompt injection by removing instruction direction from external content. The method uses 175 short samples and 18 instruction types to extract the instruction direction for each token via mean difference, intervening only on tokens from external sources, leaving system prompts and user requests unchanged. On 9 open-source instruction models, BIPIA attack success rate drops from 38.1% to 0.1%, AgentDojo with tool calling drops from 28.4% to 3.7%, and normal task utility is 76.4%, close to the undefended 78.4%. The authors claim it outperforms prompt defenses like Sandwich and Spotlighting on two benchmarks, and unlike detectors, it does not require deleting entire tool outputs or extra GPT calls; however, the method requires access to model internal states, is only applicable to open-weight or self-hosted models, and some attacks still succeed.
NVIDIA says NXDirection reduces attack success rates to 0.1% and 3.7% on two benchmarks, with almost no loss of normal task utility, and is only applicable to open-weight or self-hosted models.
- Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
- CaMeL system defending against prompt injection by design
- Refusal behavior in language models is mediated by a single direction
- Large language models encode harmfulness and refusal separately
- Assistant axis: locating and stabilizing the default persona of language models