Bypassing prompt injection detectors through evasive injection
Bypassing Prompt Injection Detectors through Evasive Injections
Quick read
Based on 论文全文 （AI 生成）; refer to the original
The researchers propose a multi-probe evasion attack that appends a GCG-optimized universal adversarial suffix to poisoned inputs, fooling activation drift detectors at all layers simultaneously, with attack success rates of 93.91% on Phi-3 3.8B and 99.63% on Llama-3 8B. The method builds on the task drift detection framework of Abdelnabi et al., adapting the GCG algorithm originally used for jailbreaks to jointly optimize the suffix against the losses of per-layer linear probes, causing activations to be misclassified as clean inputs. The authors also propose a random-suffix augmentation defense, generating activations with multiple adversarial suffixes and retraining the linear probes, achieving detection accuracy of 80% to 100% on unseen suffixes, and direct attacks against the hardened probes also struggle to succeed.
- 基于激活值漂移的线性探针（linear probes）被用来检测 RAG 系统中的 prompt injection，但这类检测器在自适应攻击者面前的鲁棒性尚未被研究。攻击者只要控制部分检索数据，就可能既注入恶意指令又躲过所有层级的漂移检测。
- 作者提出一种 multi-probe evasion attack：在含注入指令的输入后附加一段对抗后缀，并用改造的 GCG 算法，把各层探针损失的梯度注入 LLM 反向传播，联合优化出一个通用后缀，使其同时骗过所有层探针。防御方面提出对抗后缀增强：生成多个后缀、前向传播时随机附加其中一个，再用得到的激活重训探针。
- 在 Phi-3 3.8B 和 Llama-3 8B 上，单个通用后缀让全部五个探针同时误判的攻击成功率为 93.91% 和 99.63%，多数投票下接近 100%；攻击后深层探针误判率从不足 1% 升至 95% 以上。后缀训练后的探针在未见过的对抗后缀上达到 80–100% 的检测准确率，且比 PGD 对抗训练更可靠；直接针对加固探针优化后缀也很难成功。
- 防御实验仅在 Phi-3 上评估；对抗后缀对模型行为的影响因模型而异（Llama 受影响轻微，Phi 受影响明显），对后缀如何改变模型输出只做了初步分析。作者指出需要更强的防御，并强调应在自适应威胁模型下评估检测机制，而非依赖通用鲁棒性技术。
The paper gives the success rate of a universal suffix attack against activation drift detectors, and verifies that suffix-enhanced training can restore the detection rate to above 80%.