AlignSentinel: alignment-aware prompt injection attack detection

AlignSentinel is an alignment-aware prompt injection attack detection method that uses LLM attention map features to classify inputs into three categories: misaligned instructions, aligned instructions, and no instructions. Addressing the flaw in existing defenses that classify any input containing instructions as malicious, the authors constructed the first systematic benchmark covering the three input categories. On both their own and existing benchmarks, AlignSentinel can accurately detect misaligned instruction inputs, greatly surpassing baseline methods.