Ignore all instructions despite contrary conventions: Measuring prompt injection guardrails against attacks on legal contract review

The researchers measured eleven open-weight prompt injection guardrails on two legal contract corpora and found that existing guardrails perform poorly: document-level false positive rates reached as high as 92.39%, two guardrails had paired AUC below random, and a baseline consisting of five regexes was not Pareto-dominated by any guardrail. Controlled probe experiments localized model-specific and universal failure modes. A prompt-guided LLM can resolve these tested failure modes, including the second category of threat (unauthorized scope claims) on which all guardrails scored zero. The paper's proposed cascaded LLM scheme uses a mid-tier model for initial screening and a frontier model to review positive results, and is 6.5 times cheaper than using the frontier model alone with comparable performance.