Paper evaluates prompt injection detector calibration and delayed decisions

1. 04:34
   arXiv: Jailbreaks, Prompt Injection, Poisoning & Defenses

   From Detection to Decision: Calibration and Delayed Decision for Prompt Injection Detection

   An arXiv paper compares four types of prompt injection detectors on 47,611 held-out samples in terms of detection, probability calibration, and allow-intercept-delay policies, arguing that probability outputs alone are insufficient to support reliable safety decisions under distribution shift. The evaluation covers direct and indirect prompt injection, difficult benign inputs, obfuscated samples, and distribution shift; the LLM judge achieves the highest AUROC at 0.897, followed by Jev at 0.871, Llama Prompt Guard 2 86M at 0.757, and 22M at 0.714. The paper finds that Jev's native probabilities are not stably better than post-hoc calibrated classifier scores, and temperature scaling improves calibration but not discrimination. At a 1% empirical false negative rate target, all evaluated gated review systems require an escalation rate above 91.8%, with substantial false positive rates. The authors argue that effective LLM safety systems require joint evaluation of detection, calibration, reviewer behavior, and operational decision policies.

Current attention 37·Peak in the comparable period 37 (Nov 6 03:00)·Change over the comparable past 24 hours 0%