MPIB: A Benchmark for LLM Medical Prompt Injection Attacks and Clinical Safety

The researchers propose the medical prompt injection benchmark MPIB to evaluate the clinical safety of LLMs and RAG systems under direct prompt injection and RAG indirect injection. The benchmark measures high-risk clinical harm events using the clinical harm event rate CHER, and also reports ASR2 to distinguish moderate-or-higher from high-risk outcomes. MPIB contains 9,697 instances that underwent multi-stage quality screening and clinical safety lint. Evaluations across multiple baseline LLMs and defense configurations show that ASR2 and CHER3 can diverge significantly, and that attack rates vary depending on whether adversarial instructions appear in the user query or in the retrieved context. The dataset and evaluation code are released on Hugging Face and GitHub, respectively.