MedGuardBench: A Benchmark for Developing Patient-Facing Personalized Guardrails
MedGuardBench: A Patient-Oriented Benchmark for Personalized Guardrail Development
Researchers propose the medication safety guardrail task conditioned on patient profiles, predicting safety judgments and 17 risk categories from patient profiles and prescriptions, and release the MedGuardBench benchmark. The benchmark drugs do not overlap, comprising 5,001 scenarios and covering 87 drugs; the authors find that standard guardrails rely on surface cues such as wording and whether a profile is present rather than on patient facts. To address this, they build a personalized reasoning guardrail comprising personalized reasoning distillation, Context Unrolling (CU), and rule-based reward (guided GRPO). CU reduces false positives on most backbone models, and a control shows that directly providing the examination results yields no comparable benefit; guided GRPO improves unsafe recall and detection of complex risks such as allergies and adverse reactions on some backbones, but increases false positives on others. Model accuracy exceeds baseline models and a matched no-profile control, and profile removal and swapping experiments confirm that the gains come from patient information rather than fine-tuning alone.