Jev as an LLM safety guardrail: an empirical study of input filtering, selective output moderation, and over-refusal
Jev as an LLM Safety Guardrail: An Empirical Study of Input Filtering, Selective Output Review, and Over-Refusal
AI summary
An empirical study uses the probabilistic decision model Jev as a safety guardrail for a Qwen2.5-1.5B pipeline; input filtering reduced the response rate to 720 harmful prompts from 73.2% to 2.4%, but the refusal rate on 250 XSTest safe prompts rose from 1.2% to 10.8%. Under tiered filtering plus selective output moderation, the harmful response rate was 5.6% and the benign refusal rate was 4.0%; joint threshold optimization raised macro-F1 from 0.9351 to 0.9512. However, compared with a retuned single gate, its additional gain was not stable on the held-out set.