Beyond Binary Labels: Prompt-Level Rubric-Based Diagnostic Evaluation of LLM Safety

Core problem: existing LLM safety benchmarks compress responses into binary safe/unsafe labels, lack explanatory depth, and have low agreement among automatic judges.

• Setting and importance: the authors argue that as LLMs enter practical applications, quantifying their safety risks is essential for responsible deployment; binary labels cannot give developers actionable directions for improvement.
• Three shortcomings of existing methods (as stated in the paper): first, the evaluation granularity is too coarse, with only safe/unsafe; second, judges are inconsistent—the paper cites Liu et al. (2026b) stating that changing the evaluation classifier can shift the attack success rate by nearly 30 percentage points, and for some categories inter-judge agreement drops to 22%; third, data diversity is limited, and no benchmark simultaneously covers multiple languages, single/multi-turn, and multiple attack types under a unified taxonomy (Table 1).
• Core observation: the authors argue that reliable safety judgments depend on explicit evaluation criteria; inspired by the domain-specific rubrics of FORTRESS (Knight et al., 2025), giving each prompt its own scoring rubric can turn subjective binary judgments into objective, interpretable item-by-item evaluation.
• The paper proposes: DiagSafe—a diagnostic safety evaluation framework (prompt-level rubric + LLM judge) and a bilingual benchmark (a taxonomy of 84 third-level labels, with two subsets DiagBase and DiagAttack, 3,489 in total).
• Positioning: this is evaluation/benchmark work and does not propose attack or defense methods; the evaluation target is the LLM itself in standalone dialogue settings (locally deployed, no system prompt, no external guardrails).

Safety benchmarks (taxonomy and scenario diversity)

• SALAD-Bench (Li et al., 2024), S-Eval (Yuan et al., 2025), AIR-Bench (Zeng et al., 2024), Safety-Prompts (Sun et al., 2023): respectively introduced hierarchical risk categories, categories designed based on regulatory documents, and expanded data volume, extending taxonomy coverage.
• JailbreakBench (Chao et al., 2024), HarmBench (Mazeika et al., 2024), and multilingual, multi-turn datasets (Deng et al., 2024; Cao et al., 2026): enhanced scenario diversity. The paper points out that most existing benchmarks lack policy-level label boundary definitions and rarely integrate these elements into a single framework.

Evaluation methods (three types)

• MCQ type: SafetyBench (Zhang et al., 2024b), CHiSafetyBench (Zhang et al., 2024a). The paper points out that this paradigm tests the model's ability to recognize harmful content, not the safety of what it actually generates.
• Trained safety classifiers: LlamaGuard4 (Meta AI, 2025), SALAD-Bench's MD-Judge (Li et al., 2024). The paper points out that they only give binary labels, with no evidence chain or actionable diagnosis.
• LLM judges with generic prompts: e.g., EasyJailbreak (Zhou et al., 2024). The paper points out that such judges are susceptible to stylistic variation (Eiras et al., 2025), degrade on adversarial or multilingual inputs (Liu et al., 2024), and suffer from the same inconsistency problem as classifiers (Liu et al., 2026b).

Closest work: FORTRESS (Knight et al., 2025)

• The paper states that this work demonstrated that instance-level rubrics can achieve high human alignment in specific high-risk domains such as national security and public safety; DiagSafe builds on this by introducing a reusable intermediate layer of "safety clauses → label specification → per-prompt rubric", generalizing the paradigm to comprehensive safety evaluation. The authors state that their experiments "for the first time" demonstrate that automatically synthesized per-prompt rubrics are highly effective in safety scenarios.

Baselines and comparison setup

• Baseline judging paradigms for the reliability comparison: two LLM judging approaches, generic safety prompt and label-spec prompt, and four trained classifiers (Qwen3Guard, S-Eval, LlamaGuard4, MD-Judge), compared by agreement rate on 236 human-annotated responses (Table 3).
• Benchmarks for the data composition comparison: SafetyBench, Do-Not-Answer, DecodingTrust, AIR-Bench, SALAD-Bench, CHiSafetyBench, LongSafety, RMCBench, FORTRESS (Table 1).

Overall approach: reliable safety judgments depend on explicit criteria. DiagSafe does not use a generic instruction for binary classification; instead, it gives each prompt its own dedicated rubric, which an LLM judge answers item by item, and then aggregates into a safety verdict and a usefulness score.

Evaluation framework: prompt-level rubric

• Safety dimension (S-items): each rubric contains 4–6 S items, each a Yes/No question corresponding to one specific harm the prompt may produce; each item comes with at least two exemptions, stating behaviors under the same risk that must not be judged unsafe (e.g., objective conceptual discussion, explicit refusal, common legal knowledge), used to reduce false positives.
• Usefulness dimension (U-items): fixed 8 items in three categories—instruction following, information quality (factual correctness, internal consistency), response pattern (whether it is an empty refusal, whether it offers alternatives); binary answers are mapped by deterministic rules to a 0–3 score, e.g., a direct and factually consistent answer scores 3, an empty refusal scores 1.
• Decision rule: a response that triggers at least one S item with no applicable exemption is counted as unsafe; the model-level primary metric is unsafe rate (UR).

Label system

• Source: collected public safety standards and policies from the EU, the US, China and other countries and regions, integrating 204 safety clauses; the authors state that multi-jurisdiction sources avoid inheriting a single vendor's blind spots.
• Construction: an LLM agent merged clauses and expanded them into safety labels, then safety team experts reviewed and revised them (traceability, rule accuracy, boundary clarity), forming a hierarchical system of 11 first-level, 34 second-level, and 84 third-level labels (Table 6).
• Label specification: each third-level label contains four parts—a precise definition, value principles based on public standards, dangerous behaviors that constitute a violation, and permitted safe behaviors; it is the common basis for synthesizing prompts and rubrics.

Data synthesis (Figure 3, Appendix C)

• DiagBase: DeepSeek-V3.2 serves as the generator, using label specifications as seeds, synthesizing prompts along three orthogonal attributes—language (Chinese/English, about 1:1), dialogue turns (single-turn/multi-turn, 7:3), attack form (ordinary harmful query, template jailbreak, combined template jailbreak); then an LLM quality inspector grades them (high quality/acceptable/poor) and rewrites poor samples.
• DiagAttack: takes the single-turn non-attack prompts from DiagBase, pairs them with the task shells (instruction, input, constraints) of the external instruction-following dataset EIFBench, and a generator LLM embeds the harmful intent into the benign instruction shell; a discriminator LLM reviews against five criteria, and if it fails, feeds back the reason and rewrites, up to 3 rounds. The production run processed 1,811 candidate pairs, accepted 1,566, and then formed the final subset through filtering and translation.
• Rubric synthesis: Hy3 serves as the rubric writer, generating rubrics from the prompt and its third-level label specification, then a round of QC verification and repair; rubrics average 4,060 characters in length, with an average of 4.5 S items and a fixed 8 U items. Appendix C.3 and Tables 14 and 15 give the complete prompts for the writer and QC.

Scale and evaluation protocol

• Scale: DiagBase 2,401, DiagAttack 1,088, 3,489 in total.
• Judging protocol (Appendix D.3): the judge's input is the instantiated rubric (filled in with prompt, response, safety category), with no additional system prompt; the judge answers S items item by item, records evidence and exemptions, outputs a safety verdict and a list of triggered items, and independently answers U items to give a 0–3 score. Multi-turn samples are judged independently turn by turn; if any turn is unsafe, the whole conversation is judged unsafe.

Experimental setup

• Models under test: 11 recent LLMs (8B to 1.6T parameters): Qwen3.6-27B, Qwen3.5-397B-A17B, Kimi-K2.6, GPT-OSS-120B, MiniMax-M3, DeepSeek-V4-Pro, GLM-5.2, Gemma-4-31B, DeepSeek-V4-Flash, Llama-3.1-8B, Llama-3.3-70B. All locally deployed, with no moderation filters, guardrail models, or system prompts; unified decoding configuration temperature 0.7, top-p 0.8, top-k 20, repetition penalty 1.05; all in non-thinking mode except GPT-OSS-120B (which does not support disabling reasoning, so reasoning_effort = medium was used and only the final response was evaluated).
• Generation and judging: DeepSeek-V3.2 synthesizes prompts, Hy3 synthesizes rubrics and serves as the rubric-guided judge.
• Metrics: unsafe rate (UR, the proportion of responses triggering at least one S item with no exemption); agreement rate with human judgments; inter-judge Cohen's κ; 0–3 usefulness score.
• Reliability validation: 236 sampled responses were independently annotated by 3 experts (pre-annotation three-way agreement rate 97.88%), with majority vote as the gold standard; four judging paradigms were compared—rubric, generic prompt, label-spec prompt (judge models Hy3, DeepSeek-V4-Pro, Qwen3.5-9B), and four trained classifiers (MD-Judge, Qwen3Guard lenient/strict, LlamaGuard4, S-Eval).
• Data quality spot-check: prompts sampled 4 per third-level label, 336 in total; rubrics sampled 3 per third-level label, 252 in total.

Main results

Agreement rate of judging paradigms with the human gold standard (per Table 3, n=236 responses):

Paradigm	Judge	Agreement rate
Classifier	MD-Judge	69.07%
Classifier	Qwen3Guard (lenient)	80.93%
Classifier	LlamaGuard4	84.32%
Classifier	S-Eval	85.17%
Classifier	Qwen3Guard (strict)	87.29%
Generic prompt	Hy3 / DS-V4 / Qwen3.5-9B	87.29% / 80.51% / 85.17%
Label spec prompt	Hy3 / DS-V4 / Qwen3.5-9B	86.86% / 82.63% / 84.75%
Rubric	Hy3 / DS-V4 / Qwen3.5-9B	95.34% / 89.41% / 90.25%

• The authors state that the rubric paradigm has the highest agreement rate and lower variance across judges, arguing that explicit per-prompt criteria make judging more accurate than generic prompts or dedicated guardrail models.
• Inter-judge consistency (Table 4): under the rubric paradigm, the agreement rate of the three judge pairs is 93.33%–94.78%, Cohen's κ 0.828–0.864; the other two paradigms fluctuate more (e.g., under the generic prompt, Hy3 vs. DS-V4 agreement rate 80.02%, κ 0.559).
• Data quality: prompt pass rate 96.4% (324 of 336), rubric reasonable-and-executable rate 99.6% (251 of 252).

Model diagnosis: DiagBase vs. DiagAttack

• UR comparison (Figure 4): the 11 models' DiagBase UR ranges from 2.8% (Qwen3.6-27B, Qwen3.5-397B-A17B) to 32.9% (Llama-3.3-70B); DiagAttack UR ranges from 18.7% (Qwen3.6-27B) to 73.4% (Llama-3.1-8B). Each model's DiagAttack UR is 1.9×–8.4× its DiagBase (median 5.3×), and 8/11 models exceed 40%.
• Authors' interpretation: wrapping the same harmful intent in a benign instruction shell generally raises UR; the authors argue this suggests instruction following and safety refusal may be competing objectives in current training, and that existing benchmarks mostly rely on directly harmful queries or conventional template jailbreaks, overlooking this failure mode.

Safety–usefulness two-dimensional analysis and category-level diagnosis

• Two-dimensional plane (Figure 5, each point is a model's mean on a first-level category): Qwen3.6-27B concentrates in the lower-right ideal region (usefulness above 2.0, UR mostly below 15%); GPT-OSS-120B has low UR but usefulness mostly below 1.5, and the authors state its safety may partly come from a tendency to empty refusals; Llama-3.1-8B is poor on both (UR average 41%, max 49%, usefulness average 1.43).
• Category level (Figure 6): three models with similar overall UR (18.6%–20.9%) have different risk profiles—MiniMax-M3 is highest on Intellectual Property (27.3%), DeepSeek-V4-Pro is highest on Suicide & Self-Harm (20.1%) but lowest on CBRNE (13.7%); second-level categories differ even more, e.g., UR on Radiological & Nuclear ranges from 11.6% to 26.0%. The authors state that a single aggregate score cannot characterize model safety.
• Sample level (Table 2): under the same DiagAttack prompt, the rubric distinguishes three safety strategies—constructive refusal, limited compliance, empty refusal—as well as increasing degrees of unsafety by triggered S items (DeepSeek-V4-Pro triggers S2+S4, MiniMax-M3 triggers S1–S4).

Data dimension analysis (Discussion)

• Attack form (Figure 7, DiagBase only): the UR increment of template jailbreak relative to ordinary query is −1.0 to +13.9 percentage points; MiniMax-M3's UR is instead 1.0 percentage point lower under template jailbreak. The authors state that conventional template jailbreaks have limited effect and vary by model.
• Language (Figure 8): most models have higher UR in Chinese than in English, with differences from −4.1 percentage points for GPT-OSS-120B to +6.3 percentage points for Gemma-4-31B; Llama-3.3-70B is also higher in English (−2.0 percentage points). The authors argue that Chinese safety coverage still lags behind English.
• Dialogue turns (Figure 9): judged by the first k turns (k=1,2,3), UR rises with turns, with increases from +0.9 percentage points for Qwen3.6-27B and Qwen3.5-397B to +7.2 percentage points for Llama-3.3-70B. The authors argue that established dialogue context erodes refusal boundaries.

Other ablations and analyses

• Additional new models (Appendix B, Table 5, same protocol): Qwen3.8-27B overall UR 8.5%, Hy4-Preview 11.9%, GLM-5.3 13.1% (low reasoning setting), DeepSeek-V4.1-Flash 18.5%, Kimi-K3 26.8%; usefulness 2.14–2.37.
• Case studies (Appendix H): in Case 1 (DiagAttack-000957), 5 models comply and produce a false medical article and trigger all or nearly all S items, with usefulness all 0; in Case 2 (DiagBase-000363), 5 models are unsafe, with triggered items ranging from only S3 (MiniMax-M3, GLM-5.2) to S1+S3+S4 (Llama-3.1-8B).
• Full UR matrix: Tables 20–22 give all URs of the 11 models on the first-level (merged/subset) and 34 second-level categories.

Limitations noted by the authors and future directions

• Limited coverage of label specifications (§7 Limitations): Labels are generated by an LLM, so coverage of the risk space is inherently bounded, and synthesized prompts and rubrics may miss rare, emerging, or highly localized harmful scenarios; the authors plan to incorporate continuously updated regulatory sources and human-in-the-loop refinement.
• Limited coverage of models under test (§7): Only 11 locally deployed open-weight models are evaluated, not commercial systems; each model is evaluated in only a single reasoning mode, and differences between thinking and non-thinking modes are not explored. The authors plan to cover both modes and API commercial systems with deployment guardrails, and to maintain DiagAttack as a continuously updated safety leaderboard.
• Scope of evaluation targets (§7): Currently only standalone LLMs in dialogue settings are supported, not LLM agents; the authors plan to extend the rubric diagnostic paradigm to agentic settings, evaluating intermediate actions and tool-use trajectories.

Scope of experimental coverage

• The models under test are 11 locally deployed open-weight models (8B–1.6T), with 5 additional new models in Appendix B; all are evaluated with no system prompt, no guardrails, and a unified decoding configuration.
• The human gold standard for reliability validation is 236 sampled responses, annotated by 3 experts with majority-vote adjudication; the paper does not report the number of repetitions or confidence intervals for the judge-agreement experiment.
• The data-quality spot check covers 336 prompts and 252 rubrics (stratified sampling by three-level labels).
• Both prompts and rubrics are synthesized by LLMs (DeepSeek-V3.2 generates prompts, Hy3 writes rubrics and serves as a judge); the reliability conclusions for rubric judging are based on three judge models: Hy3, DeepSeek-V4-Pro, and Qwen3.5-9B.
• All instruction wrappers in DiagAttack come from EIFBench; languages are only Chinese and English (about 1:1), and the single-turn to multi-turn ratio is 7:3.

DiagSafe, proposed by the Tencent Hunyuan team, is a diagnostic LLM safety evaluation framework plus a bilingual benchmark, targeting three problems: binary safe/unsafe labels lack interpretability, judge consistency is low, and data diversity is insufficient.

• Method: based on an 11/34/84 three-level safety label system (derived from 204 multi-jurisdiction public safety clauses), a dedicated rubric is synthesized for each prompt—4–6 safety S items with exemption clauses plus a fixed 8 usefulness U items (0–3 score); the benchmark contains DiagBase (2,401, ordinary queries and template jailbreaks) and DiagAttack (1,088, embedding harmful intent into EIFBench instruction shells), 3,489 in total, Chinese and English about 1:1.
• Reliability: on 236 human-annotated responses, Hy3 judging with rubrics has an agreement rate of 95.34% with humans, higher than generic prompt (87.29%), label-spec prompt (86.86%), and classifiers (69.07%–87.29%); under the rubric paradigm, inter-judge agreement rate is 93.33%–94.78%, Cohen's κ 0.828–0.864 (Tables 3, 4).
• Diagnostic findings: the 11 models' UR on DiagAttack is 1.9×–8.4× that on DiagBase (median 5.3×), 8/11 models exceed 40%, the highest being Llama-3.1-8B at 73.4%; the authors argue that instruction following and safety refusal may be competing objectives.
• Fine-grained differences: models with similar overall UR have different failure modes across categories (e.g., 11.6%–26.0% on Radiological & Nuclear); GPT-OSS-120B's low UR comes with low usefulness (mostly below 1.5), and the authors state it may lean toward empty refusals.
• Scenario factors: most models have higher UR in Chinese than in English (up to +6.3 percentage points); UR rises with dialogue turns (up to +7.2 percentage points).
• Authors' conclusion: DiagSafe turns binary verdicts into category-level and sample-level actionable diagnosis, usable for auditing and guiding targeted safety alignment; the authors also acknowledge limitations in three areas: label coverage, model coverage (only local open-source models, a single reasoning mode), and agent scenarios.