Qwen3Guard safety guardrail model technical report

1. 01What problem does this paper attempt to solve?

   Addresses two problems: binary labels of guardrail models are hard to adapt to different safety policies, and real-time detection of streaming generation is not possible.

   Core problem: existing guardrail models only output binary safety labels and must wait for the response to be fully generated before detection, making it difficult to adapt to diverse safety policies and incompatible with streaming LLM inference.

   • Application scenarios: LLM deployment requires real-time risk detection and filtering of user inputs and model outputs; the authors believe that outputting harmful, biased, or illegal content poses risks to users, enterprises, and society.
   • Shortcomings of existing methods (according to the authors): First, safety policies differ across guardrail models and datasets, binary labels are interpreted inconsistently, and they cannot adapt to different safety tolerance levels in different domains; second, existing open-source guardrail models can only evaluate complete responses, which does not match the streaming generation paradigm of modern LLMs and prevents timely intervention, increasing the risk that users are exposed to harmful partial outputs.
   • Core design assumptions: Introduce a third label, controversial, to accommodate samples whose safety depends on context or policy, thereby adapting to review needs of different strictness; use a token-level classification head to achieve real-time detection during generation.
   • What the paper proposes: The Qwen3Guard multilingual guardrail model series, including two variants, Generative Qwen3Guard (instruction-following generative classification) and Stream Qwen3Guard (token-level streaming detection), each with three sizes: 0.6B, 4B, and 8B, supporting 119 languages and dialects; and demonstrates two applications: using it as an RLAIF reward signal for safety RL, and integrating it into the CARE framework for real-time intervention.
   • Guardrail positioning (defense targets and assumptions):
     • Defense targets: harmful queries and jailbreak attempts in user prompts (the jailbreak label is used only for input classification), and harmful content in model responses.
     • Deployment location: as an external filtering mechanism connected in series on the LLM inference path; the Stream version can interrupt the conversation during generation.
     • Assumptions and costs: the paper does not evaluate robustness under adversarial bypass; the Stream version trades partial context for low latency, and the authors acknowledge a performance loss of about 2 F1 points.

   Collapse this answer

2. 02What related research exists?

   Related work covers four lines: instruction-following guardrails, reasoning-enhanced guardrails, token-level detection, and handling of policy inconsistency.

   Instruction-following guardrail models

   • LlamaGuard (Inan et al., 2023), ShieldGemma (Zeng et al., 2024), the Aegis series (Ghosh et al., 2025), etc. — use SFT to classify inputs and outputs into discrete safety categories; this is the mainstream paradigm directly compared in this paper.
   • WildGuard (Han et al., 2024) — a one-stop open-source moderation tool covering safety risks, jailbreaks, and refusals; this paper's English evaluation follows its setup.

   Reasoning-enhanced and multilingual/multimodal guardrails

   • DuoGuard (Deng et al., 2025) — uses reinforcement learning to strengthen safety enforcement; GuardReasoner (Liu et al., 2025) — explicitly models the reasoning process of safety judgment to improve classification accuracy.
   • X-Guard, OmniGuard, PolyGuard, MLLMGuard, etc. — extend safety moderation to multilingual and multimodal scenarios.

   Token-level detection and handling of policy inconsistency

   • Sharma et al. (2025), Xuan et al. (2025), Li et al. (2025a) — explore token-level classification; the paper notes that these methods usually approximate token-level labels with sentence-level annotations or rely on indirect learning, whereas Stream Qwen3Guard uses multiple rollouts plus Gen version predictions to estimate labels for incomplete sentences.
   • Zhang et al. (2024) — proposed the concept of the controversial label and annotated it with rule-based heuristics; this paper instead uses multi-model ensemble voting to obtain controversial labels. ShieldGemma (Zeng et al., 2024) mitigates policy inconsistency by dynamically adjusting classification thresholds.

   Baselines and benchmarks: The comparison baselines are LlamaGuard3-8B, LlamaGuard4-12B, WildGuard-7B, ShieldGemma-9B/27B, NemoGuard-8B, and PolyGuard-Qwen-7B; the evaluation benchmarks include ToxicChat, OpenAIModeration, Aegis, Aegis2.0, SimpleSafetyTests, HarmBench, WildGuardTest, SafeRLHF, Beavertails, XSTest, RTP-LX, PolyGuard, and the self-built Think and PolST.

   The authors position this paper as: adding three-level severity classification to the instruction-following paradigm to adapt to policy differences, and using more accurate token-level annotations to achieve true streaming detection, while covering 119 languages.

   Collapse this answer

3. 03How does the paper address this problem?

   The Gen version models safety classification as an instruction-following task and outputs three-level labels, while the Stream version adds a token-level classification head for streaming detection.

   Overall approach: SFT on the Qwen3 instruction model yields Generative Qwen3Guard, which outputs a three-level determination of safe/controversial/unsafe, the harm category, and a refusal determination; two token-level classification heads are attached in parallel to the backbone to yield Stream Qwen3Guard, which supports real-time intervention during generation.

   Safety policy and task definition

   • 9 safety categories: Violent, Non-violent Illegal Acts, Sexual Content, PII, Suicide & Self-Harm, Unethical Acts, Politically Sensitive Topics, Copyright Violation, Jailbreak (used only for input classification).
   • Three severity levels: Unsafe (harmful in most scenarios), Controversial (harm depends on context or policy), Safe.
   • The Gen version inputs the task definition, safety policy, category table, conversation context, and format requirements, and outputs the safety level and category; response classification additionally outputs a line Refusal: Yes/No (Figure 2; full templates in Appendix Figures 10 and 11).

   Data construction (Gen version)

   • Data scale: over 1.19M positive and negative samples, including human annotations and synthetic data; language distribution in Table 1, Chinese 26.64%, English 21.9%, and the rest Korean, Indonesian, Russian, Japanese, Arabic, etc.
   • Prompt synthesis: using the Self-Instruct framework, together with keyword-guided synthesis and positive-negative sample pairing (safe/unsafe pairs with similar surface structure), to prevent the model from treating surface cues such as verbs as labels.
   • Response collection: using base models (e.g., Qwen2.5-72B-Base) to generate unsafe responses; collecting chain-of-thought responses from QwQ, Qwen3, DeepSeek-R1, and their distilled versions.
   • Automatic annotation: voting by multiple models such as Qwen2.5-72B-Instruct and Qwen3-235B-A22B; the authors report that the safety-level labels achieve F1 above 0.9 on the human-verified set; multilingual samples are translated into 15 languages using Qwen-MT and validated through language-mixing detection, LLM review, and manual spot checks.

   Training: controversial label construction and distillation

   • The authors observed that the Safe/Unsafe ratio in the training set affects the model's strictness, so they split the data evenly into parts A and B, trained Strict (more Safe samples) and Loose (more Unsafe samples) models on each, and had them cross-annotate the other part; samples with conflicting predictions were labeled Controversial (Figure 3).
   • Label distillation: the data is split in two again, and Qwen3-32B is used as the teacher model for cross-refinement annotation to reduce annotation noise.

   Stream version architecture and token-level annotation

   • Architecture: take the backbone's last-layer hidden state h, and through two independent pathways predict the risk level and category of the query and response respectively: y_r-risk = Softmax(W_r-risk  LayerNorm(W_r-pre h)), and likewise for the query side (Equation 3).
   • Token-level annotation: for samples labeled unsafe/controversial, construct prefixes token by token and use multiple models to generate k rollouts; the Gen version determines whether the complete response is unsafe or controversial, and if the proportion exceeds threshold X%, it is marked; the authors report that in pilot experiments X% = 85% aligns well with human annotations (Equation 4). Then Qwen3-235B-A22B is used as an LLM reviewer to check whether the prefix itself is already unsafe (Equation 5); only if the two agree is that token and subsequent tokens marked with the original sample-level label.
   • Training objective: the query loss is computed only at the end token, and the response loss is averaged over each token; the category loss is included only when the true risk level is unsafe/controversial.
   • Inference debouncing: only when token i and the preceding token i−1 are both judged unsafe/controversial is the entire response from token i onward marked.

   Design of the two applications

   • Safety RL: using the judgments of Qwen3Guard-4B-Gen as rewards, training Qwen3-4B with GSPO. The Guard-Only reward gives 1.0 only when both the thinking and the output are judged Safe; the Hybrid reward adds the helpfulness score from WorldPM-Helpsteer2 and applies penalties to unsafe outputs and refusals respectively (Equations 1 and 2).
   • Real-time intervention: replacing the default generative checker of the CARE framework (detection–rollback–introspective intervention) with the Stream version, with a buffer of 40 tokens, at most 5 retries, and the Qwen3-4B base model.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Compares against 7 baselines on English, Chinese and multilingual benchmarks, and conducts ablations, safety RL, detection latency and CARE intervention experiments.

   Experimental Setup

   • Models evaluated: Qwen3Guard-Gen and Qwen3Guard-Stream at 0.6B/4B/8B each; baselines LlamaGuard3-8B, LlamaGuard4-12B, WildGuard-7B, ShieldGemma-9B/27B, NemoGuard-8B, PolyGuard-Qwen-7B.
   • Benchmarks: English prompt (ToxicChat, OpenAIModeration, Aegis, Aegis2.0, SimpleSafetyTests, HarmBench, WildGuardTest) and response (HarmBench, SafeRLHF, Beavertails, XSTest, Aegis2.0, WildGuardTest, self-built Think); Chinese consists of translated versions of ToxicChat, WildGuardTest, XSTest, Beavertails plus the self-built PolST (1,412 prompts, 4,038 responses); multilingual consists of RTP-LX and PolyGuard-Response.
   • Metrics and modes: F1 (some tables give Precision/Recall); Qwen3Guard has two modes—Strict treats controversial as unsafe, Loose treats it as safe; the average score in the tables is the best mode taken per benchmark (marked with * by the authors).
   • Evaluation method: The safety RL and CARE experiments use Qwen3-235B-A22B-Instruct-2507 as the LLM judge, with WildGuard scoring independently; the paper does not report agreement between the judge and human annotation.
   • Sample sizes: The safety RL training set is 13.7k (thinking)/6.7k (non-thinking) after filtering WildJailbreak, and the test set is 2,000 harmful + 210 benign prompts; the latency test uses 813 pure-response samples and 569 samples containing chain of thought. The paper does not report the number of repetitions.

   Main Results

   According to Table 2, 3, 5 (F1, Avg column is the best mode taken per benchmark):

   Swipe horizontally to read the table

   Model	English prompt Avg	English response Avg	RTP-LX multilingual prompt Avg
   LlamaGuard3-8B	79.4	70.7	46.6
   WildGuard-7B	85.8	79.9	43.9
   ShieldGemma-27B	70.0	65.9	61.4
   NemoGuard-8B	82.9	78.1	42.7
   PolyGuard-Qwen-7B	87.0	74.0	80.9
   Qwen3Guard-0.6B-Gen	88.1	82.0	74.8
   Qwen3Guard-4B-Gen	89.3	83.7	81.6
   Qwen3Guard-8B-Gen	90.0	83.9	85.0

   • The authors state that the Gen version achieves the best result on 8 of 14 English public benchmarks, and the 0.6B version is comparable to existing models more than 10 times larger; multilingual prompt classification outperforms prior models on 6 of 10 major languages, and response classification outperforms prior models on all 10.
   • Exceptions: On RTP-LX English, NemoGuard-8B's 95.4 is higher than Qwen3Guard-8B-Gen's 92.1 (Table 5); on PolyGuard-Response, 8B-Gen averages 77.6, slightly lower than 4B-Gen's 78.1 (Table 6).
   • On Chinese, Qwen3Guard-8B-Gen averages 85.1 on prompt and 87.1 on response, higher than all baselines (Table 4).

   Policy Inconsistency and Category, Refusal Detection

   • Policy inconsistency: Figure 4 gives precision/recall scatter plots of each guardrail model on Aegis, OpenAIMod, SafeRLHF, XSTest; the authors state that WildGuard-7B aligns well with Aegis but is overly strict on OpenAIMod, and the controversial label can be assigned to unsafe or safe depending on the benchmark's strictness.
   • Category classification: A self-built human-annotated category test set (unsafe prompts taken from WildGuard, responses from BeaverTail, supplemented with Aegis2.0); the confusion matrix in Figure 5 shows that for Qwen3Guard-4B-Gen the diagonal dominates for most categories, and the authors state that Copyright samples are scarce and relatively difficult to classify.
   • Refusal detection (Table 7): On XSTest, 4B-Gen F1 94.6, WildGuard-7B 93.3; on WildGuardTest, 4B-Gen 90.2, WildGuard-7B 88.6, which the authors state is comparable to WildGuard-7B.

   Safety RL Application (Table 10)

   • Guard-Only reward: Under Qwen3-4B Non-Think, the Qwen3-235B safety rate rises from 47.5 to 99.7, but the WildGuard refusal rate rises from 12.9 to 96.6, and the ArenaHard-v2 win rate drops from 9.5 to 8.5.
   • Hybrid reward: Under Non-Think, the WildGuard safety rate rises from 64.7 to 98.1, the refusal rate drops to 5.3, and the win rate is 10.7; under Think, the safety rate rises from 59.0 to 97.4, the refusal rate is 6.2, and the win rate rises from 13.7 to 16.6; Pass@1 on AIME25, LCB-v6, GPQA is close to the base model.
   • The authors state that the Hybrid reward improves safety while avoiding excessive refusal; Figure 6 gives the safety rate and refusal rate curves during training.

   Stream Version Performance, Latency and Intervention

   • Classification performance (Table 11–15): The Stream version averages about 2 F1 points lower than the Gen version, e.g., English prompt 4B version 89.1 vs. 89.3, 8B version 88.3 vs. 90.0; the authors state it still outperforms prior guardrail models.
   • Detection latency (Figure 8): In the pure-response setting, the authors report that the proportion exactly matching the human-annotated first sentence is close to 86.0%; in the chain-of-thought setting, about 66.8% of samples are detected within the first 128 tokens.
   • Efficiency (Figure 9): The authors state that the Stream version's processing time grows nearly linearly with response length, while the Gen version, resubmitted in 32-token chunks, incurs higher overhead as length increases (the vertical axis is relative time; no absolute time cost is given).
   • CARE intervention (Table 16): Under Non-Think, the Qwen3-235B safety rate rises from 47.5 to 85.7, the quality score rises from 50.0 to 66.4, and Wait Tokens is 70.1; under Think, the safety rate rises from 43.8 to 72.0, and Wait Tokens is 101.0.

   Other Ablations and Analysis

   • Controversial label ablation (Table 8, Qwen3Guard-Gen-4B): On most benchmarks the best strict/loose score exceeds the version without this label, e.g., on ToxicChat from 71.1 to 80.9 (loose), on OpenAIMod from 70.2 to 80.2 (loose).
   • Distillation ablation (Table 9): After distillation, prompt average +0.47 (strict)/+1.10 (loose), response average +0.50/+0.76.
   • The Stream version's category classification confusion matrix is in Appendix Figure 12; the authors state that it can distinguish categories in most unsafe scenarios.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors list four limitations: adversarial robustness, annotation bias, Stream version generalization, and regional cultural adaptation.

   Limitations and future directions noted by the authors

   • Vulnerability to adversarial attacks (Limitations): the authors believe Qwen3Guard may be bypassed by adversarial prompt engineering such as paraphrasing, obfuscation, and context manipulation, and performance may degrade under targeted attacks.
   • Moderation fairness and bias (Limitations): training annotations inevitably reflect the biases and cultural assumptions of the source datasets, and may produce disproportionate misjudgments of content from certain demographic, linguistic, or cultural groups.
   • Limited generalization of the Stream version (Limitations): the token-level classification head is constrained by latency and partial context, and false positives and false negatives may be higher in out-of-distribution or low-resource language scenarios.
   • Insufficient regional and cultural adaptation (Limitations): the current model contains no geography-aware or policy-adaptive mechanism and may be insufficient to cover local legal frameworks and social norms.
   • Future directions (Conclusion): the authors state they will continue to study enhancing the model's intrinsic safety through architectural and training innovations, as well as dynamic inference-time intervention strategies that can adapt to new risks.

   Scope of experimental coverage

   • The guardrail models tested are the two Qwen3Guard variants at three sizes each, compared against 7 baselines; language coverage is English, Chinese, and the languages included in RTP-LX and PolyGuard (the training data contains 17 languages, Table 1).
   • The safety RL application is done only on the single Qwen3-4B base model, with training data from the two types of harmful prompts in WildJailbreak; the CARE intervention experiment likewise uses only Qwen3-4B and Qwen3Guard-4B-Stream.
   • The latency evaluation uses sentence-level human annotation, with 813 response-only samples and 569 samples containing chain of thought; the paper does not report specific inter-annotator agreement values, only stating that samples with inconsistent annotations were filtered out.
   • The paper does not report the number of repetitions or variance for the classification experiments, does not report the absolute latency of the Stream version (Figure 9 gives only relative time), and does not conduct adversarial attack experiments.
   • To avoid metric hacking, the safety RL evaluation does not use Qwen3Guard self-evaluation but instead uses Qwen3-235B and WildGuard for scoring (Section 3.5.2).

   Collapse this answer

6. 06Summarize the main content of the paper

   Qwen3Guard is a series of three-class guardrail models supporting 119 languages, including generative and streaming variants.

   Qwen3Guard, released by the Qwen Team, is a set of multilingual safety guardrail models, targeting two pain points of existing guardrail models: binary labels are difficult to adapt to different safety policies, and real-time detection of streaming generation is not possible.

   • Problems to solve: Binary safe/unsafe labels are interpreted inconsistently under different safety policies; existing open-source guardrails need to wait for the complete response before evaluating, which is incompatible with streaming inference.
   • Method highlights: The Gen version models safety classification as an instruction-following task, outputting three levels safe/controversial/unsafe, harm categories and refusal determination; the controversial label is produced by voting between a strict and a loose model, then denoised via Qwen3-32B distillation. The Stream version adds two token-level classification heads, query/response, on the Qwen3 backbone; token-level annotations are determined jointly by rollout voting (threshold 85%) and LLM judging.
   • Main results: The authors state that the Gen version is best on 8 of 14 English public benchmarks; 8B-Gen averages F1 90.0 on English prompt and 83.9 on response (Table 2, 3), and 85.0 on RTP-LX multilingual prompt (Table 5); the Stream version averages only about 2 points lower.
   • Application validation: Using the Gen version's judgment as reward for safety RL, Qwen3-4B's WildGuard safety rate rises from 64.7 to 98.1 (Non-Think) and the refusal rate drops to 5.3 (Table 10); after integrating the Stream version with CARE, the safety rate rises from 47.5 to 85.7 (Table 16).
   • Authors' conclusion: The authors state that Qwen3Guard achieves SOTA on English, Chinese and multilingual benchmarks, and is open-sourced under Apache 2.0; they also acknowledge that it is an off-the-shelf moderation tool rather than a complete solution, and list four limitations: adversarial robustness, annotation bias, Stream version generalization, and regional cultural adaptation.

   Collapse this answer

Related papers