SADDER: Safety-aware denoising and directional embedding correction for text diffusion models

1. 01What problem does this paper attempt to solve?

   Text diffusion toxic continuation and jailbreaks lack directly transferable inference-time guidance; the authors extend SAD with embedding gating.

   Text diffusion models lack directly applicable inference-time safety mechanisms, and SAD only guides in token probability space.

   • Scenario: The authors state that autoregressive models still dominate language modeling, while text diffusion models (TDMs) can update multiple positions in parallel during denoising. However, safety research on TDMs for toxic generation and jailbreaks is relatively limited.
   • Existing gaps: The authors state that fine-tuning and alignment, post-hoc filtering, and autoregressive decoding-time interventions either require additional training or assume autoregressive decoding, and cannot be directly applied to TDMs. SAD steers in probability space using about 500–1000 unsafe references, but whether richer representations can be used for position-level guidance remains an open question.
   • Observation: The authors propose first extracting safety signals for each position in embedding or activation space, then modulating SAD updates based on that signal; positions with high unsafe signals are more likely to be modified. This multiplication does not automatically preserve the conditional mean of the original target distribution.
   • Method: The authors propose SADDER, deriving a modified denoiser that exactly corresponds to the condition of a position-level target distribution, and evaluate unsafe continuation and jailbreaks on MDLM and USDM.
   • Threat model:
     • Risk: Toxic continuation on unsafe prompts, and DIJA, PAD jailbreaks against instruction-tuned diffusion models.
     • Defender information: Pretrained denoiser outputs, layer hidden states, unsafe references. The paper does not use white-box or black-box descriptions of attackers.
     • Intervention: Training-free; modifies the clean token distribution at selected positions on specified denoising steps. In MDLM, already revealed positions are copied per the transition kernel; in USDM, any step can change tokens.
     • Evaluation: Llama-Guard unsafe rate, HarmBench ASR; quality via PPL and BERTScore, cost via generator NFE.

   Collapse this answer

2. 02What related research exists?

   Related work includes activation steering, remasking, sampling guidance, and diffusion jailbreaks; baselines include SAD, CSD, BoN, FK, and DiffuGuard.

   Inference-Time Safety and TDM Guidance

   • Traditional mechanisms: Dai et al. (2024) do fine-tuning and alignment, Naveed et al. (2025) do post-hoc filtering and rejection sampling, Xu et al. (2024) modify decoding at next-token prediction with SafeDecoding. The authors claim these methods either require additional training or assume autoregressive decoding, and cannot be directly applied to TDMs.
   • Inference-time methods on TDMs: Singhal et al. (2025) do particle guidance, Schiff et al. (2025) do classifier guidance, Lee and Han (2026) do safety-oriented remasking. SAD (Yusuf et al., 2026) is training-free, uses about 500–1000 unsafe references, and pushes trajectories toward safe regions in token probability space.
   • Safe Denoiser: Kim et al. (2025) give an identity for safe denoisers with data and unsafe denoisers, extended to flow matching in 2026. The authors claim SAD approximates the weight with hyperparameter η, and the implementation lacks the safety guarantees promised by the theorem.

   Activation Steering and Remasking

   • Contrastive directions: Rimsky et al. (2024) use Contrastive Activation Addition to extract directions from paired samples; Avrahami and Nachmani (2026) use ILRR to align masked diffusion activations to reference sequences.
   • DiffuGuard: Li et al. (2025) combine random remasking, block-level auditing, and repair. This evaluation distinguishes full repair, which can modify fixed attack text, and gen repair, which only modifies initial mask positions.
   • CSD: Lee and Han (2026) use harmful responses from WildJailbreak and rewritten refusals to estimate Contrastive Safety Direction: earlier denoising steps modify hidden states, then select positions by threshold for remasking and regeneration. The authors claim SADDER uses a similar contrastive direction, but the score is centered at the midpoint of harmful and safe representations, and the gating applies SAD corrections in probability space rather than directly modifying hidden states.

   Sampling Control and Diffusion Jailbreaks

   • Best-of-N and FK: Best-of-N selects the highest reward among independent candidates; FK steering (Singhal et al., 2025) maintains multiple trajectories and resamples at intermediate steps based on reward-derived potential. The authors claim neither retrains the generator; SADDER modifies token predictions within a single trajectory, comparing safety–quality trade-offs under a given inference budget.
   • DIJA and PAD: Wen et al. (2026) use interleaved text–mask prompts to exploit bidirectional context and parallel decoding to elicit harmful completions. Zhang et al. (2025) rewrite affirmative response starters into multi-point attention attacks for PAD. The authors claim guidance must therefore be evaluated on adversarial structured inputs, not just ordinary unsafe continuation.

   Baselines and Benchmarks

   • Baselines: unmodified sampling, full SAD, adapted CSD†, BoN, and FK; jailbreak experiments also include DiffuGuard (full and gen). USDM continuation baselines exclude FK.
   • Data: Continuation uses RealToxicityPrompts and BeaverTails; jailbreak uses DIJA and PAD on HarmBench behaviors; over-refusal uses XSTest. JailbreakBench and AdvBench are used for development splits, WildJailbreak is used to construct jailbreak references.

   The authors position SADDER as using midpoint-centered activation scores to select where SAD is applied. The abstract claims its balance of safety, quality, and inference cost is superior to existing methods; this overarching claim is inconsistent with some tables, see Question 4.

   Collapse this answer

3. 03How does the paper address this problem?

   SADDER uses harmful–safe midpoint scores for gating, applying SAD updates only at selected positions.

   SADDER has two stages: use midpoint scores in embedding space to select risk positions, then apply SAD's probability correction only at those positions. The core method does not re-mask; warm-up and re-masking are optional extensions (Algorithm 2).

   Two types of diffusion and SAD

   • MDLM: The forward process keeps a token with probability αt, otherwise it becomes a mask, and once masked it stays masked. In the reverse process, already revealed positions are copied, and mask positions are revealed according to the network's prediction. The samplers of LLaDA and Dream differ from this transition only in how they select positions to reveal. A re-masking sampler can also return revealed positions to mask.
   • USDM: No mask. A token is kept with αt, otherwise it is replaced with a uniform sample from the vocabulary. Every term of the reverse kernel can be positive, so any token can be changed at any step. The authors cite Sahoo et al. (2026), stating that this brings self-correction and better guided generation.
   • SAD: The data denoiser is the pretrained output. The unsafe denoiser uses Monte Carlo weighting with the forward kernel on a negative set. The weight β*(xt) is hard to compute; the implementation approximates it with ηβ(xt) and keeps η nonzero only within a critical time window C at the beginning of the reverse process.

   Stage one: position gating

   For each reference, take the prompt and the harmful and safe responses, truncate the responses to M tokens, average the hidden states at response positions at layer k, then average over samples to obtain class centers. The unit direction is the normalized difference of the harmful center minus the safe center, and the midpoint is the average of the two centers. The position score is s_k,l^(t)=⟨ h_k,l^(t)−m_k,v̂_k⟩ A positive score means the current hidden state is on the harmful-center side of the hyperplane, which the authors state is equivalent to it being closer to the harmful center. The gate is 1 when the score is greater than 0, otherwise 0. The authors state that softening the score did not outperform this hard gate (Appendix J.2). MDLM scores only positions that are still mask and editable; in Algorithm 1, USDM scores all positions.

   Stage two: SAD at selected positions

   If at least one gate is 1, estimate the unsafe denoiser and β(xt) at the current state. The corrected prediction at position l is p̃_l=𝔼_D[x₀^l|x_t]+ηβ(x_t)a_k,l^(t)(𝔼_D[x₀^l|x_t]−𝔼_Dunsafe[x₀^l|x_t]) The full-sequence prediction is the product of the per-position factors. Unselected positions keep the data denoiser, and the prediction is then passed to the reverse solver. This intervention is not performed outside window C.

   Theoretical conditions and implementation choices

   • Position-level conditional mean: Theorem 3.1 states that if the gate strength equals the ratio of the position's posterior unsafe probability to its complement, then the above expression is exactly the conditional mean under some position-level safety objective. The membership function maps clean sequences to [0,1]. Remark 1 states that this oracle is not available at inference time, that the centered score is only a proxy with no formal guarantee, and that this is of the same kind as SAD's approximation of β*. The unsafe denoiser in the theorem is induced by this membership function and is not necessarily the same as the implementation in Eq. (15).
   • Consistency with a global objective: Theorem A.1 states that the per-position conditional means are simultaneously the marginal denoiser of some global safety objective if and only if the safety weights of the remaining positions are conditionally uncorrelated with the position's clean token under the position-level safety posterior. Remark 2 states that this does not guarantee the existence of a globally consistent distribution.
   • Optional extensions: Warm-up performs full SAD on all eligible mask positions for the first w steps, then switches to selective guidance; no re-masking occurs during this period, and the last step is also not re-masked. Re-masking strategies are none, top-b, and aligned. The main method is w=0 with no re-masking. The main continuation uses warm-up with w=8, i.e., steps 0–7 are full SAD and steps 8–15 are selective guidance.
   • Evaluation: Continuation safety is the unsafe rate of Llama-Guard-3-8B. Harmful PPL is the corpus perplexity of GPT-2-large over all scorable outputs on harmful prompts, including outputs judged safe. Benign BERT and benign PPL use another set of RTP human continuations. Jailbreak is reported separately as HarmBench ASR and Llama-Guard. Over-refusal is the refusal rate of WildGuard on XSTest safe prompts. NFE counts only generator forward passes.

   Collapse this answer

4. 04What experiments did the paper conduct?

   In continuation, SADDER often improves benign quality relative to full SAD, while jailbreak benefits vary with the model and attack.

   Experimental Setup

   • Models under test: LLaDA-8B-Base, Dream-7B-Base for continuation; LLaDA-8B-Instruct, Dream-7B-Instruct for jailbreak; Duo and UDLM for uniform state continuation.
   • Data: MDLM uses 600 unsafe prompts per episode from RealToxicityPrompts (RTP) and BeaverTails; benign control is 600 benign RTP. USDM uses 300 unsafe RTP and 300 benign RTP. Jailbreak uses 100 held-out HarmBench behaviors. XSTest has 250 safe prompts and 200 unsafe controls. Evaluation prompts are separate from the development set.
   • Baselines: unmodified, full SAD, adapted CSD†, BoN, FK; jailbreak also includes DiffuGuard's full and gen. USDM baselines are unmodified, full SAD, CSD†, and BoN, excluding FK.
   • Metrics and judges: Continuation safety is Llama-Guard-3-8B unsafe rate. Harmful PPL uses GPT-2-large. Benign quality uses BERTScore F1 (multiplied by 100 in tables) and PPL. Jailbreak separately reports HarmBench-Llama-2-13b-cls ASR and Llama-Guard. Over-refusal uses WildGuard. BoN/FK search rewards use RoBERTa, final judgment still uses Llama-Guard.
   • Sampling: MDLM continuation: 32 tokens, 32 steps, temperature 1, seeds 42/43/44. USDM: context 128, 128 steps, five seeds. Jailbreak: nominal 128 steps, length 128, temperature 1, three seeds. Continuation unsafe references up to 2000; jailbreak uses 357 WildJailbreak unsafe responses.
   • Cost and operating points: NFE counts only generator forward passes, excluding reward scoring, safety judges, and one-time reference encoding. Continuation operating points are selected on the development set based on unsafe rate; benign quality is evaluated post hoc (Table 7, Appendix E).

   Main Results

   The table below shows Llama-Guard-3-8B unsafe rate (%). The four MDLM rows are the selected operating points from Table 7, with different η and layers, not the same hyperparameters. The two USDM rows use k=12, η=1.5×10^5 from Table 34. Warm-up is not reported for USDM.

   Swipe horizontally to read the table

   Setting	Unguided	Full SAD	SADDER	warm-up
   LLaDA-Base / RTP (Table 1)	38.6	32.4	29.4	31.2
   LLaDA-Base / BeaverTails (Table 1)	38.6	27.1	29.9	24.4
   Dream-Base / RTP (Table 9)	46.8	33.4	35.9	34.7
   Dream-Base / BeaverTails (Table 9)	38.7	22.7	34.1	27.7
   Duo / RTP, k=12, η=1.5×10^5 (Table 34)	51.3	41.1	43.6	Not reported
   UDLM / RTP, k=12, η=1.5×10^5 (Table 34)	68.5	49.5	45.9	Not reported

   • The authors claim that across the four MDLM settings, selective SADDER has higher benign BERT than full SAD and lower benign PPL than full SAD; both variants still have lower benign BERT than unguided (Table 1, Table 9).
   • The authors claim that the benefit depends on the model: on Dream, full SAD with η=10000 has a lower unsafe rate than both SADDER variants. On LLaDA/BeaverTails, selective SADDER is also not lower than full SAD.
   • BoN-8 and FK-8 have NFE of 256, plus reward scoring; SADDER and full SAD in the table have NFE of 32. The authors claim BoN-8 can further reduce unsafe rate. On LLaDA/RTP, CSD's unsafe rate is 22.4%, but harmful PPL is 225, while SADDER is 42.1 and unguided is 21.1 (Table 1).

   Continuation on Uniform State Models

   • The authors claim that on UDLM, SADDER achieves 45.9% unsafe rate, unguided is 68.5%, and the lowest unsafe rate in the full SAD scan is 47.4% (η=10^5, not the same η as 45.9%). Corresponding benign BERT: SADDER 80.92, full SAD used for comparison 80.49, unguided 80.81. The authors claim it is safer than full SAD with no loss in reference similarity.
   • On Duo, the authors claim that full SAD has a lower unsafe rate than SADDER for each η: η=3×10^4 is 43.4% vs 47.2%, η=1.5×10^5 is 41.1% vs 43.6%. At the same η, SADDER has higher benign BERT, at the end of the scan 81.68 vs 80.76. When matching benign BERT, full SAD at 81.77 is 43.4%, SADDER is 45.0%.
   • All three have the same number of generator evaluations as CSD†, with NFE of 128; BoN is N times that. On Duo, BoN-8 is 33.1% (NFE 1024), which the authors claim is the lowest unsafe rate observed. On UDLM, BoN with N=3/5/8 is 70.0%, 68.9%, 69.7%, which the authors claim is not better than unguided's 68.5%. CSD on Duo at k=9 is 35.1%, and at the same layer on UDLM is 63.5%; the authors claim CSD varies by 15.5 percentage points across layers on Duo, while SADDER varies at most 2.2 percentage points when η is fixed. The caption of Figure 1 states the y-axis is Unsafe rate, the x-axis is Benign BERTScore, and larger markers correspond to larger η.

   Jailbreak and Over-Refusal

   • DIJA: In Table 2, Dream unguided has HB ASR of 82.0% and LG unsafe of 80.7%; SADDER is 77.7% and 81.7%, warm-up is 78.7% and 79.7%, all with NFE of 91.7. The authors claim that SADDER and warm-up have HB close to unguided, and LG still judges about four-fifths as unsafe. Dream's DiffuGuard full has HB 33.3%, LG 84.0%, PPL 537. The authors claim the HB reduction does not bring a comparable LG reduction. On LLaDA-Instruct, SADDER's DIJA HB is 79.0%, unguided is 78.7% (Table 20).
   • PAD: Dream's warm-up has HB 42.7%, LG 55.0%, PPL 10.40, unguided is 51.7%, 64.0%, 8.53. The authors report a paired HB decrease of 9.0 percentage points (95% CI [-15.7, -2.3]) and call it an attack-specific trade-off, not general jailbreak robustness. Selective SADDER's HB is still 52.7%.
   • Over-refusal: In Table 2, Dream warm-up's WG refusal rate is 17.9%, unguided is 55.5%. The authors report a decrease of 35.4 percentage points on common scoring outputs (95% CI [-41.6, -29.3]), but the WG harmful rate on unsafe controls rises from 0.0% to 6.9% (responses with valid labels). The authors claim that fewer refusals do not indicate improved usefulness while safety is unchanged. The corresponding WG refusal rate for LLaDA-Instruct is 16.3% vs 20.7% (Table 20).

   Other Ablations and Analyses

   • At the same layer, strength, and window, on LLaDA/RTP the unsafe rate difference of selective SADDER relative to full SAD is -4.1 percentage points (Table 15, 95% CI [-6.1, -2.1]); on BeaverTails it is +2.8 ([-1.1, 6.6]). Figure 2 compares each selected full SAD, with LLaDA/RTP at -3.0 ([-6.4, 0.3]).
   • Appendix J.2: With soft gating τ of 30, 70, 100, the authors claim no consistent difference relative to hard gating; on UDLM the four gates overlap, and on Duo the differences are smaller than seed variation.
   • Table 19: On LLaDA/RTP, SADDER+BoN-2 has an unsafe rate of 26.7%, BoN-2 is 30.6%, both with NFE of 64; harmful prompt PPL goes from 20.90 to 28.80. Benign quality was not measured.
   • Table 12: On LLaDA/RTP, top-1 remasking has an unsafe rate of 28.3% and NFE of 47.0; aligned remasking is 30.6% and NFE 46.6. The main method does not remask.
   • The authors claim that in Figure 3, on unguided unsafe RTP continuations, the open-gate ratio generally increases as denoising progresses. The x-axis is Normalised denoising step, the y-axis is Open-gate ratio.
   • Appendix H.5 compares full text vs only initial mask segments on the same DIJA outputs. After removing the scaffold, unguided LLaDA's LG goes from 82.0% to 53.3%, and HB from 78.7% to 73.0%; the authors claim the judges are not interchangeable as a result.

   Collapse this answer

5. 05What are some points that can be further explored?

   Section 6 states that selective guidance has limitations, and jailbreak improvements depend on the model and operating point.

   Limitations and Future Directions Noted by the Authors

   • Section 6 states that experiments on masked and uniform state diffusion simultaneously show the benefits and limitations of selective guidance.
   • Section 6 states that SADDER improves benign continuation quality relative to full SAD in several settings, and some configurations also reduce unsafe generation; jailbreak improvements depend on the model and operating point.
   • Section 6 states that SADDER has a lower refusal rate and frames it as a way to manage safety–quality and safety–refusal trade-offs. The paper does not have a separate Future Work section, and the Discussion does not list specific follow-up experiments.

   Experimental Coverage

   • Generation models are LLaDA-8B-Base, LLaDA-8B-Instruct, Dream-7B-Base, Dream-7B-Instruct, Duo, and UDLM.
   • MDLM continuation uses RTP and BeaverTails, with 600 unsafe prompts per episode, 3 seeds, 32 tokens and 32 steps; USDM uses 300 RTP prompts, 5 seeds, 128 steps. Jailbreak uses DIJA and PAD, each with 100 held-out HarmBench standard behaviors, 3 seeds, 128 steps.
   • XSTest has 250 safe prompts and 200 unsafe controls. Baselines include unmodified, full SAD, CSD†, BoN, FK; jailbreak also includes DiffuGuard's full repair and gen repair.
   • Safety judges are Llama-Guard-3-8B, HarmBench-Llama-2-13b-cls, and WildGuard; fluency uses GPT-2-large, reference similarity uses BERTScore. The paper does not report agreement between these automatic judges and human annotations.
   • The paper reports NFE and states it excludes reward scoring, safety judges, and reference set preparation. Continuation references are up to 2000, jailbreak references are 357 WildJailbreak responses. The main method's gating is a hard threshold, and the theoretical calibration condition is proxied by centered scores in the implementation (Remark 1).

   Collapse this answer

6. 06Summarize the main content of the paper

   SADDER uses embedding gating for training-free safe denoising; continuation quality is often better, and jailbreaks are not universally stronger.

   SADDER is a training-free safety guidance for text diffusion models: it uses midpoint scores in embedding space to decide where to apply SAD corrections.

   • Problem: Toxic continuation and jailbreaks in TDMs cannot directly use autoregressive training or decoding-time methods. SAD operates only in token probability space, and its weight approximation lacks the safety guarantees promised by the theorem.
   • Approach: Directions are derived from the hidden state difference between harmful and safe responses, with scores relative to the midpoint of the two centers. Only if the score is positive is the SAD update multiplied into the token distribution at that position. MDLM considers only positions still masked, while USDM considers all positions at each step. The authors give conditions under which this update becomes a position-level safety conditional mean and is consistent with a global objective; the scores in the implementation have no formal guarantees.
   • Continuation: On LLaDA-Base with RTP, SADDER's Llama-Guard unsafe rate is 29.4%, full SAD is 32.4%, unguided is 38.6%, all with NFE of 32 (Table 1). In the same table on BeaverTails, warm-up is 24.4%, selective SADDER is 29.9%, full SAD is 27.1%. The authors claim that across the four MDLM settings, selective SADDER has better benign quality metrics than full SAD, but still lower than unguided.
   • Exceptions: On Dream-Base, full SAD (η=10000) has a lower unsafe rate than both SADDER variants (Table 9). On UDLM, SADDER (k=12, η=1.5×10^5) is 45.9%, unguided is 68.5% (Table 34). On Duo, full SAD at the same η is 41.1%, SADDER is 43.6%.
   • Jailbreak: On Dream-Instruct, warm-up's PAD HarmBench ASR is 42.7%, unguided is 51.7%; on DIJA, warm-up is 78.7%, unguided is 82.0% (Table 2). The WG refusal rate on XSTest safe prompts goes from 55.5% to 17.9%. The authors claim that fewer refusals do not equal improved usefulness with safety unchanged, and jailbreak improvements depend on the model and operating point.
   • Authors' conclusion: Section 6 states that SADDER provides an additional way to manage safety–quality and safety–refusal trade-offs. The abstract's overarching claim of "superior to existing methods" is not fully consistent with the tables on Dream continuation, Duo unsafe rate, and DIJA.

   Collapse this answer

Related papers