Skip to content

When Guidance Becomes a Guardrail: Early Interception of Unsafe Inputs in Text-to-Image Generation

When Guidance Becomes a Guard: Early Interception of Unsafe Inputs in Text-to-Image Generation

arXiv: Adversarial Attacks, Backdoors & Poisoning· arXiv:2610.26452· Yanchun Li, Yujin Wu, Gangkai Shen, Ming Yang, Li Zeng, Xianlong Wang, Hangtao Zhang, Shujuan TianXiangtan University; Changsha University of Science and Technology; City University of Hong Kong; Huazhong University of Science and TechnologyIndexed by this site · 2026-10-20· Originally publishedDefenses

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

CSTI uses early CFG comparison to intercept unsafe inputs; across five generators, the Macro TPR at a matched 5% FPR is 92.39%–95.75%.

Threat modelAttackers use constructed or obfuscated prompts to induce text-to-image models to generate protected risk concepts, such as nudity, violence, or other harmful content. The defender reads the conditional and unconditional activations already computed by CFG in the frozen generator and makes a single block at update 15; if flagged, generation stops, and accepted requests continue along the original trajectory. The detector is fitted with canonical concept samples and safety samples, not attack samples, and the threshold comes from an independent safety set.

Questions
Text-to-image diffusion models can be induced by constructed prompts to generate unsafe content such as nudity and violence, while output filtering only intervenes after synthesis ends, by which point the full generation cost has already been incurred. The authors ask whether native generation signals can detect unsafe and adversarial inputs early while falsely rejecting fewer benign requests.
Methods
CSTI reads the difference between the conditional and unconditional activations of CFG on a frozen generator, takes the channel mean and RMS at updates 5, 10, and 15, and fits a linear risk head using canonical concept samples and safety samples, without attack samples. An independent safety set is used for upper-tail rank calibration, and at update 15 it decides to intercept or continue along the original trajectory.
Experiments and results
On five generators, at a matched 5% FPR, CSTI's Macro TPR is 92.39%–95.75%, and the authors state it is highest in 37 of 40 settings. SD1.4's FPR on OVERT-mini is 5.27%. The early-stopping latency reduction across the five generators is 61.41%–71.72%.
Limitations and future work
The paper does not specifically discuss limitations in the conclusion or appendix. The experiments cover five generators, nudity and violence, and OVERT-mini on SD1.4 and SD3.5 Medium. The paper does not report the number of items in each set, the number of repetitions, or human review agreement.
Experimental setupSD1.4、SD1.5 et al. · Ring-A-Bell、MMA-Diffusion et al. · TPR@5% FPR、Macro TPR et al.
Evaluated models
SD1.4SD1.5SDXLSD3.5 MediumZ-Image
Benchmarks
Ring-A-BellMMA-DiffusionCustomP4DI2PUnLearn sourceOVERT-mini
Metrics
TPR@5% FPRMacro TPRAUROCFPRlatency saving
AI summaryResearchers propose Conditional Semantic Trajectory Interception (CSTI), which uses the conditional responses of text-to-image diffusion models to detect and intercept unsafe or adversarial inputs early in generation. The method builds on the safety signals revealed in conditional responses by classifier-free guidance, and calibrates a risk detector by aggregating activation contrasts between conditional and unconditional responses at early checkpoints through compact statistics and a linear readout. CSTI trains concept-specific detectors using only standard concepts and safe samples, requiring no attack samples, intermediate image decoding, or modification of the generator. Across five generators, four adversarial attack sets, and two general NSFW datasets, CSTI performs strongest overall among six baselines at a matched 5% FPR; on the interception path, early interception reduces generation latency by more than 60%.

Researchers propose Conditional Semantic Trajectory Interception (CSTI), which uses the conditional responses of text-to-image diffusion models to detect and intercept unsafe or adversarial inputs early in generation. The method builds on the safety signals revealed in conditional responses by classifier-free guidance, and calibrates a risk detector by aggregating activation contrasts between conditional and unconditional responses at early checkpoints through compact statistics and a linear readout. CSTI trains concept-specific detectors using only standard concepts and safe samples, requiring no attack samples, intermediate image decoding, or modification of the generator. Across five generators, four adversarial attack sets, and two general NSFW datasets, CSTI performs strongest overall among six baselines at a matched 5% FPR; on the interception path, early interception reduces generation latency by more than 60%.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    Intercept unsafe inputs before synthesis completes using the text-to-image generator's native CFG signal.

    Whether unsafe and adversarial inputs can be detected and intercepted before text-to-image synthesis completes, using the generator's native CFG signal.

    • Scenario: The authors state that text-to-image models can be induced by constructed prompts to generate nudity, violence, or other harmful content; final-image moderation can only intercept after the full generation cost has occurred (Figure 1).
    • Existing gaps: The authors state that IGD, Wukong, and FlowGuard have made in-generation detection a viable route, but there remains room to examine the safety information in native conditional computation. The authors state that interception must be sensitive to unsafe and adversarial inputs while also reducing false rejections of benign requests.
    • Observation: The authors state that the difference between conditional and unconditional internal activations of CFG on the same latent state carries safety information early in sampling; compact channel statistics and a linear readout can extract this information.
    • Method: They propose Conditional Semantic Trajectory Interception (CSTI). For each generator and risk concept, the detector is fit using canonical concept samples and safety samples, without attack samples, without decoding intermediate images, and without modifying the generator or accepted sampling.
    • Threat model
      • Goal: The attacker uses constructed or obfuscated prompts to make generated images fall into protected concepts specified by the defender. Examples in the introduction include nudity and violence.
      • Victim system: An unmodified text-to-image diffusion generator. The main experiments use SD1.4, SD1.5, SDXL, SD3.5 Medium, and what the main text calls full Z-Image.
      • Defender capabilities: Read the conditional and unconditional activations already computed by CFG; the generator remains frozen; stop early after flagging, while accepted requests continue along the original trajectory. The paper does not state the attacker's knowledge of weights or internal states, nor does it use terms such as white-box or black-box.
      • Success criterion: The authors state that interception is not judged by whether the attack generates an unsafe image; unsuccessful attacks may still degrade output quality. The main experiments use TPR at a matched empirical 5% FPR.
  2. What related research exists?

    Related work covers pre-sampling screening, generation control, in-generation detection, and representation calibration.

    Pre-sampling screening

    • Text classification: NSFW Text Classifier (michellejieli, 2022) and Detoxify (Hanu and Unitary team, 2020) screen potentially unsafe requests before sampling.
    • Embedding detection: Latent Guard (Liu et al., 2024) detects harmful concepts in learned representations of text-encoder embeddings; GuardT2I (Yang et al., 2024) uses a language model to convert text guidance embeddings into natural language and then detects adversarial prompts. The authors state that such screening cannot see how the generation state evolves during sampling.

    Generation control and adversarial prompts

    • Modifying the generation process: ESD (Gandikota et al., 2023) updates weights to remove target concepts; SLD (Schramowski et al., 2023) and CRG (Choi et al., 2026) use guidance at inference time to suppress unsafe concepts.
    • Adversarial prompts: Ring-A-Bell (Tsai et al., 2024) and MMA-Diffusion (Yang et al., 2024) use constructed prompts to elicit unsafe outputs. The authors state that these works motivate evaluating risk during generation without changing accepted sampling.

    In-generation detection

    • Intermediate states: IGD (Yang et al., 2025) classifies intermediate predicted noise; Wukong (Liu et al., 2026) uses a safety decoder that reuses pretrained cross-attention parameters to read early U-Net features; FlowGuard (Yang et al., 2026) linearly decodes intermediate latents into an approximate image space and then performs visual safety classification.
    • Guidance and screening: DAG (Li et al., 2025) uses cross-attention maps associated with optimized guideline tokens to identify harmful concepts from noisy latents and then applies adaptive safety guidance. InGuard (Wang et al., 2026) combines prompt-embedding screening with a detector on the estimated clean latent during denoising. The authors state that these methods differ in observed signals and readout mechanisms.

    Representation and threshold calibration

    • Diffusion features: DIFT (Tang et al., 2023) extracts intermediate diffusion features for visual correspondence; Diffusion Hyperfeatures (Luo et al., 2023) combines representations across layers and timesteps for semantic matching. The authors state that this motivates using native model features, but this paper focuses on early safety decisions rather than visual correspondence.
    • Calibration: The authors state that discrimination scores alone cannot specify how often benign requests are rejected at a chosen operating point; conformal methods (Angelopoulos and Bates, 2021) use held-out data to calibrate decisions. CSTI separates concept-head fitting from threshold selection, avoiding selecting thresholds on the adversarial sets used for evaluation.

    Baselines and datasets

    • Comparisons: Table 2 compares NSFW Text Classifier, Latent Guard, GuardT2I, IGD, Wukong, and DAG (detection part only). The nudity sets are RAB, MMA-Diffusion, Custom, P4D, I2P, and UnLearn source; violence is RAB and I2P; over-refusal is OVERT-mini (Cheng et al., 2025).

    The authors position CSTI as reading the paired conditional and unconditional responses that CFG already computes, using channel statistics from a few early updates and a linear concept readout, without reconstructing intermediate images and without an attention-based auxiliary safety decoder; unlike methods that rewrite generation, CSTI terminates flagged requests and leaves accepted trajectories unchanged (Section 2). In Table 1, only CSTI is marked as supported in the safe calibration column.

  3. How does the paper address this problem?

    CSTI aggregates the early CFG activation difference and performs a single interception with a linear head and an independent safety set.

    On a frozen generator, CSTI compresses the CFG conditional activation contrasts of a few early checkpoints into a trajectory, scores it with a linear risk head, and performs a single block using an independent safety set.

    Problem setting

    • The objects are a generator G, a prompt p, a random seed r, and a risk concept c specified by the defender. z^(0) is the initial state, z^(t) is the state after the t-th sampling update, for T updates in total.
    • The main evaluation fixes T=50, makes one decision after early update h=15, and h<T。被标记的生成在最终图像解码前停止。
    • 接受的请求从当前状态继续,引导与调度器不变。生成器在拟合和部署中都保持冻结。

    条件对比与轨迹

    • CFG在每个更新、同一前一状态与同一调度时间步上,计算条件激活与生成器原生无条件输入上的激活。条件对比为 Dt=Htp−Ht∅,即提示条件带来的内部响应变化。两支路已由CFG计算,提取该差不需要额外的风险专用生成支路。
    • U-Net上该差为空间特征,Transformer上为图像token特征。每个通道在空间位置或token集合上取均值与RMS;论文写ϵ>0 to ensure numerical stability; the value of ϵ is not given. Each checkpoint concatenates the mean and RMS, with dimension 2C.
    • The experiments use the checkpoint set {5, 10, 15}, concatenated in sampling order, called the conditional semantic trajectory. The observation positions in the main experiments are the U-Net middle block, SD3.5 Transformer block 11, and Z-Image layer 14. The statistical form is the same for both backbones, and the channel dimension varies with the generator.

    Concept adaptation on attack-free samples

    • Fitted separately for each generator and each concept. Positives are canonical samples of the target concept, not selected by whether a given attack succeeded. Attack prompts, attack-generated images, and attack-success labels are not used.
    • Standardize first, then apply 128-dimensional whitening PCA, then class-balanced logistic regression, using its linear decision value as the risk score, where higher means higher risk. The standardizer, PCA, and classifier are fitted only on the specified fitting split order, and weights are not shared across generators or concepts.
    • The authors state that this simple readout is used to examine whether early representations directly expose safety information, rather than to introduce a complex detection network. The main experiments fit on 144 canonical concept samples and 256 benign samples.

    Calibration and blocking

    • After fitting, an independent safety sample not used in concept fitting turns the score into a blocking decision. The upper-tail rank of the score relative to the safety calibration scores is tested as psafe(s)=(1+∑i=1ncal 1[si≥ s])/(ncal+1).
    • This quantity is a ranking score, not the probability that the input is benign. Given a working level α, the input is blocked if this rank is not greater than α. The authors state that when the calibration samples and test safety samples are exchangeable, this rank rule bounds the marginal FPR at α. The main experiments use α=0.075 and a calibration set of 200 prompts.
    • The readout and calibration rule for the same generator and concept are fixed across attack sets. Under the 50-update protocol, blocking at update 15 avoids the remaining 35 updates. Intermediate images are not decoded, and no external vision encoder is used.
  4. What experiments did the paper conduct?

    The authors state that at a matched 5% FPR, CSTI has the highest TPR in 37/40 settings, with Macro from 92.39% to 95.75%.

    Experimental setup

    • Generators: SD1.4, SD1.5, SDXL, SD3.5 Medium, full Z-Image, all unmodified, with 50 sampling updates each. SD1.4/1.5 and SDXL use DDIM, CFG 7.5, at resolutions of 512² and 1024², respectively. SD3.5 Medium and Z-Image use FlowMatch Euler, 1024², with CFG 7.5 and 5, respectively. Hardware is an NVIDIA RTX 4090.
    • Data: The six nudity sets are RAB, MMA-Diffusion, Custom, P4D, I2P, and UnLearn source; violence is RAB and I2P. Each set is paired with an equal number of benign prompts. Over-refusal uses OVERT-mini. The abstract says four adversarial attack sets and two general NSFW datasets; the paper does not explain how these correspond to the set names. The paper does not report the number of items in each set.
    • Splits: 40 settings, i.e., 5 generators times 8 risk prompt settings. Fitting uses 144 canonical concept samples plus 256 benign samples per generator–concept, with no attack samples. Calibration uses 200 benign samples with α=0.075. The benign evaluation set is 300 samples. For over-refusal, trainable methods adapt on the same 200 OVERT samples and are evaluated on 1,600.
    • Baselines: Prompt-level are NSFW Text Classifier, Latent Guard, GuardT2I; in-generation are IGD, Wukong, DAG (detection part only, denoted DAG† in the table). The authors state that they follow published implementations where applicable and made architecture-related adaptations.
    • Metrics: The main metric is TPR at a matched empirical 5% FPR, with AUROC as a supplement. The 5% threshold is determined by the 300 benign evaluation prompts (at most 15 false positives) and is separate from the α=0.075 safety calibration, so Table 2 is not the α=0.075 operating point. Macro first averages over sets within a concept, then weights the two concepts equally.
    • Judgment: Automated evaluation is supplemented by human review (Section 4.1). The paper does not report the review sample size, agreement, or reviewer model, nor the number of repetitions. TPR targets unsafe inputs and is not labeled by whether an attack successfully generated an image.

    Main results

    According to Table 2, TPR (%) at a matched 5% FPR. The comparison Macro is the maximum Macro among the six baselines in that generator block.

    Swipe horizontally to read the table

    GeneratorCSTI MacroComparison MacroCSTI MMA nudityCSTI I2P violence
    SD1.494.8690.80 (IGD)95.6483.07
    SD1.594.7687.05 (IGD)90.2684.66
    SDXL94.7583.81 (IGD)92.1884.27
    SD3.5 Medium92.3977.56 (Wukong)78.3585.65
    Z-Image95.7579.76 (IGD)92.9091.67
    • The authors state that CSTI has the highest TPR in 37 of 40 settings, with Macro from 92.39% for SD3.5 Medium to 95.75% for Z-Image; it is 4.06 to 15.99 percentage points above the strongest comparison baseline for each generator. The authors state that this holds on both U-Net and Transformer.
    • The three cells in Table 2 where CSTI is not highest: SD1.4 P4D nudity is 95.69, IGD 96.49; Z-Image MMA nudity is 92.90, Wukong 96.30; Z-Image UnLearn nudity is 95.07, DAG 96.88.
    • The authors state that results still depend on the prompt sets, and point out the relatively low MMA nudity for SD3.5 Medium and I2P violence for SDXL. Text Classifier has the same numbers in each column across the five generators, with a Macro of 24.95.

    Early readability

    • Layers: Figure 3(a) gives per-layer TPR at a matched 5% FPR across five generators and two concepts. The authors state that multiple layers are readable, with middle layers usually stronger; SDXL nudity goes from 73.57% at the earliest checkpoint layer to 97.62% at the middle block; Z-Image violence goes from 42.41% at layer 0 to 95.50% at layer 14. The authors state that the strongest observed position varies with generator and concept. The column alignment of the remaining per-layer cells in the text extraction is uncertain, so they are not listed here.
    • Checkpoints: The main text states that at update 5, the TPR at a matched 5% FPR is 88.4% for nudity and 85.2% for violence; after combining updates 5, 10, and 15, the overall Macro TPR goes from 92.2% with update 15 only to 94.5%. Figure 3(b) also has labels of 91.6, 95.0, 96.5 and 83.3, 89.4, 92.5; the average of 95.0 and 89.4 is 92.2, and the average of 96.5 and 92.5 is 94.5, consistent with the overall Macro above. The main text does not give per-concept TPR for Step 10 item by item.
    • FPR under calibration and readout: In Figure 3(c), each variant is fit and calibrated separately, without matching the empirical FPR to 5%. The average empirical FPR of the combined representation is 4.2%, and for updates 5, 10, and 15 it is 15.9%, 13.5%, and 10.4%, respectively. The authors state that safety information is present at very early stages, that combining checkpoints provides complementary information, and that this reduces false positives at that operating point. In Figure 3(d), logistic regression has a TPR of 96.5% for nudity and 92.5% for violence; the authors state that the overall average is higher than SVM, MLP, and random forest.

    Over-refusal

    • What was measured: Near-concept benign prompts on OVERT-mini. Trainable methods adapt on the same 200 OVERT samples and report FPR on 1,600; the remaining baselines use their best published configuration. Generators are SD1.4 and SD3.5 Medium (Table 4). This is not the matched 5% FPR protocol of Table 2.
    • Results: On SD1.4, CSTI is 5.27, and the six baselines in the table are DAG 7.78, IGD 15.78, Latent Guard 16.38, Wukong 20.38, GuardT2I 32.50, Text Classifier 38.43. On SD3.5 Medium, CSTI is 8.25 and DAG is 8.11.
    • Authors' interpretation: The authors state that CSTI is lower than all six baselines on SD1.4 and ranks second on SD3.5, 0.14 percentage points behind DAG; and that the FPR of the remaining five baselines ranges from 14.20% to 38.43%. The authors believe that limited additional adaptation can reduce false rejections of difficult benign inputs, and also state that false positives on semantically similar benign prompts remain a limitation, especially on SD3.5.

    Latency and memory

    • What was measured: Under the 50-update protocol, the median latency of full generation and of early stopping at update 15; savings are the paired median reduction. Peak memory is reported in the order of standard generation, monitored full generation, and early stopping (Table 5).
    • Results: The reduction ranges from 61.41% for SD3.5 Medium (21.247s to 8.213s) to 71.72% for SDXL (6.600s to 1.867s). For SD1.4, SD1.5, and Z-Image, Full, Early Stop, and Saving are 2.674s, 0.789s, 70.56%; 2.378s, 0.709s, 70.22%; and 62.360s, 23.488s, 62.21%. Peak memory (GiB) is 3.49/3.49/2.12 for SD1.4, 3.49/3.49/2.11 for SD1.5, and 18.49/18.50/7.09 for SDXL; for SD3.5 Medium and Z-Image it is 12.06 and 12.52, unchanged after early stopping.
    • Authors' interpretation: The authors state that early stopping reduces peak memory on SD1.4, SD1.5, and SDXL, and leaves it unchanged on SD3.5 Medium and Z-Image. The abstract and contributions state that generation latency on the interception path is reduced by more than 60%; the range in Table 5 is 61.41% to 71.72%.

    Other ablations and analyses

    • Table 3, layer-wise Macro across five generators, two concepts, and their prompt sets: mean only has TPR@5% FPR of 85.93 and AUROC of 0.9170; RMS only is 82.56 and 0.8846; mean plus RMS is 94.50 and 0.9802.
    • The authors do not give separate reasons for the three cells in Table 2 where CSTI is not highest, or for the relatively low sets they point out.
  5. What are some points that can be further explored?

    The paper does not specifically discuss limitations in the conclusion or appendix.

    Limitations noted by the authors and future directions

    The paper does not specifically discuss limitations or future directions in Limitations, Discussion, Conclusion, Future Work, or the appendix.

    Scope of experiments

    • The generators tested are SD1.4, SD1.5, SDXL, SD3.5 Medium, and full Z-Image, 5 in total; all use 50 updates, with the decision at update 15 (Section 4.1).
    • The risk concepts are nudity and violence. The nudity sets are RAB, MMA-Diffusion, Custom, P4D, I2P, and UnLearn source; violence is RAB and I2P. Over-refusal evaluation is OVERT-mini, with generators SD1.4 and SD3.5 Medium, 1,600 near-concept samples (Section 4.4, Table 4).
    • The numerical comparison in Table 2 includes NSFW Text Classifier, Latent Guard, GuardT2I, IGD, Wukong, and DAG (detection part). Detoxify, ESD, SLD, CRG, FlowGuard, and InGuard appear in Section 2 or Table 1; Table 2 does not give their TPR.
    • Detector fitting does not use attack samples. The main TPR uses a matched 5% FPR separate from the α=0.075 calibration. The paper does not report the number of prompts per risk set, the number of repetitions, or variance.
    • Automated results are supplemented by human review (Section 4.1); the paper does not report the review sample size or agreement. Efficiency numbers are median latency and peak memory for early stopping across the five generators (Table 5).
  6. Summarize the main content of the paper

    CSTI uses early CFG contrast comparisons for calibrated blocking; the Macro TPR across five generators is 92.39%–95.75%.

    CSTI intercepts risky inputs before sampling ends using the difference between the conditional and unconditional activations of the text-to-image generator's native CFG, without modifying the generator or the accepted sampling trajectory.

    • Problem: Final-image filtering must pay the full generation cost. The authors want to intercept unsafe and adversarial inputs early while controlling false rejections of benign requests.
    • Approach: At updates 5, 10, and 15, take the channel mean and RMS of the activation difference and concatenate them. For each generator and each concept, fit standardization, a 128-dimensional whitened PCA, and a class-balanced logistic regression using 144 canonical concept samples and 256 benign samples, without attack samples. An independent set of 200 safety samples is used for upper-tail rank calibration; the main experiments use α=0.075, and the decision to stop or continue is made at update 15.
    • Main results: In Table 2, at a matched 5% FPR, CSTI's Macro TPR is 94.86% for SD1.4, 94.76% for SD1.5, 94.75% for SDXL, 92.39% for SD3.5 Medium, and 95.75% for Z-Image. The authors state that it is highest in 37 of 40 settings, 4.06 to 15.99 percentage points above the strongest baseline for each generator.
    • Exceptions and other numbers: The three cells where it is not highest are SD1.4 P4D nudity (CSTI 95.69, IGD 96.49), Z-Image MMA nudity (92.90, Wukong 96.30), and UnLearn nudity (95.07, DAG 96.88). At update 5, the matched 5% FPR TPR is 88.4% for nudity and 85.2% for violence (Figure 3(b)). On the 1,600 OVERT-mini samples, CSTI's FPR is 5.27% for SD1.4 and 8.25% for SD3.5 Medium (Table 4). The median latency reduction for early stopping across the five generators is 61.41%–71.72% (Table 5).
    • Authors' conclusion: The authors state that native guidance computation provides a basis for early safety interception, and that a simple linear readout is usable at early checkpoints. The conclusion does not list future work.

Related papers

Show 3 more papers
Read originalarxiv.org