From concepts to interactions: dynamic graph semantic recomposition for safe visual generation
From Concepts to Interactions: Dynamic Graph Semantic Recomposition for Safe Visual Generation
The authors propose DGSR, which performs safety intervention according to dynamic semantic relations on text-to-image/text-to-video models such as FLUX.1-dev; the authors claim it is lowest on 10 of 12 T2I ASR evaluations.
Targets the risk that flow-matching T2I/T2V models (FLUX.1-dev, Open-Sora, Wan2.2-TI2V-5B) generate unsafe content such as nudity, violence and blood under prompt induction; the defense intervenes on attention at inference time with the backbone frozen, and uses prompt sets such as I2P, P4D, UnDiff and RAB (including adversarial prompts adapted to each method) to measure residual risk by ASR.
- T2I/T2V models may generate unsafe content such as nudity and violence, and whether a concept is harmful depends on contextual relations (e.g., a knife used for cooking or for violence); the authors argue that existing concept erasure methods indiscriminately suppress static concepts, which harms benign scenarios.
- Proposes DGSR: during sampling, it dynamically constructs a semantic interaction graph using textual semantic spans and the current visual grounding, predicts the suppression strength of directed relations via a two-layer Edge-GNN, and then applies semantic-level and relation-level interventions to the attention of a frozen backbone through risk gating.
- On FLUX.1-dev, the authors state that DGSR achieves the lowest value in 10 of 12 T2I ASR evaluations; on Open-Sora and Wan2.2-TI2V-5B, the unsafe frame rate is lower than SAFREE, EraseAnything++, and T2VUnlearning; the overall SES in the counterfactual evaluation is 70.4%.
- The paper does not specifically discuss limitations. The evaluation covers 3 generation backbones and three categories: nudity/violence/gore; safety judgment relies on NudeDetector, Q16, and Qwen2.5-VL-7B-Instruct; the paper does not report human evaluation agreement, and the theoretical analysis is limited to fixed-step Euler sampling.
- Evaluated models
- FLUX.1-devOpen-SoraWan2.2-TI2V-5B
- Benchmarks
- I2PPrompting4Debugging (P4D)UnlearnDiffAtk (UnDiff)Ring-A-Bell (RAB)MS-COCO-10KGen prompt set
- Metrics
- ASRN-FIDFIDCLIP ScoreSSIMNudity/Gore RateSubject ConsistencySES
The paper's proposed DGSR (Dynamic Graph Semantic Recomposition) achieves the lowest attack success rate (ASR) on 10 of 12 T2I evaluations, and reduces the average unsafe frame rate for T2V by 41.2% compared with the strongest baseline. The method dynamically constructs a semantic interaction graph during sampling, using paired safe/unsafe supervision to selectively intervene on high-risk semantic relations while preserving benign semantics. The method freezes the pretrained backbone, supports image and video generation, and maintains high generation fidelity and temporal consistency.
Paper analysis
What problem does this paper attempt to solve?
The authors aim to address the problem in T2I/T2V safe generation that "whether a concept is harmful depends on contextual relations," and propose relation-level dynamic intervention.
Core problem: how to suppress only harmful semantic interactions (e.g., "attacking a person with a knife") in T2I/T2V generation while preserving benign interactions (e.g., "cutting vegetables with a knife").
- Problem setting: flow-matching Transformers (such as FLUX) improve T2I/T2V quality but also amplify the risk of generating unsafe content such as nudity and violence; the authors argue that safety alignment must preserve legitimate generation capability while suppressing unsafe content.
- Shortcomings of existing methods: the authors argue that most concept erasure methods treat a single concept as the control unit and perform indiscriminate suppression based on static semantic representations, implicitly assuming that "unsafety is an inherent static property of a concept"; even though CoGFD, GrOCE, SafeRoPE, etc. move toward relation-level control, they still rely on text priors to define relations and are not coupled with the intermediate visual states that evolve during generation.
- Core observation: the same concept carries different risk under different interactions—both "a chef cutting vegetables" and "attacking with a knife" involve a person and a knife; the difference in risk lies not in the knife itself but in how the semantics are combined into a specific interaction; globally suppressing a shared concept harms its legitimate expression (Figure 1).
- Method proposed in the paper: proposes DGSR (Dynamic Graph Semantic Recomposition), which dynamically constructs a semantic interaction graph from evolving text–visual states during sampling, identifies high-risk relations through graph reasoning and selectively intervenes, freezes the pretrained backbone, and uniformly supports T2I and T2V.
- Threat model (defense perspective):
- Risk source: harmful semantic interactions induced by the prompt p, formalized in the paper as the probability that a generated sample contains a harmful interaction (Eq. 1).
- Victim systems: flow-matching generative models FLUX.1-dev (T2I), Open-Sora and Wan2.2-TI2V-5B (T2V).
- Defender capability: intervene on Transformer attention at inference time, with backbone parameters frozen and only auxiliary modules trained.
- Attack surface evaluated: four benchmarks I2P, P4D, UnDiff, RAB, where the adversarial prompts of P4D and UnDiff are adapted to each method.
What related research exists?
Related work falls into three categories: inference-time intervention, training-based erasure, and closed-form weight editing; the authors claim they mostly remain at static concept-level suppression.
Inference-time intervention
- SLD, SAFREE, VideoEraser—suppress unsafe content through safety guidance or representation adjustment.
- SafeRoPE—performs selective attention modulation through risk-aware rotation; the authors argue that such methods still rely on text priors to define semantic relations and do not couple the intermediate visual states during generation.
Training-based erasure
- ESD, EraseAnything—erase target concepts through parameter fine-tuning or auxiliary modules; work in the video direction (such as EraseAnything++, T2VUnlearning) also considers non-target semantic retention and temporal consistency.
Closed-form weight editing
- UCE, RECE, ICE—perform concept erasure through analytical weight updates, among which ICE supports both image and video generation.
Relation-level/compositional control
- CoGFD—introduces a concept logic graph for compositional concept erasure; GrOCE—graph-guided online concept erasure. The authors argue that, like SafeRoPE, they move toward relation-level control but still do not couple semantic relations with the visual states evolving during sampling.
Baselines and benchmarks: T2I compares with ESD, EraseAnything, MACE, SafeRoPE, CoGFD, SAFREE, GrOCE and SLD, with benchmarks I2P, P4D, UnDiff and RAB (1,000 test prompts each) and MS-COCO-10K; T2V compares with SAFREE, EraseAnything++ and T2VUnlearning, using the Gen and RAB prompt sets.
Positioning of this paper: the authors claim that DGSR builds a semantic graph from evolving text–visual representations, uses contextual relation reasoning to guide "generation-state-aware" intervention, shifts safety control from static concept suppression to dynamic semantic relation intervention, and uniformly covers image and video generation.
How does the paper address this problem?
DGSR dynamically constructs a semantic interaction graph during sampling, uses an Edge-GNN to predict relation suppression strength, and then applies two-level interventions to the attention of a frozen backbone.
Overall idea: change safety control from "suppressing isolated concepts" to "identifying and suppressing high-risk semantic relations in the current generation state." DGSR freezes the backbone and proceeds in three steps: dynamically construct a semantic interaction graph (DSIG) → context relation prediction → inference-time semantic recombination; it also uses three-stage progressive training.
Dynamic Semantic Interaction Graph (DSIG)
- Nodes: after NLP tokenization and stop-word filtering of the prompt, N semantic spans (span, such as entities and actions) are obtained as nodes; function words do not become separate nodes, and the original prompt remains unchanged (Appendix A.1, Figure 8).
- Visual grounding: use the text–visual attention inside the Transformer to compute the response distribution Pi of each span over visual tokens (Eq. 3–4); node features concatenate text features, grounding-weighted visual features and response statistics (maximum response, mean, grounding entropy), and under the FLUX configuration the node dimension is 6147 (Table 5).
- Edges: candidate directed edges are fully connected N(N−1) edges; the first-order Wasserstein distance measures the geometric difference between the grounding distributions of two nodes and is converted to the initial edge weight wij=exp(−dij/τot), with τot=0.10; the authors claim that Wasserstein distance can distinguish spatially close from spatially distant distributions, whereas cosine/JSD/Top-K overlap only measures similarity or degree of overlap (Appendix A.2).
Context relation prediction
- Node and edge attributes are first projected into a shared latent space, then a two-layer Edge-GNN alternately performs edge updates, relation-conditioned message passing and node updates (residual MLP + LayerNorm, Appendix A.3).
- The relation suppression head outputs the suppression strength of each directed edge Rij−=fsup([hi(2);hj(2);gij(2)])∈[0,1], with the diagonal set to zero, forming the relation suppression matrix R−.
Inference-time semantic recombination
- Risk gating: take the mean of the top 10% of non-self-loop relation scores as the graph-level gating score sG, and intervene only when the threshold τgate is reached (FLUX.1-dev 0.45, Open-Sora 0.46, Wan2.2 0.30, Table 6); otherwise the original attention is retained.
- Semantic-level suppression: each node obtains a suppression strength ci from its highest-risk outgoing edge, scaling the corresponding text token states (Eq. 11).
- Relation-level recombination: S=A(R−)⊤ gives position-dependent relation suppression, which adjusts the semantic response and is then added to the visual attention output via the span-level Value matrix (Eq. 12–13).
- The intervention ratio is selected on the development set: 40% of sampling steps for Nudity and Blood, 50% for Violence (Appendix B.3).
Three-stage progressive training
- Graph-level risk learning: add an auxiliary classification head on the Edge-GNN and train with BCE using unsafe/benign graph-level labels (lr 1×10⁻⁴, batch 16, 10,000 steps).
- Counterfactual relation supervision: build graphs for paired unsafe/safe conditions at the same latent state, construct a candidate relation set for the modified target interaction, and use a contrastive loss (margin m=0.3) to separate the suppression scores of unsafe and benign interactions (lr 5×10⁻⁵, batch 8, 8,000 steps; Appendix A.4 gives the pairing construction method and example Table 4).
- Generation alignment and retention: optimize the intervention effect with the velocity prediction under safe conditions as reference (Lalign, including normalized MSE and a cosine term), and use Lretain to constrain the prediction before and after intervention to be consistent under benign conditions (lr 1×10⁻⁵, batch 1, 2,000 steps; all loss weights initially 1.0, random seed 42).
Theoretical analysis
- Proposition 1: relation suppression relatively reweights the semantic response; the larger the penalty difference, the lower the response ratio, and equal penalties do not change the ratio (proof in Appendix A.5).
- Proposition 2: under the assumptions of a Lipschitz reference field and local alignment error η, the trajectory deviation upper bound for fixed-step Euler sampling is C(L,T)·η; the authors note that this bound applies only to fixed-step Euler, controls latent distance, and does not directly guarantee perceptual similarity or final output safety (Appendix A.6).
What experiments did the paper conduct?
Three types of unsafe content were evaluated on FLUX.1-dev and two video backbones; the authors state that DGSR achieves the lowest value in 10 of 12 T2I ASR items.
Experimental Setup
- Backbones tested: FLUX.1-dev for T2I; Open-Sora and Wan2.2-TI2V-5B for T2V.
- Baselines: T2I compared with ESD, EraseAnything, MACE, SafeRoPE, SAFREE, SLD, CoGFD, GrOCE; T2V compared with SAFREE, EraseAnything++, T2VUnlearning.
- Benchmarks and scale: T2I evaluates three categories, Nudity, Violence, Blood, using 1,000 test prompts each from I2P, P4D, UnDiff, RAB; MS-COCO-10K evaluates general generation capability; additionally, semantically neighboring concepts and paired benign prompts are constructed to evaluate non-target preservation. T2V evaluates Nudity and Gore, using the Gen and RAB prompt sets.
- Metrics: T2I uses ASR (lower is better), N-FID, COCO FID, CLIP Score, SSIM; T2V uses Nudity/Gore Rate (unsafe frame rate) and Subject Consistency.
- Judgment method: NudeDetector (NudeNet) judges nudity, Q16 judges violence/gore; the counterfactual evaluation uses Qwen2.5-VL-7B-Instruct for unified visual review (fixed parameters, deterministic decoding). All methods use the same detection models, thresholds, and protocol.
- Seeds and protocol: the main evaluation uses a fixed seed of 42, and benchmarks with shared prompts use the same prompt–seed pairs; the adversarial prompts of P4D and UnDiff are adapted to each method, so the original FLUX.1-dev is marked "–" in these two columns in Table 1. The counterfactual protocol uses 3 random seeds.
Main Results
According to Table 1 (FLUX.1-dev, ASR%, lower is better; N-FID/FID lower is better, CS/SSIM higher is better):
Swipe horizontally to read the table
Method Nudity I2P Nudity RAB Violence I2P Violence RAB Blood I2P Blood RAB N-FID (Viol.) COCO FID CS SSIM FLUX.1-dev 15.7 89.7 65.5 75.7 57.3 83.5 – – 30.9 1.000 EraseAnything 7.5 29.4 33.2 51.5 37.9 58.5 30.9 21.8 30.2 0.584 ESD 9.2 11.0 54.9 66.8 34.6 55.1 35.2 23.1 28.4 0.493 SafeRoPE 7.7 22.5 42.9 57.1 39.5 45.9 26.1 21.9 30.8 0.672 GrOCE 8.1 29.0 40.2 52.4 34.4 51.3 27.5 22.3 30.1 0.715 DGSR (Ours) 3.2 19.8 34.8 41.3 23.5 40.7 23.8 21.5 30.8 0.752 (Only some methods and columns are listed in the table; the full 9 methods × 12 ASR items and the P4D and UnDiff columns are in Table 1.)
- The authors state that DGSR achieves the lowest value in 10 of 12 T2I ASR evaluations; exceptions can be found in Table 1: in the RAB column of Nudity, ESD is 11.0, lower than DGSR's 19.8.
- The authors state that on Violence, ASR is relatively reduced by 11.4% compared with the strongest baseline.
- On preservation metrics, DGSR's N-FID is the lowest across the three categories (21.6/23.8/21.7), and SSIM 0.752 is the highest among all methods (Table 1).
According to Table 2 (T2V, unsafe frame rate %, lower is better; Cons. higher is better):
Swipe horizontally to read the table
Method Nudity Gen (Open-Sora) Nudity RAB Blood/Gore Gen Blood/Gore RAB Cons. Open-Sora 58.29 19.21 61.38 35.75 95.92 SAFREE 31.67 14.27 44.71 25.38 93.25 EraseAnything++ 17.10 7.71 27.63 14.21 94.18 T2VUnlearning 16.50 9.53 30.83 12.67 93.20 DGSR (Ours) 10.79 6.83 24.92 13.04 95.04 - On Wan2.2-TI2V-5B, DGSR likewise has the lowest frame rates on each item (Nudity Gen 20.92, RAB 12.04; Blood/Gore Gen 26.00, RAB 15.29, Table 2).
- Exception: in the Blood/Gore RAB column of Open-Sora, T2VUnlearning is 12.67, lower than DGSR's 13.04.
- The authors state that the average T2V unsafe frame rate is relatively reduced by 41.2% compared with the strongest baseline, while maintaining subject consistency (Cons. 95.04 on Open-Sora, close to the original model's 95.92).
Counterfactual Safety–Preservation Trade-off
- What was measured: 180 matched unsafe/benign prompt pairs (60 pairs per category), changing only the key action or interaction target, evaluating Unsafe Interaction ASR and Benign Interaction Accuracy (Appendix C.4).
- Results: the authors state that DGSR substantially reduces unsafe interaction ASR and that benign interaction accuracy is close to the original model (Figure 6); under the stricter SES metric, DGSR reaches 70.4% overall and is the highest in all three categories and overall (Figure 12).
- Authors' interpretation: DGSR more effectively preserves key shared concepts while eliminating harmful interactions.
Ablation and Mechanism Analysis
- Component ablation (Table 3, FLUX.1-dev, Violence): Text-only Graph, Static Relation Policy, and Pairwise MLP (w/o MP) all have higher ASR than full DGSR (e.g., in the I2P column they are 52.4, 57.6, 44.1 respectively, versus 34.8 for the full version); removing semantic-level suppression or relation-level recomposition likewise raises ASR (43.7, 39.5). The authors state that this reflects the contributions of visual-state-aware modeling, message passing, and two-level intervention.
- Relation dynamics (Figure 5, Figure 9): the authors state that the semantic edge strengths of different categories show different evolution patterns over sampling, and that DGSR consistently assigns higher suppression scores to unsafe relations than to benign relations (solid lines higher than dashed lines).
- Risk gating (Figure 10): the authors report a false activation rate of 0% and a miss rate of 6.8% for FLUX.1-dev; both are 0% for Open-Sora; for Wan2.2, unsafe detection is complete but benign false activation is 12.6%.
Other Ablations and Analyses
- Edge metric (Table 7, Violence average ASR / COCO FID): Wasserstein 45.3/22.4, better than Cosine 50.1/22.8, JSD 48.6/22.5, Top-K Overlap 49.4/22.2.
- Intervention ratio (Figure 7): the authors state that increasing the intervention ratio improves safety but gradually increases inference overhead and semantic loss, with gains eventually saturating, so a medium ratio is chosen.
- Computational overhead (Table 8): DGSR's proportion of modified parameters is 0.0224%, modification time is about 1,333 s, and inference is 5.07 s/sample (the original FLUX baseline is not listed in this table; SAFREE is 5.36, SLD 4.71).
- Video spatiotemporal graph construction (Figure 11): the authors state that the grounding and relation strength of video tokens evolve with time position, supporting the same graph framework for modeling dynamic interactions in video.
What are some points that can be further explored?
The paper does not specifically discuss limitations; the experiments cover 3 backbones and 3 categories of unsafe content, and safety determination relies on detectors and an LLM judge.
Limitations and future directions noted by the authors
- The paper does not specifically discuss limitations (there is no Limitations section, and no explicit limitations or future work items are listed in the Conclusion or the appendix).
- Appendix A.6 notes the scope of applicability of the theoretical analysis: the trajectory deviation bound applies only to fixed-step Euler discretization, and other samplers require separate derivation; the local error condition must hold on the states actually visited during sampling, and a small average training loss does not guarantee this pointwise condition; the bound controls latent distance and does not directly guarantee perceptual similarity or the safety of the final output.
Experimental coverage
- The generation backbones tested are FLUX.1-dev (T2I), Open-Sora and Wan2.2-TI2V-5B (T2V), 3 in total; unsafe content covers three categories: Nudity, Violence, Blood (for T2V, Nudity and Gore).
- The T2I evaluation benchmarks are I2P, P4D, UnDiff and RAB, with 1,000 prompts each; the main evaluation uses a fixed seed of 42, and the paper does not report multi-seed repetition or variance for the main table results.
- Safety determination relies on two detectors, NudeDetector and Q16; the counterfactual evaluation uses Qwen2.5-VL-7B-Instruct as the judge, and the paper does not report the agreement between the judge and human judgment.
- The counterfactual evaluation scale is 180 prompt pairs (60 pairs per category) × 3 seeds, covering only the three comparison methods EraseAnything, SafeRoPE and CoGFD.
- The theoretical analysis is limited to fixed-step Euler sampling with a shared time grid (Appendix A.6); the paper does not report evaluation on adaptive adversarial prompts (bypass attacks specifically optimized against DGSR itself).
Summarize the main content of the paper
DGSR changes safety control for visual generation from static concept suppression to dynamic semantic relation intervention, reducing ASR on T2I/T2V while retaining benign semantics.
A defense paper on T2I/T2V safe generation, whose starting point is: whether a concept is harmful depends on the semantic interactions in context, not on the concept itself.
- Problem to solve: the authors argue that existing concept erasure methods indiscriminately suppress static isolated concepts, which harms generation quality in benign scenes containing sensitive words (such as cutting vegetables with a knife), and do not use the visual states evolving during sampling.
- Method highlights: DGSR dynamically constructs a semantic interaction graph during sampling using semantic spans and the current text–visual grounding (Wasserstein edge weights); a two-layer Edge-GNN predicts directed relation suppression strengths; and through risk gating it performs two-level intervention, semantic-level and relation-level, on the attention of the frozen backbone; training has three stages: graph-level risk learning, counterfactual relation supervision, and generation alignment and retention.
- Main results:
- T2I: the authors claim DGSR is lowest on 10 of 12 ASR evaluations on FLUX.1-dev, e.g., ASR 3.2% on Nudity/I2P (original model 15.7%, Table 1); the exception is ESD's lower 11.0 on Nudity/RAB.
- T2V: on Open-Sora, the Nudity/Gen unsafe frame rate is 10.79% (strongest baseline 16.50%), and the authors claim the average frame rate is reduced by 41.2% relative to the strongest baseline (Table 2).
- Counterfactual evaluation: overall SES 70.4% on 180 prompt pairs, the highest among the methods (Appendix C.4).
- Retention and overhead: N-FID is lowest in all three categories, SSIM 0.752 is highest (Table 1); the proportion of modified parameters is 0.0224% (Table 8).
- Authors' conclusion: the authors claim that dynamic semantic interaction modeling is a feasible path to balancing generation safety and semantic retention, and that the framework uniformly supports image and video generation for flow-matching and diffusion models.