Skip to content

Not All Steps Matter: Time-Adaptive Spectral Control for Concept Removal in Diffusion Models

Not All Steps Matter: Temporally Adaptive Spectral Control for Concept Removal in Diffusion Models

arXiv: 其他 AI 安全论文· arXiv:2611.18678· Chanaka Thushitha Bandara, Lakmali Nadeesha Kumari, Sen-Ching Samson CheungUniversity of Delaware; University of California, Davis; University of KentuckyIndexed by this site · 2 days ago· Originally publishedDefenses· Its place in the scan record →

Translation in progress — untranslated text is omitted.

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

AI summaryTASN is a training-free concept removal method for text-to-image diffusion models that achieves lower attack success rates than uniform SN on five red-team benchmarks while recovering most of its image quality loss. Time-resolved spectral analysis shows that concept energy is temporally hierarchical: content decays fastest, objects are intermediate, and style persists longest; TASN accordingly adjusts suppression strength and spectral components at each timestep. Ablations confirm that the gain comes from time-directed suppression rather than weaker suppression, and the method generalizes across concept categories and architectures.

Questions
Existing training-free spectral erasure methods apply the same intervention at every denoising step; the authors argue this wastes intervention at low-concept-energy steps and harms image quality, and that more precise concept removal is needed.
Methods
First perform timestep-resolved spectral analysis and find that concept energy decays by category (content fastest, style most persistent); then propose TASN accordingly, making suppression strength proportional to concept energy and tracking only the spectral directions active at each step.
Experiments and results
On SDv1.4, TASN has lower ASR than uniform SN on all five red-team benchmarks (e.g., P4D 10.93% vs. 18.54%), and CLIP recovers from 30.64 to 31.37; the authors state that causal windows and equal-budget controls support that the gain comes from temporal localization.
Limitations and future work
The paper does not specifically discuss limitations. Experimental coverage: the main experiments use only SDv1.4 (30-step DDIM), the temporal hierarchy is validated on SDXL; per-class evaluation uses 20 prompts per concept; the paper does not report human evaluation of generated images.

Related papers

Show 3 more papers
Read originalarxiv.org