Beyond Prompt Attacks: Least-Action Auditing of Concept Erasure in Text-to-Image Diffusion Models
Beyond Prompt Attacks: Least-Action Auditing of Concept Erasure
The authors propose a least-action audit of concept-erased models: on a frozen diffusion model, measure the minimum control cost to revive a concept, which predicts deployment leakage across 18 concept-mechanism pairs (Spearman ρ=−0.76).
- Text-to-image models after concept erasure are usually audited only with prompt attacks, but refusing to answer a certain prompt does not mean the model no longer generates that concept; the authors argue that one should directly measure the residual probability that the frozen generator still assigns to the concept.
- The authors formalize the audit as path-space stochastic control on a frozen reverse SDE: adding a control in the noise channel and minimizing the control energy that drives the trajectory into the CLIP∧DINOv2 dual-detector consensus event yields the least-action barrier ΔSrob, and the external leakage rate is measured with N=1500 deployment samples.
- Across 18 concept-mechanism pairs, ΔSrob and deployment leakage have Spearman ρ=−0.76, with 18/18 pairs correctly ordered within the same concept; SLD barrier is near zero, with 85.5% leakage under explicit prompts; the audit-guided closed-form patch reduces UCE nudity leakage from 16.2% to 0/1500, and RECE church leakage by 3.3×.
- The authors point out that the absolute value of ΔS shrinks with the optimization budget (about 40%), and only the ordering is a conclusion; a high barrier is only an upper bound and does not constitute a proof; the conclusions are limited to erasers on public SD v1.x weights, and the repair is demonstrated only on five targets in two mechanism families.
- Threat model
- This is not an attack paper but an auditing method: the auditor holds a frozen post-erasure Stable Diffusion checkpoint (open weights), adds a control quantity in the noise channel to drive the reverse process, and does not use adversarial prompts; the goal is to estimate the residual probability the model still assigns to the erased concept and to predict the leakage rate under deployment sampling.
- Evaluated models
- ESDUCERECESLDBK-SDM-small
- Benchmarks
- I2P (sexual category, 931 prompts)ImageNette objects (church, parachute, garbage truck, golf ball, gas pump)eight artistic styles
- Metrics
- ΔSrob (least-action barrier, nats)Pleak (sampling-certified leak rate, 95% Clopper–Pearson CI)Spearman ρFID/KIDretained CLIP score
The paper proposes a least-action auditing method for concept erasure in text-to-image diffusion models, which no longer uses prompt attacks to test whether a single prompt can recall an erased concept, but instead measures the residual probability that the frozen generator still assigns to that concept. The least-action barrier is the minimum control cost required to drive the frozen reverse process into the concept event defined by the detector, i.e., the negative log-probability of that event, estimated as ΔS_rob using a CLIP∧DINOv2 consensus terminal, without retraining or adversarial prompts. On 18 concept–erasure-mechanism pairs covering object categories and NSFW nudity, ΔS_rob predicts deployment leakage (counted by an out-of-audit detector on 1,500 new samples per model, pooled Spearman ρ=-0.76, p=5×10^-4), and within each concept, erasure methods with a higher barrier leak less (18/18). The same trajectories can also patch deployed checkpoints: audit-guided closed-form patches reduce UCE nudity leakage from 16.2% to 0/1,500 and reduce RECE church leakage by 3.3×, whereas random edits matched in direction count cannot.
Paper analysis
What problem does this paper attempt to solve?
The authors argue that prompt attacks cannot measure the residual generative capability of erasure models, and propose directly auditing the residual probability via minimum control cost.
Core question: for a text-to-image diffusion model after concept erasure, how to measure the residual generative probability it still assigns to the erased concept, rather than only testing whether some prompt can still evoke it.
- Setting and importance: erased checkpoints are deployed directly with open weights. The authors point out that a model can refuse the target prompt while retaining the concept in its generative dynamics, and deployment leakage depends on the probability mass the frozen reverse process assigns to the target event.
- Shortcomings of existing audits: the authors state that this field audits almost entirely through the conditional interface—re-prompting or running prompt/embedding attacks—whose conclusions depend on the chosen attack and do not provide leakage rates under the deployment protocol.
- Core observation: a diffusion model induces a path distribution over reverse trajectories; adding a control in the noise channel and minimizing the control energy is equivalent to the path-space KL divergence, i.e., the negative log probability of the event. If the residual concept is still common, the action to reach it is small.
- What the paper proposes: the least-action auditing framework and its practical estimate ΔSrob (CLIP∧DINOv2 dual-detector consensus terminal), requiring no retraining and no adversarial prompts; and external ground-truth validation using 1,500 deployment samples per model, with leakage counted by detectors outside the audit; the same trajectory is also used to derive repair directions.
- Audit setup (not an attack paper):
- The auditor holds a frozen erased checkpoint (SD v1.4 public weights), and weights are not updated.
- The control variable is a control vector per DDIM step in the noise channel; no prompts are searched.
- The victim/audited object is the deployed concept-erasure model (ESD, UCE, RECE, SLD four families).
- Success criterion: dual-detector consensus confirms concept revival; external ground truth is the deployment leakage rate counted by NudeNet/ImageNet classifiers.
What related research exists?
Related work falls into four groups: erasure mechanisms, conditional-space attacks, intrinsic diagnostics, and stochastic control theory; this paper is positioned as a post-hoc audit of four types of erasure mechanisms.
Erasure mechanisms (the audit targets of this paper)
- ESD (Gandikota et al., 2023a): fine-tunes the U-Net toward a negative guidance direction.
- UCE (Gandikota et al., 2023b) and RECE (Gong et al., 2024): closed-form editing of cross-attention key/value projections; the paper states that RECE is a strong closed-form eraser.
- SLD (Schramowski et al., 2022): inference-time safety guidance that does not change weights. Li et al. (2026) extend erasure to hundreds of concepts and new backbones.
Condition-space attacks
- UnlearnDiff (Zhang et al., 2023), the text-inversion attack CCE (Pham et al., 2023), the concept-vector attack Ring-A-Bell (Tsai et al., 2023), prompt-search red teaming (Chin et al., 2023), MMA-Diffusion (Yang et al., 2023), and the RECORD recovery attack (Richardson et al., 2025).
- Lu et al. (2025) and Liu and Zhang (2025) show across multiple methods that erased concepts reappear under such probes; Saha et al. (2025) benchmark the side effects of erasure on neighboring concepts.
- The paper points out that these attacks all search the conditioning interface, and the judgment depends on the chosen attack; this paper's barrier searches the reverse trajectory and does not require prompt attacks.
Intrinsic diagnosis
- PROBE (Xie et al., 2026): uses representation probes to diagnose the residual concept capability of erased text-to-video models, closest to this paper's residual-capability perspective; the paper states that this paper's barrier adds a path-space probability interpretation and is validated against externally measured deployment leakage rates.
- SCORE (Fu et al., 2025): uses an information-theoretic robustness bound as the training objective for self-trained models; the paper states that this paper's barrier is a post-hoc metric computable on any frozen checkpoint, including models trained by SCORE.
Stochastic control and rare events
- The least-action perspective of Freidlin–Wentzell and Onsager–Machlup and the Boué–Dupuis representation (Boué and Dupuis, 1998) are the theoretical basis; Premkumar (2024, 2025) connects diffusion likelihood with control cost.
Baselines and benchmarks: the baselines for the repair experiments are three audit-free edits (B1 concept-token reprojection, B2 single-step closed-form adversarial embedding, B3 random directions with matched direction count); leakage evaluation uses the 931 sexual-category prompts of the I2P benchmark and an explicit-prompt deployment distribution constructed by the authors. The authors position this paper as: turning concept-erasure auditing from interface testing into in-model diagnosis of residual generation capability (Table 1 compares the audit families by search variable).
How does the paper address this problem?
The audit is formulated as path-space stochastic control on a frozen reverse SDE, the residual probability is estimated by ΔSrob with a dual-detector consensus terminal, and external validation is done with the deployment sampling leakage rate.
Overall idea: a frozen diffusion model defines a reverse path measure P; adding a control u in the noise channel gives a tilted measure Qu, and the minimum control energy needed to reach the target concept event equals the path-space KL divergence, i.e., the negative log probability of the event. The framework is called least-action auditing, and the practical estimate is ΔSrob.
Ideal barrier and control representation
- Target event E={h(xT)≥τ} (concept present); the ideal audit quantity is the residual generative capability B(E) := −log P(xT ∈ E), i.e., the negative log probability that the frozen model regenerates the concept under its own sampler.
- By the Boué–Dupuis representation, under the hard constraint B(E) = infu: Qu(E)=1 KL(Qu | P) = KL(Q⋆ | P), where Q★ is the Doob h-transform of P conditioned on E; the barrier measures the information needed to tilt the model's natural path distribution to the concept-conditional distribution.
- A low barrier means concept-conditional trajectories are close to the model's natural behavior; a high barrier means reaching the concept requires changing the entire path measure.
Practical estimate ΔS
- Solved with Adam and automatic differentiation through the discrete reverse trajectory of the frozen U-Net (gradient checkpointing, weights not updated): minimize ½∑‖ut‖²Δt + λ·relu(τ−h(xT)).
- The control enters the ε space in the same way as the standard guidance channel; the control family U_B is one control vector per DDIM step, optimized from multiple seeds up to budget B.
- Solver settings (Appendix A): deterministic DDIM (CFG 7.5, 20 steps), τ=3.5, learning rate 0.05, terminal weight λ=20; ΔS takes the mean action over reaching seeds.
- Two by-products: the terminal x★T is the instanton image (the most likely member of E); the energy profile ‖u★t‖² is the spectrum that localizes the barrier to noise times.
Detector-consensus terminal ΔSrob
- A single differentiable indicator h may be partially gamed: the least-action path may satisfy h≥τ but an independent detector does not accept it. ΔSrob uses a soft-AND terminal, requiring both the optimized indicator (CLIP ViT-L/14) and an independent classifier (DINOv2 nearest centroid) to confirm.
- ΔSrob is the minimum action required for two different detector families to both confirm concept revival; a high value means the concept is hard to recover from the score field, a low value means revival is cheap and verifiable.
Validation protocol and external ground truth
- Audit target: SD v1.4 (all four eraser families have public weights), concepts are eight artistic styles and five ImageNette objects; solving is done on the deployment sampler (CFG, concept prompts) so that the unerased model has ΔS≈0.
- Estimator validation: recovers rare-event log probabilities on a 2-D Gaussian-mixture diffusion (Spearman ρ=1.0, slope 0.83); in an event-threshold sweep at SD scale, ΔS rises monotonically with empirical −logP (Spearman 1.0, Girsanov-calibrated variant 0.89, Figure 5); the authors emphasize that this validates only ordering, not absolute calibration, and that all conclusions in the paper are ordering conclusions.
- External ground truth: N=1500 deployment samples per model (CFG, concept prompts, random seeds), counting revivals with NudeNet (NSFW) or ImageNet top-5 (objects), giving exact two-sided 95% Clopper–Pearson intervals; the leakage detector and the barrier's detectors do not overlap at all.
What experiments did the paper conduct?
Across 18 concept-mechanism pairs, ΔSrob and deployment leakage have Spearman ρ=−0.76, with 18/18 pairs correctly ordered within the same concept; the audit-guided patch reduces UCE nudity leakage to 0/1500.
Experimental Setup
- Audited models: original (unerased), ESD, UCE, RECE, and SLD four families of erasers on SD v1.4, 31 erasure models in total (Table 2); concepts are eight artistic styles, five ImageNette objects, and NSFW nudity; transfer is also tested on BK-SDM-small.
- Barrier solving: CLIP ViT-L/14 indicator + DINOv2 nearest-centroid consensus; RECE-vs-UCE paired solving uses nseed=3, niter=60 (Appendix A).
- Leakage measurement: N=1500 deployment samples per model, NudeNet (NSFW, threshold 0.5) or ImageNet top-5 (objects), exact 95% Clopper–Pearson intervals; I2P uses its 931 sexual-category prompts.
- Statistics: Spearman correlation (10⁴–10⁵ permutation test), concept-cluster bootstrap, within-concept permutation test, binomial mixture model, leave-one-concept-out isotonic prediction.
- Repair evaluation: three disjoint seed pools (extraction, tuning, held-out N=1500 certification), criteria fixed in advance; baselines B1/B2/B3 use the same closed-form operator.
- Repetition and variance: solver three-restart standard deviation 8–38 nats; budget sweep {15,30,60,150} iterations.
Main Results
NSFW deployment leakage (NudeNet, N=1500, per Table 6) and the corresponding ΔSrob (Section 4 of the main text):
Model ΔSrob (nats) Pleak [95% CI] original ≈0 0.882 [0.865, 0.898] SLD near zero 0.855 [0.837, 0.873] UCE 3.9 0.164 [0.146, 0.184] ESD 75.5 0.098 [0.083, 0.114] RECE 224.5 0.014 [0.009, 0.021] - The authors report: pooled Spearman ρ=−0.76 on 18 deployment erasers (p=5×10−4; ρ=−0.73 including the unerased anchor, n=24); object domain ρ=−0.82 (n=14), among nudity-domain erasers ρ=−1.0 (Figure 2).
- In within-concept pairwise comparisons, the higher-barrier one has lower leakage in 18/18 (7/7 when limited to weight-editing erasers); leave-one-concept-out isotonic prediction ranks held-out cells correctly in 18/18 (predicted vs. measured ρ=0.61).
- Exception: gas-pump/UCE is a high-barrier outlier (ΔSrob=286, leakage 26%); re-solving at 2.5× budget drops it to 152, which the authors call a uniform budget contraction with all ranking statistics unchanged.
Mechanism Diagnostics (Table 2, Table 3)
- Mechanism ordering: mean ΔS across 31 models is ESD 257.6 ≫ UCE 145.9 ≫ SLD 4.2; median ΔSrob is ESD 209.7 vs UCE 116.2 (Mann–Whitney p≈0.03); consensus-probe confirm-revive is only 2/5 ESD models vs 8/8 UCE models (ESD cells that did not reach record the solver's best attempt, which the authors call a conservative separation).
- RECE vs UCE (Table 3, paired same solver): RECE mean barrier 277.3 vs UCE 107.4 (2.58×, per-concept 1.8–3.7×); confirm-revive 1/3 vs 3/3; the authors state that although RECE is a closed-form method, its barrier falls in the range of fine-tuning score-field removal.
- Targeted exclusion: retained CLIP scores on eight unrelated prompts are maintained for each eraser (0.259 ESD, 0.262 UCE vs 0.262 original), from which the authors conclude that the high barrier is targeted removal rather than overall model damage; ESD has slightly larger drift on unrelated samples (DINOv2 similarity 0.78 vs 0.84, collateral FID 53.8 vs 46.2).
Audit-Guided Repair (Table 4, Figure 4)
- UCE nudity (pre-repair leakage 16.2%, ΔS≈0): the audit-guided patch reduces held-out leakage to 0/1500 (≤0.25%), an independent ViT detector confirms 21.8%→0.13%, retained CLIP change is within 0.0001; the patch simultaneously installs the barrier from ≈0 to 245 (121 at 2.5× budget).
- RECE church (pre-repair 7.1%): the same procedure reduces leakage 3.3× to 2.1% (independent ViT 6.5%→1.2%), barrier 236→291; the cost is collateral FID 89, and the PCA-direction variant has FID 61 at the same leakage of 2.3% (Table 5).
- audit-free baselines: B1 reprojection raises church to 28.5%; B2 adversarial embedding reintroduces the concept on nudity (87.2%); B3 random directions give 8.8% on church (higher than unpatched) and 53.8% on golf-ball/UCE (audit patch 4.1%); the authors state that the audit direction is the active ingredient rather than the number of projection directions.
Other Ablations and Analyses
- Detector robustness: re-evaluating the same samples with Falconsai ViT, method ordering Spearman=1.0 (Table 7); the CLIP zero-shot detector has different absolute rates (κ≤0.21) but perfectly consistent ordering.
- Prompt distribution: on I2P, SLD leakage is 14.1% vs 85.5% on explicit prompts (6× swing); ESD 1.9/9.8%, RECE 1.4/1.4% are stable across mechanisms (Table 8).
- DDIM steps {12,20,28}: ESD mean is higher than UCE in every setting (250>208, 243>207, 244>212).
- Budget sweep: from budget 60→150 the barrier drops by about 40% (church/RECE 240→146), ordering unchanged.
- Action-spectrum causal test: zeroing the early half of the control almost removes all action (church/RECE 173→5), and high-barrier erasers require the entire path (reach 1.00→0.00).
- Second backbone BK-SDM-small: UCE raises the barrier and reduces leakage (church 0→121, 100%→87%; parachute 0→228, 80%→16%).
What are some points that can be further explored?
The authors point out that absolute magnitudes shrink with budget and only ordering is a conclusion, and that the repair is demonstrated only on five targets in two mechanism families; experiments are limited to erasers on public SD v1.x weights.
Limitations and future directions noted by the authors
- Magnitude depends on budget (Section 7): the estimator reaches non-convex local optima, and any feasible control is only an upper bound on the ideal barrier; a low ΔSrob witnesses one cheap confirmed revival (one attack), while a high value is only a bound rather than a proof; from budget 60→150 the barrier drops by about 40%, so the authors cite only ordering as evidence.
- Absolute calibration does not transfer (Sections 3.1, 7): the continuous theory predicts unit slope, but at SD scale the slope does not transfer (Spearman 0.89 is maintained, slope is not), and the authors state that they cite B for meaning and ΔS ordering for evidence.
- Reliance on discriminative indicators (Section 7 Scope): a weak indicator CLIP ViT-B/32 collapses the signal, and DINOv2 consensus can only mitigate this.
- Magnitude of mechanism separation depends on concept (Section 7 Scope): two of the thirteen style concepts are close to tied, while directional consistency always holds.
- Backbone extension is theoretical work (Section 7 Scope): the main results are anchored to public SD v1.x weights; extending to rectified-flow backbones requires redefining the path measure (e.g., small-noise regularization or an initial-noise change of variables), which the authors call a concrete theoretical extension.
- Repair scale is a future empirical direction (Section 7 Scope): the repair is demonstrated only on five targets in two eraser families, and the authors call extending the repair to more mechanisms a direct empirical extension.
Experimental coverage
- Audited models are original, ESD, UCE, RECE, and SLD on SD v1.4, 31 erasure models in total; concepts are eight artistic styles, five ImageNette objects, and NSFW nudity; two concepts, church and parachute, are also tested on BK-SDM-small (Appendix B).
- External ground truth for leakage is N=1500 deployment samples per model (931 prompts for I2P), with detectors NudeNet and ImageNet top-5, plus Falconsai ViT and CLIP zero-shot for robustness checks.
- Repair experiments cover the two main targets UCE nudity and RECE church, with direction-count-matched controls extended to five targets; criteria and seed pools are fixed in advance (Appendix A).
- The paper does not report the number of random seeds for ΔS solving (only the RECE-vs-UCE pairing specifies nseed=3, niter=60), nor the solving budget values for the main grid.
- The rare-event importance-sampling estimator is validated only on a 2-D solvable diffusion (Pearson 0.9997), and no leakage estimate below the 3/N sampling floor is given at SD scale.
Summarize the main content of the paper
The authors propose a least-action audit metric for the residual generation capability of erased models; the barrier predicts deployment leakage and guides repair, turning auditing from interface testing into in-model diagnosis.
A paper on an auditing method for concept erasure in text-to-image diffusion models. The authors argue that mainstream prompt attacks only test the conditional interface and cannot measure the residual probability that the frozen generator still assigns to the erased concept, whereas deployment leakage depends precisely on the latter.
- Method: the audit is formalized as path-space stochastic control on a frozen reverse SDE—adding a control in the noise channel, where the minimum control energy to reach the concept event equals the negative log probability of the event; the practical estimate ΔSrob uses a CLIP∧DINOv2 dual-detector consensus terminal, with no retraining and no adversarial prompts.
- External validation: N=1500 deployment samples per model, with leakage counted by detectors outside the audit. Across 18 concept-mechanism pairs, ΔSrob and leakage have pooled Spearman ρ=−0.76 (p=5×10−4); within the same concept, the higher-barrier one has lower leakage in 18/18; leave-one-concept-out prediction ordering is 18/18.
- Mechanism diagnostics: mean ΔS is ESD 257.6 ≫ UCE 145.9 ≫ SLD 4.2; SLD barrier is near zero, with 85.5% leakage under explicit prompts (statistically indistinguishable from the unerased model's 88.2%), while on I2P it is only 14.1%, which the authors call distribution-specific suppression that a single-benchmark evaluation would miss.
- Repair: the audit trajectory localizes residual concept patterns, and after CCE inversion a closed-form projection is applied: UCE nudity leakage drops from 16.2% to 0/1500, RECE church drops 3.3× to 2.1%; random edits with matched direction counts fail on high-barrier targets or even raise leakage.
- The authors' conclusions and boundaries: the authors state that this turns erasure auditing from interface testing into an in-model diagnostic of residual generative capability; they also emphasize that absolute magnitudes shrink with the optimization budget, that all conclusions are ordering conclusions, that a high barrier is an upper bound rather than a proof, and that the repair is currently demonstrated only on five targets in two mechanism families.