Counterfactual feature steering for identity unlearning with adjacency preservation
Counterfactual Feature Steering for Adjacency-Preserving Identity Unlearning
The authors propose CFS: bounded inference-time identity unlearning in the SAE latent space of the SD 1.4 text encoder, and the authors claim it achieves the best AWUS of 0.296 on 4,000 prompts derived from Celeb-100.
The deployment scenario is hosted inference: the model provider controls the intervention and can activate target directions without maintaining separate weight-edited checkpoints; the authors explicitly state this is not a tamper-proof removal mechanism for users who control the service code. The goal is to remove a text-to-image model's ability to generate a specific identity while preserving adjacent non-target identities.
- Identity unlearning in text-to-image diffusion models requires removing a target identity while preserving semantically adjacent non-target identities (adjacency preservation); the authors argue that existing steering methods have unbounded intervention strength and treat all prompts the same, making it easy to inadvertently affect adjacent concepts.
- Proposes Counterfactual Feature Steering (CFS): trains a one-time k-SAE on layer 9 of the frozen CLIP text encoder of SD 1.4, uses forget/retain statistics to obtain a target direction, performs bounded projection at inference according to each token's alignment with that direction, and provides accompanying AWUS/FaceAWUS/LPIPS-gap metrics.
- On 4,000 prompts derived from Celeb-100 and 8 target identities, CFS's AWUS 0.296, FaceAWUS 0.636, and LPIPS-gap 0.402 are all the highest in Table 1; its I2P nudity detection rate of 0.47% is the lowest in Table 2, and ESD is lower on MMA-Diffusion; on COCO-30K, FID changes from 15.71 to 16.11.
- The authors note that this direction lacks a systematic benchmark for identity unlearning plus adjacency preservation, and that CFS depends on the quality of the underlying SAE representation (Conclusion). Experiments were conducted only on SD 1.4, with 8 celebrities as identity targets; in the ArcFace evaluation, face detection failed for 20.8% of target outputs.
- Evaluated models
- Stable Diffusion 1.4
- Benchmarks
- Celeb-100 派生的 4,000 提示词语料(100 个名人身份)COCO-30K(10,000 条 caption)I2PRing-a-Bell(K77、K38、K16)MMA-DiffusionUnlearn Canvas 风格提示词语料
- Metrics
- AWUSFaceAWUSLPIPS-gapFIDCLIPScoreNudity Detection Rates/image
Counterfactual feature steering (CFS) unlearns target person identities in text-to-image diffusion models while preserving adjacent non-target identities. It requires one-time SAE training and, at inference time, performs bounded concept editing in the sparse autoencoder latent space attached to the text encoder. On 100 celebrity identities and 4,000 prompts from Celeb-100, CFS achieves the strongest trade-off in the evaluation between target erasure and preservation of adjacent and general concepts, and the authors propose the AWUS score to measure this trade-off.
Paper analysis
What problem does this paper attempt to solve?
Studies adjacency preservation in identity unlearning for text-to-image models: removing a target identity without harming semantically adjacent non-target identities.
Core question: how to remove a text-to-image diffusion model's ability to generate a specific identity while preserving semantically or visually adjacent non-target identities (which the authors call adjacency preservation).
- Scenario and motivation: text-to-image models are trained on large-scale web data and may reproduce individuals' identities without permission, raising privacy, safety and compliance concerns (e.g., GDPR); targeted unlearning is seen as a way to remove undesirable concepts while preserving overall utility (per the authors).
- Shortcomings of existing methods: the authors argue that existing steering methods apply fixed-strength interventions in latent space and have two structural problems: the strength is unbounded, and beyond a certain point outputs degrade or representations collapse; and the same strength is applied uniformly to all prompts and tokens, which cannot distinguish strong from weak expressions of the target concept and increases the risk of harming adjacent concepts.
- Core observation: the authors propose first measuring how strongly the current activation aligns with the target concept, then applying a controlled correction; tokens with non-positive alignment are left unmodified, yielding a bounded, input-adaptive intervention.
- Contributions of the paper: proposes the Counterfactual Feature Steering (CFS) method, the adjacency-aware evaluation metric AWUS and its extensions (FaceAWUS, LPIPS-gap), and evaluates on identity unlearning, nudity suppression and artistic style removal.
- Deployment setting: the authors envision deployment in a hosted inference scenario, where the model provider controls the intervention and activates target directions on demand without maintaining separate weight-edited checkpoints; the authors explicitly state this is not a tamper-proof removal mechanism for users who control the service code.
What related research exists?
Related work falls into three groups: diffusion model concept unlearning (weight editing and inference-time steering), SAE interpretability, and machine unlearning beyond diffusion models.
Diffusion model concept unlearning — weight editing direction
- ESD [7]: fine-tunes U-Net cross-attention weights with a contrastive objective to push outputs away from the target concept.
- UCE [8]: closed-form weight update that preserves manually specified anchor concepts.
- MACE [13]: combines LoRA fine-tuning with segmentation-guided masks for large-scale multi-concept erasure. The paper notes that such methods require modifying and storing updated weights, making them hard to reverse or compose.
Diffusion model concept unlearning — inference-time steering direction
- SLD [19]: modifies the classifier-free guidance signal at each diffusion step to suppress unsafe content.
- One Step Is Enough [21]: applies a fixed mean concept vector to the UNet residual stream of SDXL-Turbo; SAeUron [4]: multiplicatively suppresses concept-related SAE features on UNet activations; Concept Steerers [12]: trains an SAE on the CLIP text encoder to find concept directions and subtracts them at inference time.
- FADE [23]: formalizes Concept Neighborhood to preserve related concepts during erasure; EraseBench [1]: documents collateral degradation of visually and semantically related concepts. The paper states that CFS differs from these works in its inference-time, input-dependent projection mechanism and its retention analysis for identities.
SAEs and mechanistic interpretability
- SAEs decompose dense activations into sparse, approximately monosemantic features [2, 22], first used for language models [10] and later for concept identification in diffusion models [24, 25, 12, 4, 21].
Machine unlearning beyond diffusion models
- Machine unlearning was initially aimed at classification models, removing the influence of specific training samples [3, 9]; recent work extends it to LLMs [6, 27] and biometric recognition systems [20]. The paper notes that generative unlearning must remove semantic concepts distributed across many prompts and parameters, which is fundamentally different from single-sample unlearning.
Baselines and benchmarks: the comparison baselines are Concept Steerers, SAeUron, One-Step, SLD (inference-time steering) and ESD, UCE (weight editing); identity evaluation uses the authors' own Celeb-100-derived 4,000-prompt corpus, nudity suppression uses I2P, Ring-a-Bell, MMA-Diffusion, and the style experiment uses the Unlearn Canvas prompt corpus.
The authors position this paper as: addressing the adjacent-retention problem with an input-adaptive, bounded inference-time projection, and providing identity-level retention analysis, in contrast to fixed-strength subtractive steering and weight editing.
How does the paper address this problem?
CFS: computes a target direction in the k-SAE latent space at layer 9 of the frozen text encoder, and performs bounded projection at inference according to token alignment.
Overall approach: project the dense activations of the text encoder's intermediate layers into the SAE latent space, identify the target concept direction there and perform a bounded, input-adaptive per-token projection, then decode the edit back into the hidden space and inject it into the frozen diffusion model. The method is named Counterfactual Feature Steering (CFS); an overview is shown in Figure 1.
Problem setting and SAE representation
- The backbone is Stable Diffusion 1.4: prompts are tokenized into a sequence of T = 77, passed through the CLIP text encoder to obtain hidden states, and condition the U-Net via cross-attention.
- The intervention point is layer ℓ = 9 of the CLIP text encoder, selected by ablation (highest AWUS among the tested layers 4, 6, 8, 9, 10, 11); the authors state this is an empirical choice dependent on task and backbone, not a general layer-selection conclusion.
- A k-sparse autoencoder (k-SAE) is trained on layer 9 activations: zt = TopK(ReLU(Wenc ht + b), k); the decoder is linear in z, so differences in feature space map directly to differences in hidden space. The identity-domain SAE is trained once (about 2 hours) and reused for all 8 identity targets; new targets only require cacheable, gradient-free forget/retain statistics (about 925 KB per target), with no need to retrain the SAE or modify diffusion model weights.
Target concept direction
- The prompt corpus is split into a forget set Pf and a retain set Pr according to whether the target concept c is mentioned, and the mean and standard deviation of the two groups' activations are computed at each token position.
- Top-k features are selected by a discriminativeness score: the numerator rewards strong activation on the forget set and weak activation on the retain set (with a retain penalty λr), and the denominator penalizes features with large activation variance on the retain set, on the grounds that such features have no stable baseline on non-target prompts and subtracting them would introduce inconsistent edits in non-target generation.
- The top-k features form a per-token direction: the unit vector d̂t from the retain centroid toward the forget centroid (coordinates outside I* set to zero).
Counterfactual projection (at inference time)
- A forward hook is attached to the output of layer 9 in the text encoder conditioning branch, performing three steps per token.
- Step one measures the alignment signal: αt = max((zt − μtr) · d̂t, 0), i.e., the positive component of the sparse code relative to the retain centroid along the target axis; the clamp prevents amplifying the concept signal for activations already on the retain side.
- Step two projects in SAE space: z_cf = zt − s·αt·d̂t, s ∈ [0, 1]; s = 0 restores the original representation, and s = 1 is the minimum Euclidean norm projection onto the retain half-space (Proposition 1, proof in Appendix B): positively aligned tokens are projected to the boundary without crossing to the retain side, and non-positively aligned tokens are unchanged.
- Step three decodes back to hidden space: h_CFS = ht − s·αt·d̂t·W_dec^T; the decoding direction is precomputed once per target, so no explicit counterfactual code needs to be constructed at inference time.
Evaluation method
- Proposes AWUS = Target Erasure − Collateral Damage, computed in CLIP embedding space as the difference between the normalized score drop on target prompts and the drop on non-target (adjacent + generic) prompts (Appendix C).
- FaceAWUS uses ArcFace face embeddings to define adjacency and measures the difference between the change in the target face and the change in adjacent faces; LPIPS-gap measures how much more target images change than adjacent/generic images.
- The authors note that CLIP scores and whole-image LPIPS also respond to attributes such as background; ArcFace provides identity-specific evidence but depends on baseline face recognizability and detection success.
- Common hyperparameters: k = 48, λr = 0.1, shared across all experiments, with the projection rule unchanged; the nudity and style experiments use SAEs trained in their respective domains.
What experiments did the paper conduct?
On the 8 celebrity targets, CFS has the highest of the three net utility metrics in Table 1; nudity suppression is lowest in most settings; FID on COCO rises slightly.
Experimental setup
- Model tested: Stable Diffusion 1.4; the intervention is at layer 9 of its frozen CLIP text encoder.
- Baselines: Concept Steerers, SAeUron, One-Step, SLD (inference-time steering); ESD, UCE (weight editing). All methods use the same targets, prompts, seeds, inference steps and guidance; CFS uses s = 1, strength-controllable baselines report their best operating point from their AWUS sweep, and ESD/UCE use the corresponding edited checkpoints.
- Data: 4,000 prompts derived from Celeb-100, covering 100 celebrity identities; 8 targets are selected (Obama, Biden, Musk, Gates, Messi, Ronaldo, Cooper, Johansson), with 4 adjacent identities per target (Appendix A) and 9 shared generic prompts.
- Metrics: AWUS (CLIP space), FaceAWUS (ArcFace space), LPIPS-gap, all higher-is-better; also FID, CLIPScore, Nudity Detection Rate (lower-is-better).
- Evaluation protocol: ArcFace evaluates detected face regions under paired prompts/seeds/generation settings, with references taken from generations of unmodified SD 1.4 rather than real celebrity photos; the nudity detection rate is given by a detector (the paper does not state the detector's name).
- Other: generality evaluation uses 10,000 COCO-30K captions, averaged over the 8 identity directions; runtime is measured on 1 A30, 50 DDIM steps, 20 measurements (3 warm-up runs).
Main results
Table 1: Net utility metrics averaged over the 8 celebrity targets (higher is better; per Table 1)
Swipe horizontally to read the table
Method AWUS ↑ FaceAWUS ↑ LPIPS-gap ↑ SAeUron 0.233 0.300 0.108 Concept Steerers 0.279 −0.077 0.038 UCE 0.198 0.543 0.163 ESD 0.262 0.138 0.051 SLD 0.003 0.093 0.022 One-Step 0.106 0.136 0.033 CFS (Ours) 0.296 0.636 0.402 - The authors state that CFS is the highest of all methods on all three metrics.
- The authors state that methods with higher raw erasure scores incur 12–22 times CFS's collateral damage, which the net metrics correctly penalize (Appendix K's TER/CDR: CFS TER 0.306, CDR 0.011; Concept Steerers TER 0.479, CDR 0.447).
- The per-target radar charts in Appendix I (Figure I.1) show: CFS has balanced shapes across identities on AWUS, occupies the outer ring on FaceAWUS, and encloses the other methods for every identity on LPIPS-gap; SAeUron and One-Step, on most targets, erase so aggressively that ArcFace detects no face, and their polygons collapse toward the center.
Independent face identity verification and general generation quality
- ArcFace paired evaluation: at full strength, CFS's mean target face similarity is 0.160 (n = 38) and adjacent identity retention is 0.760 (n = 56); the face detection failure rate is 20.8% for target outputs and 3.4% for adjacent outputs. The authors state this is consistent with target faces being perturbed more than adjacent faces, but detection failures and fluctuations in baseline identity rendering limit interpretation.
- Baseline self-consistency: the unmodified model's self-consistency across three prompt templates is about 0.15–0.66, with most targets at 0.45–0.66; the authors note that low-consistency targets are weaker tests of identity removal.
- COCO-30K general preservation: on 10,000 captions, unmodified SD 1.4 has FID 15.71 and CLIPScore 27.39, while CFS has FID 16.11 and CLIPScore 26.95 (changes of +0.40 and −0.44).
Strength stability, geometry and semantic alignment
- Strength stability (Figure 3): the authors state that as s increases, CFS gradually removes the identity while pose, lighting and background remain stable, and AWUS rises with strength up to full projection; subtractive methods often degrade at high strength due to unbounded edits. Figure F.1 in Appendix F shows CFS lies in the region of high target-face change and low adjacent-face change.
- Direction geometry (Figure 4 left): most identity directions have low pairwise cosine similarity, which the authors state is consistent with limited overlap among the chosen axes but does not guarantee orthogonality; Ronaldo–Messi is a more entangled pair, and collateral effects remain possible.
- Semantic alignment (Figure 4 right): for Elon Musk, "CEO of Tesla" and "Founded SpaceX" reach 63% and 55% of the explicit-name signal respectively, while Bill Gates stays low; Obama-related circumlocutions activate the target direction more strongly than the adjacent identity Joe Biden. The token-level heatmaps in Appendix H show the alignment signal concentrated on the tokens carrying the name.
Generalization: nudity suppression and style removal
Table 2: Nudity Detection Rate (%, lower is better; SD 1.4 is unmodified; per Table 2)
Swipe horizontally to read the table
Benchmark SD 1.4 ESD SLD-MAX CFS I2P 17.80 2.87 1.74 0.47 Ring-a-Bell K77 85.26 20.00 23.16 4.27 Ring-a-Bell K38 87.37 29.47 32.63 7.42 Ring-a-Bell K16 93.68 35.79 42.11 12.63 MMA-Diffusion 95.70 12.70 52.10 17.48 - The authors state that CFS outperforms inference-time SLD-MAX on all reported settings and is lowest on I2P and Ring-a-Bell; on MMA-Diffusion, ESD is lowest (12.70, vs CFS's 17.48).
- Style removal is a qualitative result (Figure 5): the authors state that CFS progressively removes Van Gogh style features while preserving object and scene structure; Appendix G gives qualitative examples of combining two identity directions and erasing an identity from circumlocutions.
Other ablations and analyses
- top-k and λr (Table 3 left, single target Elon Musk, AWUS at s = 1): nearly 0 for k ≤ 30; 0.375 for k = 48, λr = 0.1; 0.366 for k = 96, λr = 0.8; the authors state that larger k gives a stronger signal and results are relatively stable across λr.
- Intervention layer (Table 3 right): layer 4 0.167, layer 6 0.245, layer 8 0.311, layer 9 0.375, layer 10 0.302, layer 11 0.288, with middle layers best.
- SAE space vs. original hidden space (Table 4, single-target ablation under matched settings): in the original space the target erasure rate is 0.399, slightly higher than 0.387 in SAE space, and AWUS is 0.391, slightly higher than 0.381; but SAE space has lower CLIP collateral drift (0.0065 vs 0.0078) and higher adjacent face similarity retention (0.848 vs 0.722). The authors state the evidence supports SAE's contribution to selectivity, rather than being better on every scalar metric.
- Runtime (Table J.1): CFS 9.50 s/image, unmodified SD 1.4 9.52; Concept Steerers 9.96, One-Step 10.42, SAeUron 15.75, UCE 18.75, ESD (including fine-tuning) 21.23, SLD 25.44.
What are some points that can be further explored?
The authors point to two directions: the lack of a systematic benchmark and SAE representation quality; experiments cover only SD 1.4 and 8 celebrity targets.
Limitations and future directions noted by the authors
- Lack of a systematic benchmark (Conclusion): the authors note that a key limitation of this research direction is the absence of a systematic benchmark for evaluating the "identity unlearning + adjacency preservation" setting; this paper uses public resources such as Celeb-100 to select adjacent identities as best it can, and building a dedicated benchmark is future work.
- Dependence on SAE representation quality (Conclusion): CFS and related concept steering methods depend on the quality of the underlying SAE representations; deeper analysis of SAE learning and its ability to capture concept-specific features is future work.
- Geometric properties do not guarantee retention (Introduction): the authors state that geometric properties such as boundedness along the chosen target axis do not by themselves guarantee retention of all adjacent concepts or image attributes.
- Exceptions to direction entanglement (Section 4.3): most identity directions have low pairwise cosine similarity, but orthogonality is not guaranteed; Ronaldo–Messi is a more entangled pair, and collateral effects may still occur.
- Limited interpretation of the ArcFace evaluation (Section 4.3): detection failures (20.8% for target outputs) and fluctuations in baseline identity rendering limit interpretation; targets with low self-consistency are weaker tests of identity removal.
- Deployment boundary (Introduction): CFS is not a tamper-proof removal mechanism for users who control the service code.
Scope of experiments
- The only model tested is Stable Diffusion 1.4; identity targets are 8 celebrities, with 4 adjacent identities and 9 generic prompts per target, using a Celeb-100-derived corpus of 4,000 prompts.
- Generalization evaluation covers nudity suppression (I2P, Ring-a-Bell K77/K38/K16, MMA-Diffusion) and Van Gogh style removal (qualitative); general generation quality uses 10,000 COCO-30K captions.
- In the main comparison CFS fixes s = 1, and strength-controllable baselines report their respective best-AWUS operating points; Appendix K additionally reports a full-strength CLIP-space comparison, but not all methods have face-space metrics under that comparison.
- The sample sizes for the independent ArcFace validation are n = 38 (target) and n = 56 (adjacent); the paper does not report variance or statistical tests over repeated runs of the main results.
- The paper does not report evaluation against adaptive attacks (bypass methods designed specifically against the CFS mechanism); Ring-a-Bell and MMA-Diffusion in the nudity benchmarks are what the paper calls adversarial prompt sets.
Summarize the main content of the paper
CFS performs bounded, input-adaptive identity unlearning in the SAE latent space, and the authors claim it achieves the best net utility on adjacency preservation.
A paper on identity unlearning in text-to-image diffusion models, focused on the problem of preserving semantically adjacent non-target identities (adjacency preservation) when removing a target identity.
- Problem: the authors argue that existing steering methods have unbounded intervention strength and apply it uniformly to all prompts, easily harming adjacent concepts; identity unlearning requires balancing target erasure against preserving adjacent identities.
- Method: CFS trains a one-time k-SAE at layer 9 of the frozen CLIP text encoder in SD 1.4, uses forget/retain statistics to select discriminative features and construct a target direction; at inference time it measures per-token alignment αt with the target direction and projects by s·αt, with s ∈ [0, 1] ensuring the retain-side boundary is not crossed and non-positively aligned tokens are unchanged. It also proposes three net utility metrics: AWUS, FaceAWUS, LPIPS-gap.
- Main results: on the Celeb-100-derived 4,000 prompts and 8 celebrity targets, CFS's AWUS 0.296, FaceAWUS 0.636 and LPIPS-gap 0.402 are all the highest in Table 1; the authors state that methods with higher raw erasure incur 12–22 times its collateral damage.
- Other evidence: in the ArcFace paired evaluation, target face similarity is 0.160 (n = 38) and adjacent retention is 0.760 (n = 56); on 10,000 COCO-30K captions, FID goes from 15.71 to 16.11 and CLIPScore from 27.39 to 26.95; the nudity detection rate is 0.47% on I2P (lowest in Table 2), and on MMA-Diffusion ESD is lower (12.70 vs 17.48); runtime is 9.50 s/image, roughly on par with 9.52 for unmodified SD 1.4.
- Authors' conclusion: the authors state that CFS achieves the best trade-off among the evaluated methods between target erasure and adjacent retention, and is stable across strengths, whereas subtractive methods degrade under strong interventions; the same projection rule transfers to nudity suppression and style removal.
- Boundaries: the authors note that this direction lacks a systematic benchmark, the method depends on SAE representation quality, and the deployment positioning is hosted inference rather than tamper-proof removal.
Related papers
- SharedSAE: a unified sparse dictionary across u-net blocks and denoising time for diffusion modelsAnalysis / interpretabilityarXiv