RepIt: Steering Language Models with Concept-Specific Refusal Vectors

1. 01What problem does this paper attempt to solve?

   The authors state that existing steering vectors are too broad in effect, and that the lack of a method for concept-specific isolation itself constitutes an evaluation evasion vulnerability; they propose RepIt.

   Core question: how to isolate concept-specific refusal representations from language model activations, so as to selectively suppress refusal of only one harmful concept while preserving refusal of other concepts.

   • Problem setting and importance: the authors argue that behavioral attributes such as refusal and factuality share overlapping directions in representation space, and this entanglement causes side effects in steering; current safety evaluations assume models will consistently refuse across related concepts, so benchmark-based certification may be evaded, constituting a governance risk.
   • Shortcomings of existing methods: the authors state that existing refusal vectors indiscriminately suppress both harmful and benign responses (citing Arditi et al., 2024; Lee et al., 2024); adversarial fine-tuning tends to produce broad emergent misalignment, but misalignment in a single concept is difficult to achieve (citing Betley et al., 2025; Turner et al., 2025).
   • Core observation: the untreated difference-in-means (DIM) vector generalizes to harmful topics beyond the target concept (transparent bars in Figure 2); the authors argue that decoupling can narrow the edit to the target concept.
   • Proposed method: RepIt (Representing Isolated Targets) decouples the DIM vector through three steps—reweighting, whitening, and orthogonalization; the authors report that it can selectively suppress refusal in the WMD domain using only 12 target samples and editing 100–200 dimensions.
   • Threat model (Section 9):
     • Goal: embed narrow harmful capabilities in the model while maintaining performance on safety benchmarks, so that the model passes standardized safety evaluations yet retains exploitable functionality in a specific domain.
     • Knowledge: the authors assume the attacker has white-box access to model activations.
     • Capability: can compute steering vectors with as few as 12 target samples plus public harmful datasets; three attacker types: training-time (controls the training pipeline), distribution (distributes via model-sharing platforms), inference-time (controls the serving infrastructure, injecting steering vectors during the forward pass).
     • Victim system: open-weight language models and their downstream users; the authors state that attack and defense resources are asymmetric—an attack only needs to find a concept not covered by benchmarks, whereas defensive evaluation faces a combinatorial explosion of concepts.

   Collapse this answer

2. 02What related research exists?

   Related work is divided into three groups: safety alignment and bypass, activation steering and concept erasure, and representation geometry; the baseline is the unprocessed DIM vector.

   Safety alignment and its circumvention

   • Ouyang et al. (2022), Bai et al. (2022), Ganguli et al. (2022)—alignment via fine-tuning and RLHF, the source of refusal behavior.
   • Lermen et al. (2023), Yang et al. (2023), Qi et al. (2024) and Andriushchenko et al. (2024), Zou et al. (2023b), Chao et al. (2023), Wang et al. (2025)—circumvent refusal mechanisms via fine-tuning and adversarial prompts, respectively; the paper states that these works expose gaps in model safety.
   • Betley et al. (2025) and Turner et al. (2025)—the paper cites the former as showing that fine-tuning can introduce broad misalignment across categories, and the latter as showing that misalignment in a single category is difficult to achieve; Marks et al. (2025) implant hidden goals via RL and find they can be detected by SAEs.

   Activation steering and concept erasure

   • Arditi et al. (2024), Marshall et al. (2024)—find that refusal behavior is mediated by directions in activation space; the paper points out that the refusal vectors of Arditi et al. (2024) and Lee et al. (2024) indiscriminately suppress both harmful and benign responses, which is the shortcoming this paper targets.
   • Zou et al. (2023a), Panickssery et al. (2023), Burns et al. (2023)—use contrastive data pairs to extract feature directions for behavior steering.
   • Ravfogel et al. (2020), Haghighatkhah et al. (2022), Belrose et al. (2023), Guerner et al. (2023)—concept removal/erasure techniques, related to this paper's orthogonalization approach.

   Representation geometry

   • Park et al. (2024)—linear representation hypothesis; the paper cites it to explain that orthogonality does not guarantee mechanistic independence.
   • Wollschläger et al. (2025)—proposes that refusal occupies multi-dimensional "concept cones"; the paper states that RepIt operationalizes the partition of entanglement and independent contributions, consistent with this view.
   • Lee et al. (2024), Li et al. (2023), Wang et al. (2024)—condition interventions on behavior at inference time.

   Baselines and benchmarks: the paper's direct comparison baseline is the untreated DIM vector vt (i.e., the Arditi et al., 2024-style approach, the transparent bars in the figures); evaluation benchmarks include WMDP (Li et al., 2024), JailBreakV (Luo et al., 2024), StrongREJECT (Souly et al., 2024), TDC2023, JailbreakBench, AdvBench, Malicious Instruct, HarmBench, ToxiGen.

   The authors position this paper as: existing steering methods achieve high ASR but with overly broad effects, whereas RepIt provides concept-specific isolation capability, and thereby exposes blind spots in benchmark evaluation.

   Collapse this answer

3. 03How does the paper address this problem?

   RepIt decouples the DIM vector through three steps—reweighting, whitening, and controlled orthogonalization—then uses COSMIC to select directions and ACE to apply interventions.

   Overall approach: first use difference-in-means (DIM) to obtain the original vector of the target concept, then use RepIt in three steps to decouple the subspace components shared with non-target concepts, and finally use COSMIC to select the intervention location and ACE to apply the edit.

   Problem Setup

   • Two types of concepts: target concepts (the specific concept to jailbreak) and non-target concepts (harmful queries that should remain refused); success is measured by a dual objective—maximizing target-concept ASR while minimizing the change in non-target-concept ASR.
   • Concept representation: target concepts are represented by categories within the dataset (WMDP-rewritten Bio/Chem/Cyber), non-target concepts are the union of all categories in JailBreakV and StrongREJECT, nntgt = 21; only one target concept is used at a time.
   • DIM vector: for each layer ℓ and each post-instruction token position i, compute the difference between the mean of the harmful category and the mean of the harmless baseline (Alpaca), v_i,ℓ = v⁺_i,ℓ − v⁻_i,ℓ, i.e., the concept vector at that position.

   RepIt Three-Step Decoupling

   • Reweighting: each non-target vector is multiplied by its inverse-norm weight wj = 1/(‖vntgt,j‖+ϵ), ϵ = 1e−6, to prevent large-norm vectors from dominating subsequent analysis.
   • Whitening: the authors report that the covariance matrix of non-target vectors has condition numbers on the order of 10^6 to 10^9 and is near-singular (Appendix C.1), so the target and non-target vectors are whitened using the Cholesky factor L of the ridge-regularized covariance C = 1/nR_w^⊤ R_w + λ I, with λ = 1e−4·mean(Rw²)+1e−12.
   • Controlled orthogonalization: in the whitened space, a thin QR decomposition of the non-target matrix yields an orthonormal basis Q; the projection P of the target vector onto the subspace spanned by Q is computed, and only a controlled proportion αP is subtracted, where α = 1−√(1−ρ), ρ∈[0,1] controls the removal strength; the authors state that full orthogonalization may eliminate the target signal, and cite Park et al. (2024) and Wollschläger et al. (2025) to argue that orthogonality does not guarantee mechanistic independence, hence partial removal is used, and finally the result is mapped back to the original space.

   Direction Selection and Intervention Application

   • COSMIC position selection: COSMIC (Siu et al., 2025b) is used to select the layer and position (i*, ℓ*) from candidate directions; the paper states it was chosen because it uses hidden states rather than substring matching and can adapt to reasoning models with diverse refusal forms; because COSMIC only supports a binary harmful/harmless setting, the search is restricted to the non-target validation set.
   • ρ search: a grid search over ρ on (0,1), selecting the smallest ρ that makes the non-target validation set ASR below 0.1; the validation step does not use target data.
   • ACE intervention: Affine Concept Editing (Marshall et al., 2024) is used to apply the edit at the input of the selected layer on all tokens; the paper states this is equivalent to a static weight edit and can be permanently embedded in the model weights.

   Data and Run Settings

   • Target data: WMDP multiple-choice questions are rewritten by GPT-4.1 into multi-sentence free-response instructions, divided into Bio/Chem/Cyber (train/validation/test 508/127/637, 162/40/205, 509/127/637, Table 4).
   • Data splits: WMD, JailBreakV, StrongREJECT are split 40%/10%/50% into train, validation, test; for cyberattack, the "Malware" category is excluded from JailBreakV.
   • Judging method: LlamaGuard 3 is used to judge whether a response is harmful, and ASR is computed.
   • Run resources: a single RTX A6000 (two for Mistral); max new tokens is 1500 for reasoning models and 500 for non-reasoning models.

   Collapse this answer

4. 04What experiments did the paper conduct?

   On 5 open-source models, RepIt maintains WMD target ASR at 0.4–0.7 and non-target ASR around 0.1, and preserves refusal on unseen benchmarks.

   Experimental Setup

   • Models tested: GLM-4.1V-9B-Thinking, Qwen3-4B-Thinking-2507, Mistral-Small-3.2-24B-Instruct-2506, Phi-4-Mini-Instruct, Llama-3.1-Nemotron-Nano-4B-v1.1, 5 open-source models in total.
   • Datasets: targets are the WMDP-rewritten Bio/Chem/Cyber (test sets 637/205/637 items, Table 4); non-targets are JailBreakV (2000 harmful prompts, 16 categories, Table 2) and StrongREJECT (313 items, 6 categories, Table 3); generalization evaluation uses TDC2023, JailbreakBench, AdvBench, Malicious Instruct; there are also HarmBench same-category and ToxiGen toxicity experiments.
   • Baselines: the original ASR of the un-intervened model (baseline) and the unprocessed DIM vector vt.
   • Metrics and judging: ASR, judged by LlamaGuard 3; the ρ search threshold is non-target validation set ASR below 0.1.
   • Samples and repetitions: main results are reported once per target concept, non-targets aggregated by dataset; the data efficiency experiment uses 5 seeds (20–24) and reports mean and range; the paper does not report the number of repetitions for the main experiments.

   Main Results

   Main results are in Figure 2 (target vs non-target ASR) and Figure 1 (generalization to unseen benchmarks); the paper does not give a per-model numerical table, and the ranges reported in the main text are as follows:

   Swipe horizontally to read the table

   Condition	Target (WMD) ASR	Non-target ASR	Source
   RepIt (vRepIt), 5 models	0.4–0.7	about 0.1	Figure 2
   Unprocessed DIM vector vt	high (authors state comparable to or higher than RepIt)	clearly generalizes, higher than baseline	Figure 2 transparent bars
   RepIt on 4 unseen benchmarks	target categories up to 0.7	non-target ASR increase about 0.1	Figure 1

   • The authors state that RepIt pushes non-target success rates back to baseline levels on all models while maintaining target performance, achieving isolation of category-specific signals.
   • The authors state that entanglement is more pronounced in larger models: the 4B model shows less DIM generalization, while GLM-4.1V (9B) and Mistral-3.2-Small (24B) generalize more.
   • The authors point out that the cyberattack experiment still refuses semantically similar malware prompts that did not participate in training, and argue that representation-level concept vectors can be separated from surface category labels.

   Mechanism Decomposition (Figure 3)

   • What was measured: comparison of four steering vectors—the original target vector vt, the non-target mean vector R(p*,ℓ*), the removed projection αP, and the final vector vRepIt.
   • Results: the authors report that the non-target vector and the projection αP alone can each jailbreak the target concept, with effects comparable to or even better than vt; after removing αP, the steering performance of vRepIt decreases.
   • Authors' interpretation: the same unsafe behavior has multiple representational paths; RepIt reduces spillover by removing the contaminated part. The Chem and Cyber results for LlamaNemo4B use ρ = 0 (projection set to zero).

   Localization and Sparse Ablation (Table 1, Table 6)

   • What was measured: z-scores are computed for the coordinates of the projection αP, only heavy-tailed coordinates with |zi| ≥ 2 are kept, the rest are set to zero, and steering is re-run.
   • Results: ∆ASR (magnitude 1e−2) is essentially unchanged, with deviation within ±0.05 absolute ASR; the number of heavy-tailed dimensions is 96–213, accounting for 3.8%–5.1% of hidden-layer dimensions (e.g., Qwen3T-4B Bio is 96/3.8%, Mistral-3.2-Small Cyber is 213/4.2%).
   • Exception: Table 6 shows that the aggregate stability comes from a balance of bidirectional flips—1.91%–9.43% of samples per model-category pair flip 1→0 or 0→1, which the authors describe as a redistribution of error rather than strict preservation of performance.

   Data Efficiency (Figure 4)

   • What was measured: the pipeline is re-run using only 12 or 24 target-category training samples (2.5–5% of the Bio/Cyber training sets, 7–15% of Chem), reusing p, ℓ, ρ from the full-data run, with 5 seeds.
   • Results: the authors report that 12 samples suffice to isolate the refusal direction, with target ASR suppressed and non-targets remaining low; 24 samples reduce variance, with the mean close to full data.

   Other Ablations and Analyses

   • Euclidean orthogonalization ablation (Figure 8): without reweighting and whitening, partial orthogonalization is done directly in Euclidean space; the ASR differs from full RepIt by 1–3 percentage points, and the authors state the difference mainly comes from the choice of ρ.
   • Low-rank structure (Figure 9): the first 10 principal components of the non-target basis explain over 90% of variance (over 95% for most models), which the authors state explains why the Euclidean approach is also numerically stable in this test.
   • HarmBench same-category (Figure 6): the ASR of Bio/Chem models on the HarmBench chemical biological category (28 prompts) and of the Cyber model on the cybercrime intrusion category (40 prompts) is far lower than on the real target categories; the authors state that benchmarks with same-category labels underestimate the harmful capabilities of the attacked models.
   • Toxicity generalization (Figure 10): with ToxiGen toxicity prompts as the target, RepIt maintains high target ASR and near-baseline non-target ASR on 5 models; the authors point out that on LlamaNemo4B and Phi4-mini, toxicity is nearly orthogonal to other refusal concepts, and the original DIM vector itself rarely spills over.
   • Projection diagnostics (Table 5): covariance condition numbers are in the range 10^6–10^10, and after whitening κspan is always 1.00; the Gini coefficient is about 0.99 for all models; cosine similarity ranges from 0.59 for GLM4.1V Cyber to 1.00 for LlamaNemo4B.

   Collapse this answer

5. 05What are some points that can be further explored?

   In Appendix A, the authors list limitations including semantic overlap, evaluation scope, and COSMIC's binary optimization, and state that results may be conservative.

   Limitations and future directions noted by the authors

   • Semantic overlap and noise (Appendix A): harmful categories themselves are ill-defined and overlap between categories exists; the authors state that they cannot guarantee that cleaning up categories from a human perspective eliminates semantic overlap, that decoupling is incomplete, and that the model may still have vulnerabilities to related harmful behaviors not directly targeted; target concepts may also be entangled with concepts in unevaluated datasets.
   • Evaluation scope (Appendix A): evaluation is mainly conducted on WMD-related concepts; the authors state that future work should extend to other domains when semantically more independent data becomes available.
   • Conservativeness of direction selection (Appendix A): COSMIC only performs harmful/harmless binary optimization and cannot preferentially strengthen steering of the target concept; the authors state that results may be conservative; future work could jointly optimize the target vector and non-target basis, or jointly search ρ together with (p, ℓ); RepIt is compatible with other direction selection methods.
   • Impure refusal category (Appendix A): the DIM vector may be contaminated by prompts the model does not originally refuse, and the ρ search may not converge cleanly; the authors state that the effect measured after correction may be stronger, i.e., the current results are conservative estimates.
   • Limitations of the ASR metric (Appendix A): ASR as judged by LlamaGuard does not mean the output contains actionable, real, weaponizable WMD information (e.g., Qwen3T-4B hallucinates virus strain names).
   • Future directions (Sections 11, 12): the authors state that future work should explore detection methods for undisclosed activation modifications, architecture designs naturally resistant to concept-specific steering, decomposing vRepIt into interpretable latent features, and dynamic, representation-aware auditing.

   Experimental coverage

   • The models tested are 5 open-weight models (4B to 24B); the paper does not report experiments on closed-source models.
   • Target concepts are the three WMDP categories Bio/Chem/Cyber, plus a supplementary experiment on the ToxiGen toxicity target (Appendix C.2).
   • Attack success is judged using only LlamaGuard 3; the paper does not report human evaluation or inter-rater agreement.
   • Data efficiency experiments cover 12 and 24 target samples and 5 seeds; the paper does not report the number of repetitions for the main experiments.
   • All experiments were completed on one or two RTX A6000 GPUs (Appendix B.5); the paper does not report adaptive attack evaluations against existing defenses (e.g., detectors).

   Collapse this answer

6. 06Summarize the main content of the paper

   RepIt decouples the DIM vector into a concept-specific refusal vector, which the authors state exposes a blind spot in benchmark safety evaluation.

   An activation steering attack paper: the authors propose RepIt, which isolates a concept-specific refusal direction from the difference-in-means vector to produce models that pass standard safety benchmarks yet retain capabilities in specific harmful domains.

   • Problem addressed: the authors state that existing refusal vectors are too broad in effect and that misalignment of a single concept is difficult to achieve, while benchmark-based safety certification assumes refusal is consistent across concepts; this gap makes evaluation evasion possible.
   • Method highlights: after computing the DIM vector for the target concept, subtract its subspace components shared with 21 non-target concepts through three steps—reweighting, whitening, and controlled orthogonalization (strength controlled by ρ)—use COSMIC to select the layer and position, choose ρ with non-target validation set ASR below 0.1, and then apply an intervention with ACE that is equivalent to a static weight edit.
   • Main results: on 5 open-source models, RepIt keeps ASR at 0.4–0.7 for WMD target categories and reduces non-target ASR to about 0.1 (Figure 2); on four unseen benchmarks—TDC2023, JailbreakBench, AdvBench, Malicious Instruct—the non-target increase is about 0.1 (Figure 1).
   • Efficiency and localization: only 12 target samples are needed to isolate the direction (Figure 4); edits are concentrated in 100–200 residual dimensions, and after sparsification ∆ASR is within ±0.05 (Table 1).
   • Exceptions and negative results: mechanism decomposition shows that the removed projection αP itself has a jailbreak effect comparable to or even better than the original vector (Figure 3); the aggregate stability of sparsification is actually a balance of bidirectional flips, with up to about 10% behavior change at the single-sample level (Table 6); ASR on HarmBench same-category prompts is far lower than on the real target (Figure 6).
   • Authors' conclusion: the authors argue that RepIt exposes a flaw in current safety evaluation practice—passing a benchmark does not equal being safe—and call for developing dynamic, representation-aware auditing and methods to detect undisclosed activation modifications; the authors plan to release the WMD prompt data through controlled access, and an anonymized version of the code is available.

   Collapse this answer

Related papers