Understanding and mitigating dataset contamination in large language model steering

Whether contrastive steering, a one-dimensional activation direction learned from contrastive samples, can still control generation according to the target behavior after the training data is corrupted by noise or adversarial changes.

• Use case: The authors state that this method can modulate various generation behaviors and is being integrated into common products, so understanding robustness is needed. This paper focuses on the dataset used to train the steering direction.
• Existing gap: The authors state that although steering is increasingly used for AI safety, its robustness to noisy or adversarial data corruption remains unclear. Much training data is automatically generated, and the creation process may not be carefully reviewed or protected.
• Authors' categorization: Data that correctly characterizes the target behavior is called inlier, and corrupted data is called outlier, divided into random, mislabeled, and coordinated into another behavior. The key to learning the direction is the high-dimensional mean; the authors believe that if the inlier mean is recovered, the rest of the pipeline can proceed as usual.
• This paper's approach: Observe how corruption changes steerability and activation geometry on multiple open-source models and the Anthropic behavior set, and replace the ordinary mean with the robust mean of Lee & Valiant (2022).
• Threat model:
  • What they want to change: After data becomes stale or manipulated, the steering effect of the target behavior changes; coordinated corruption may also introduce a second unwanted behavior.
  • What they can change: Contrastive samples entering the mean can be replaced with random sentences, positive and negative labels swapped, or replaced with data representing another behavior.
  • Knowledge setting: The paper does not use white-box, black-box, or gray-box to describe whether the attacker can see weights or activations.
  • Victim pipeline: Take the mean difference between activations with and without the behavior at a selected intermediate layer, and at inference time add the vector multiplied by α to the activations.

Contrastive steering. Zou et al. (2023) and Rimsky et al. (2024) use contrast pairs of behaviors such as power-seeking, self-awareness, helpfulness, and sycophancy for steering. The common mechanism is to take the difference of positive and negative activation means at layers where the difference is pronounced (Dev & Phillips, 2019; Subramani et al., 2022), then add it to that layer scaled by coefficient α.

• Other control methods: The paper lists variants such as gradients, sparse autoencoders, gating, and one-shot optimization (Im & Li, 2025; Oozeer et al., 2025a; Parekh et al., 2025; O’Brien et al., 2024; Arad et al., 2025; Nguyen et al., 2025; Dunefsky & Cohan, 2025). The authors say these methods have generally not been shown to be substantially better than simple contrastive methods (Wu et al., 2025) and are more cumbersome to implement. Prompting or fine-tuning is sometimes slightly better, but the authors say it is less stable in long contexts (He et al., 2025) and does not provide the same mechanistic information (Turner et al., 2024).
• Steerability: Braun et al. (2024) discuss that steering reliability varies. Tan et al. (2024) say steerability is driven mainly by dataset quality and generalization, not the model at hand. Arditi et al. (2024) use steering to inject refusal.

Data poisoning and LLM safety.

• Fine-tuning poisoning: Hubinger et al. (2024) say malicious examples in fine-tuning can implant backdoors such as code vulnerabilities or hate under trigger phrases. Fu et al. (2024) say as little as 3% poisoned data can inject content such as political entities and harm helpfulness and instruction following; the paper lists this alongside Wan et al. (2023) and Bowen et al. (2025).
• Label and data manipulation: Taheri et al. (2020) do class label flipping, Shafahi et al. (2018) do clean-label poisoning of vision models, and Zhao et al. (2025) survey large-scale data manipulation. Li et al. (2025) say poisoning effects can persist under higher-level control such as system prompts.
• Steering as defense: The paper says activation steering is considered a defense against backdoors (Oozeer et al., 2025b), general jailbreaks (Zeng et al., 2025), and refusal of dangerous requests (Lee et al., 2024), and says it is often better than computationally heavier fine-tuning (Zhang et al., 2026).

High-dimensional robust mean.

• Provable estimation: Diakonikolas et al. (2019) and Lai et al. (2016) estimate the mean within ℓ2 distance ε when inliers are Gaussian with identity covariance and a constant fraction ε of points can be arbitrarily corrupted. Subsequent work relaxes the inlier assumptions (Diakonikolas & Kane, 2019; Kamath, 2025; Anderson & Phillips, 2025). The paper says inliers need structural assumptions for the problem to be well-defined.
• Sample size: With bounded norm, n = Ω(R²/ε²) is needed; with Gaussian-like inliers, n = Ω(d/ε²), i.e., n much larger than d. The paper gives an example where dimension can be d = 4096, so this is hard to satisfy on LLM activations.
• Empirical comparison: Anderson & Phillips (2025) find that even when n much larger than d does not hold, quantum entropy scaling (Dong et al., 2019), median of means (Lugosi & Mendelson, 2019), and Lee & Valiant (2022) are often better than other methods, but it varies with the setting. After comparing, the authors say Lee-Valiant is the most consistently effective.

Baselines and benchmarks. The comparisons are sample diff of means on poisoned data, diff of means on inliers only, and robust estimators such as Lee-Valiant. Behaviors come from six datasets in the Anthropic evaluation set (following Rimsky et al., 2024; Tan et al., 2025). General capability uses TinyMMLU (Polo et al., 2024).

The authors frame the paper around how dataset poisoning changes contrastive steering: if the inlier mean can be recovered, the downstream hook can remain unchanged. In the impact statement, the authors say attacks on these steering data have not been pointed out before, and say the paper is not introducing steering techniques themselves.

The authors write contrastive steering as the difference between the means of positive and negative behavior activations, and at inference time shift activations along that direction. Contamination changes the dataset that goes into the means; the mitigation is to replace the ordinary sample mean with the Lee-Valiant robust mean.

Problem setup and steering vectors

• Data: The standard form is a triple: prompt, a response without the behavior, and a response with the behavior. The prompt is fed to the model with one of the two responses, activations are read at a fixed layer, and each yields a negative vector and a positive vector.
• Difference in means: At the layer the authors call most effective, the mean of each of the two activation classes is computed separately, and the steering vector is their difference. A hook is added at that layer, and the new activation is shifted by that vector times α. The paper writes that typically α = +1 induces the behavior, α = −1 removes it, and intermediate values are milder.
• Layer and magnitude: For each model and each behavior, the optimal layer found in prior work is used, and the magnitude is tuned on the ground truth steering vector, then shared across estimators, contamination types, and contamination proportions. Appendix A takes the largest α within the range where performance rises monotonically on [−2, 2]. Corrigibility is flipped and denoted incorrigible-neutral-HHH; the authors state that without steering the model itself already has high corrigibility, especially for the two larger model families.

Three types of dataset contamination

• Random: A certain proportion of contrastive samples are replaced with activations of sentences generated randomly character by character, with token length matched to the training distribution. The authors call this the most benign: the bad data does not correspond to the target behavior, has no describable pattern, and could come from automated generation errors.
• Mislabeled: For a certain proportion of samples, the positive and negative examples are swapped, which amounts to adding points that steer toward the negative behavior. The paper says this corresponds to Massart noise and is common in data.
• Coordinated behavior: A certain proportion of steering data is replaced with data representing another behavior. The authors say it is harder to remove, pulls the direction further from the target behavior, and may induce a second unwanted behavior. The relatedness of the two behaviors is the cosine of the steering vectors computed from their inliers alone.
• Proportions: The corruption percentage is the fraction of contrastive pairs that are contaminated. Each set has 1000 pairs, 800 training and 200 test. Results are averaged over 3 draws of which inliers are contaminated; except for mislabeling, they are also averaged over which outliers are injected. Shading is one standard deviation.

Activation-space warm-up and robust estimation

• Warm-up: Section 3.1 first leaves the original text unchanged and only adds positive-negative pairs in activation space: a random direction is chosen and 30% of outliers are clustered far enough away to distort the angle.
• Lee-Valiant: Robust means are computed separately for the positive and negative classes and then differenced, labeled diff. The paper's simplified description is: use the sample mean to determine a central region, downweight points outside the region in proportion to distance, and return the reweighted average.
• Baselines: On contaminated data, the sample difference of means, the inlier-only difference in means, and the Lee-Valiant diff on inliers plus outliers. The authors note that the inlier-only result is not achievable on contaminated data and serves only as a baseline.
• Other estimators: Section 5 compares median-of-means, quantum entropy scoring, and coordinatewise pruning, with hyperparameters at the defaults of Anderson & Phillips (2025). The match variant first differences each paired sample and then takes a robust mean of the difference vectors; under the sample mean the two orders are identical, under robust means they differ.

Readouts

• average score: Following Tan et al. (2025), the average of the logit difference between the positive and negative options on test questions; higher means the behavior is induced more strongly.
• percent steered: The proportion of two-choice questions where the model picks the positive option, taking values in [0, 1].
• Geometry: Cosine relative to the ground truth steering vector, and the norm of the projection onto that direction. Several figures embed schematic relative angles and lengths of the three vectors at 30% contamination.
• Other: Open-ended generation with the options removed, scored by LLM-as-judge for consistency with the target behavior; the authors say they do not use it as the default metric because of scale, with details in Appendix C. General capability uses TinyMMLU.

Experimental Setup

• Models: Llama-3.2-3B-Instruct, Mistral-7b-Instruct-v0.3, OLMo-2-1124-7B-Instruct. When not otherwise specified, Llama is discussed.
• Data: Six behaviors: coordinate-other-ais, myopic-reward, power-seeking-inclination, survival-instinct, incorrigible-neutral-HHH, wealth-seeking-inclination. 1000 pairs per episode, 800/200. The expanded set is shiv96 (2026), 8268 items, 8068 training, 200 test.
• Corruption and repetition: random, mislabeled, coordinated behaviors, and activation space warm-up. Each behavior can be injected with the remaining 5 behaviors. Inlier selection averages 3 times; outlier selection is also averaged except for mislabeled.
• Baselines: sample diff of means on corrupted data, diff of means on inliers only, Lee-Valiant; also median-of-means, quantum entropy scoring, coordinatewise pruning, and diff / match. The no steer dashed line is in the figures, no values are given in the text.
• Metrics: average score, percent steered, cosine and projection norm with the ground truth vector. TinyMMLU is 100 multiple-choice questions, 46 subjects, 5-shot. LLM-as-judge is in Section 4.3 and Appendix C, no thresholds or reviewer model names are given in the text.
• Steering setup: α is tuned on the ground truth vector and then fixed across conditions. The optimal layer comes from prior work, no layer number is given in the text.

Main Results

The main results are figures; there is no average score table in the text to copy. The table below only includes quantitative conditions stated in the text. Figures not otherwise specified belong to Llama-3.2-3B-Instruct.

Swipe horizontally to read the table

Corruption	Quantitative conditions in text	Authors' description of average score	Lee-Valiant
Random	Figure 2, up to 40%, 6 behaviors	Authors say often little effect, especially relative to the no steer dashed line	Up to 30% indistinguishable from inliers only
Mislabeled	Figure 3, above 20%, most behaviors	Authors say it reduces steering	Close to inliers only except at 40%
Coordinated, anti-correlated	Figure 4, cosine -0.67 and -0.57	Authors say inliers decrease and inject outlier bias	Close to diff of means on inliers only
Coordinated, correlated	Figure 6, cosine 0.80 and 0.77	Authors say smaller scale, sometimes inliers even improve, outlier bias remains	More inconsistent, often slightly amplifies the decrease in outlier steerability

• Random and mislabeled: Authors say some steering effect may still be visible under random corruption. Mislabeled at very high corruption rates, Lee-Valiant also no longer approaches inliers.
• Coordinated corruption: Authors distinguish anti-correlated and correlated based on the cosine of the inlier steering vector. Anti-correlated is more consistent with "inliers worsen, outliers strengthen"; for correlated, authors speculate it may be better generalization to a set of related concepts, and say the estimator sometimes treats overlapping samples as outliers and prunes some inliers. Authors still consider suppressing outlier behavior the greater concern and say the estimator usually reduces this effect.
• Two summaries: The abstract states that various corruptions are mostly robust within 10-20% of training data, and are greatly affected beyond that, with coordinated corruption being the strongest. Section 6 states that up to 20% the effect is very limited. Corresponding figures for Mistral and OLMo are in Section 5; the text has no average score to copy, so they are not filled into the table.

Geometry

• Random (Figure 5, survival-instinct): Authors say cosine is almost unaffected, the vertical axis scale is small, but projection norm shrinks; and say when the angle does not move, magnitude corruption may still affect downstream steering. Lee-Valiant aligns with diff of means on inliers only up to the 40% written in the text.
• Mislabeled (Figure 7, same behavior): Authors say the effect on cosine is smaller, and much larger on projection norm. Lee-Valiant's angle roughly follows the corrupted data, with improvement in projection norm under moderate corruption. Authors believe length corruption seems less impactful than angle corruption; if the mechanism can self-regulate length, it can be largely unaffected by this type of corruption.
• Coordinated (Figures 8, 9): Authors say coordinated corruption distorts both cosine and projection norm, with corrupted data moving toward the outlier direction. Lee-Valiant partially mitigates when anti-correlated or uncorrelated. In Figure 9, authors say it mistakenly treats inlier activations as outliers, and after reweighting it is closer to the injected direction. In the appendix, effects vary across different vector pairs.

Metrics, Models, and Ablations

• Percent steered (Figure 10): Authors say discreteness makes results noisier, but mostly similar; Lee-Valiant looks slightly more effective. The lower panel of Figure 10 labels coordinate-other-ais and survival-instinct as corr: -0.67. All experiments are in Appendix B.
• LLM-as-judge (Figure 11): Authors say there is correlation with average score, but overall noisier. The upper panel is coordinate-other-ais and incorrigible-neutral-HHH (-0.67), the lower panel is myopic-reward and power-seeking-inclination (-0.12).
• Other two models (Figures 13, 14): Authors say most corruption experiments are very similar to Llama. The few inconsistencies they show are: under coordinated corruption, inliers still resemble Llama 3B, but injected data does not increase the average score of outlier behavior. Authors consider this a minority case, possibly related to different feasible regions of steering vectors across models; and say Lee-Valiant still mitigates the effect of anti-correlated injection on each model.

Other Ablations and Analyses

• Activation space (Figure 1): Outliers account for 30%, steering worsens as angle increases, error bars are large across 3 trials; authors believe it is still effective within a cone of directions.
• TinyMMLU (Figure 12): The effect of data corruption is small, and authors use this to say activations are not pushed out of distribution; activation space corruption has higher score variance, and authors think it may sometimes push out of distribution. No scores are given in the text.
• Other robust means (Figure 15): The anti-correlated pair is coordinate-other-ais corrupted by incorrigible-neutral. Authors say other methods are consistently worse than Lee-Valiant, most nearly identical to corrupted diff of means; quantum entropy scoring often finds no outliers at all. Authors attribute this to inliers being non-Gaussian and possibly having more dimensions than points. The abstract calls this unexpected.
• Match (Figure 15, bottom row): Authors say there is no consistent improvement, usually less able to remove outlier effects than the default diff, so only standard Lee-Valiant is reported later.
• Data scale (Figure 16): 8068 training items. Authors say corruption still has an effect, Lee-Valiant partially mitigates but does not completely remove it, and variance also decreases; thus the problem is not solely due to small data scale.
• Appendix B: Random, mislabeled, coordinated corruption for three models and six behaviors, plus percent steered and some geometry figures. SNR is labeled next to figures, but in the accompanying text the numbers and curves are crowded together; mismatched columns are not cited.

Limitations and future directions noted by the authors

• Mitigation is not always effective: Section 6 says Lee-Valiant can mostly mitigate these effects, but it is not always effective.
• Other estimators: The same section says many robust estimators designed for similar problems help less consistently; the authors think the most likely reason is that the data do not meet the assumptions of those algorithms.
• Future directions: The authors hope future work will design robust algorithms and training data distributions to mitigate most of these dataset poisonings. (Section 6)

Experimental coverage

• Models and behaviors: The experimental models are Llama-3.2-3B-Instruct, Mistral-7b-Instruct-v0.3, and OLMo-2-1124-7B-Instruct; behaviors are the 6 Anthropic evaluation set behaviors from Section 3.
• Scale and repetition: Main experiments use 1000 pairs per set, 800/200, with poisoning selection averaged over 3 runs; the expanded set has 8268 items, 8068 for training (Sections 4.1, 5).
• Poisoning and estimators: Coverage includes activation-space angle distortion, random sentence replacement, positive-negative swapping, and each behavior injected with the remaining 5 behaviors; compared Lee-Valiant, median-of-means, quantum entropy scoring, coordinatewise pruning, and diff/match (Sections 3–5, Appendix B).
• Readouts: average score, percent steered, cosine and projection norm, TinyMMLU (100 questions, 46 subjects, 5-shot); LLM-as-judge practice is pointed to Appendix C in the main text (Section 4.3).
• Fixed settings: The optimal layer comes from prior work, and α is tuned on ground truth vectors and then shared across estimators and poisoning levels; Section 3 does not give layer numbers.

The authors study how much behavior control at inference time remains after contrastive steering's training data is contaminated, and whether robust means can catch it.

• Object: At a certain intermediate layer, the difference of means between activations with and without the behavior is the steering vector, and at inference time this vector times α is added. The authors care about random bad data, label swapping, and samples replaced with another behavior.
• Comparison: On Llama-3.2-3B-Instruct, Mistral-7b-Instruct-v0.3, OLMo-2-1124-7B-Instruct and six Anthropic behavior sets, they compare the contaminated difference of means, the inlier-only difference of means, and Lee-Valiant, and use cosine and projection norm to look at direction and magnitude.
• Random and mislabeled: The authors state that random replacement up to 40% often has little effect, and that Lee-Valiant up to 30% is indistinguishable from inlier-only. After mislabeling exceeds 20%, steering for most behaviors decreases, and Lee-Valiant is close to inlier except at 40%.
• Coordinated contamination: For anti-correlated examples the cosines are -0.67 and -0.57; the authors state that inlier gets worse and the outlier behavior is injected, and Lee-Valiant is close to inlier-only. For correlated examples the cosines are 0.80 and 0.77, the scale is smaller, and sometimes inlier even improves, but the outlier bias remains, and the estimator may also treat inlier as outlier.
• Other readouts: percent steered trends are mostly similar but more dispersed; LLM-as-judge is correlated but noisier. On TinyMMLU the score change from data contamination is small. In a few plots for the other two models, the outlier behavior score does not rise with injection. Most other robust means are close to the contaminated difference of means.
• Authors' conclusions: The abstract writes that within 10-20% it is mostly still robust, and beyond that the effect grows; Section 6 writes that up to 20% the effect is very limited. When the main behavior is not changed very much, other traits inserted may go unnoticed. Lee-Valiant has little effect on uncontaminated data, but is not always effective; the authors hope that follow-up algorithms and data distributions can mitigate most such contamination.