Resisting harmful instructions via training-free weight editing

How to reduce a model's compliance with specified undesirable instructions without training, by only changing weights, such that prompt rewriting cannot undo the change.

• Scenario: Instruction following lets models adapt to many tasks but also elicits outputs that conflict with expected behavior and safety alignment. The authors note that undesirable instructions can also arrive indirectly via retrieved documents; in agents, outputs become actions, and failures must be stopped before action.
• Existing shortcomings: System prompts can be removed by the party giving instructions when that party can also modify the system prompt. The authors state that even when a defensive instruction remains, prompting alone still leaves substantial undesirable compliance in several settings. Activation steering does not change weights, and its effect depends on the prompts used to construct the vector and the application setting.
• Approach: Pair undesirable and desired instructions for the same content, score weights using forward-activation contrasts, and rescale the highest-scoring weights in each matrix. The edit is stored in the parameters, requiring no additional defensive prompt and no inference-time steering.
• Threat model:
  • Goal: Reduce compliance with specified undesirable behaviors. This paper evaluates instructed lying and instructed sexism, with masks constructed separately per behavior.
  • Attacker input: Undesirable instructions can appear in the system prompt, user turns, or retrieved documents. Rewriting or deleting these instructions cannot undo the weight modifications already written.
  • Defender capability: Can modify model weights but cannot control the context at use time. Constructing the mask requires only forward passes on a small number of prompt pairs, with no gradients or labeled responses.
  • Assumptions: The adversary cannot modify or replace deployed weights (Limitations). The paper does not describe the attacker as white-box or black-box.
• Comparison scope: Five instruction-tuned models, two behavior domains, compared with no-training interventions: unedited model, system prompt, activation steering, and set-difference weight editing.

Activation contrast and weight importance

• Wanda (Sun et al., 2024): Scores weights by weight magnitude times input channel norm over calibration tokens. The authors instead compute channel norms per prompt and then use the contrast between paired instructions.
• Wei et al. (2024): Contrast pruning scores between a safety dataset and a utility dataset to find a small set of weights whose removal breaks refusals.
• MathNeuro and CreativityNeuro (Christ et al., 2025; Schapiro et al., 2026): Use forward set differences to select weights for changing mathematical reasoning or divergent generation. The authors state that this paper scores by contrast degree rather than taking the difference of two separately ranked sets.

Other parameter edits

• Task arithmetic, weight arithmetic, fact editing, and refusal direction (Ilharco et al., 2023; Fierro and Roger, 2026; Meng et al., 2022; Arditi et al., 2024): Use training differences, closed-form updates, or projections.
• DPO detoxification (Lee et al., 2024): The authors describe it as bypassing toxic capability rather than removing it, and state that this paper's multi-turn results also suggest a similar situation for editing.

Inference-time steering

• Contrast directions (Turner et al., 2023; Rimsky et al., 2024; Stolfo et al., 2025; Zou et al., 2023): Add contrast directions into activations at inference time.
• Generalization citations: The authors cite Tan et al. (2024), Braun et al. (2025), Nguyen et al. (2026), and Pinto (2026), stating such directions generalize unevenly, depend on whether contrast pairs are consistent, and vary with evaluation setting.

Representations and evaluation of lying

• Truth directions and probes (Burns et al., 2023; Marks and Tegmark, 2024; Azaria and Mitchell, 2023): Used for lie detection (Pacchiardi et al., 2024).
• LIARS' BENCH and MASK (Kretschmar et al., 2025; Ren et al., 2025): Evaluate lying. The authors state this paper blocks instructed lies rather than detecting lies.

Baselines and benchmarks

• Baselines: Unedited model, honest system prompt, CAA single-layer activation steering, CreativityNeuro set-difference mask. The authors state that fine-tuning for honesty requires labeled responses and gradients, a different cost type, so it is not compared.
• Benchmarks: LIARS' BENCH instructed-deception split and AI Liar; instructed sexism items and seven prompt formats are constructed by the authors.

The authors position this paper as: training-free weight editing with graded activation contrast, compared under the same protocol with training-free interventions in activation space.

Overall this is training-free weight editing by contrastive activation: construct paired prompts for a specified undesirable behavior, run only forward passes, select and rescale a small number of weights by activation contrast, yielding an architecture-invariant checkpoint.

Paired prompts and channel contrast

• Pairing: F requires the undesirable behavior, T requires the desired behavior, with the same underlying content. One mask is built for lying and one for sexism.
• Lying materials: 25 commonsense questions, or 24 true/false statements (12 true, 12 false, half with negation). Frameworks include fiction, explicit lying requests, and honest mistakes, totaling five families, selected per model on validation data (Appendix C).
• Channel statistic: For each prompt, compute token-level L2 norms of input channels, then average within the set. Prompts are equally weighted; norms are not normalized by length. The authors give an upper bound of 19.1/12.1≈1.26 for length alone raising the norm ratio, below the default threshold.
• Contrast threshold: Default lying setting r=0.37, so the channel contrast is positive if and only if the average norm under F exceeds that under T by about 2.7 times. The authors state this r and the edit proportion come from an earlier parameter search and were not retuned for this paper. Sexism r is selected from {0.37, 0.5, 0.6, 0.7, 0.8, 0.9} on the validation set.

Scoring, selection, and rescaling

• Edit scope: Weight matrices of attention and MLP; embeddings and output head are untouched. A linear layer is y=Wx, and input channel j corresponds to column j of W.
• Score: Weight magnitude times channel contrast. c_j=vn̄^F_j−(1−v)n̄^T_j, v=r/(1+r). Meaning: cj is positive if and only if the average channel norm under F exceeds that under T by a factor of 1/r.
• Mask and update: Take the top ρ=4% of signed scores in each matrix. W'=W\odot(1+\alpha M). Meaning: selected weights are multiplied by 1+α; -1<α<0 为衰减，α=-1 为删除，α<-1 为反号，未选中权重不变。
• 名额填充：正分权重排在零分和负分之前。若正分条目不足 ρ，其余名额由最高的非正分填充，其中很多接近零。作者在 Appendix L 给出固定输入下的局部扰动界，并写明该界不保证编辑后的行为。

两个行为与图示参数

• 说谎强度：α 在 {-0.25, -0.5, -0.75, -1, -1.125, -1.25} 中按验证集选取。
• 性别歧视：25 道性别相关问题，F 要求性别歧视回答，T 要求公平回答。评分规则和 4% 预算与说谎相同。α 在 -0.25 到 -1.5、步长 0.25 的集合中与 r 联合选取（Appendix R）。
• Figure 1：以 Llama-3.1-8B 图解说谎编辑，标注 cj>0 channels account for 1.1%, weights with cj>0 account for 0.8%, near-zero filling 3.2%, top 4% per matrix, α=-1.25.

Selection constraints and judgment

• Data split: Validation and test are split in half by hash. Among options passing usability checks, the one with the best target metric on the validation set is used for testing. Checks are listed in question 4.
• Honesty rate: First run the unedited weights without a lying instruction under greedy decoding to get the model's own beliefs; questions answered incorrectly are removed for all methods. A lie is a response contradicting that belief. The metric is the proportion of deception questions answered according to that belief.
• Construction cost: The mask needs no gradients and no labeled model responses.

Experimental setup

• Main-table models: Llama-3.1-8B-Instruct, Phi-4-mini-instruct, Falcon3-3B-Instruct, OLMo-2-7B-Instruct, Qwen2.5-3B-Instruct, bfloat16, one per family (Section 3.2). Table 1 row names OLMo-2-7B; Table 12 records this setting as OLMo-2-1B (16 layers). The sexism table uses OLMo-2-1B; the format experiment uses Qwen2.5-7B.
• Lying data: LIARS' BENCH instructed-deception has 537 deception questions and 263 honest controls; AI Liar has 1012 commonsense questions. Split in half by hash. Questions the unedited model answers incorrectly without a lying instruction under greedy decoding are removed; the paper does not give the count after removal.
• Baselines: Unedited model, honest system prompt, CAA (same 25 prompt pairs, difference vector at the last token, added to every position of one layer; three layers, five strengths, scaled by the median residual of that layer), set-difference mask. Fine-tuning is not compared.
• Main-table protocol: temperature 0.7, three sampling seeds. Format and sexism experiments use greedy decoding; sexist responses are capped at 200 tokens.
• Evaluation: LIARS' BENCH test half is judged by a mix of GPT-5.6 and a keyword parser; AI Liar is judged by keywords. Option selection mostly uses Qwen2.5-7B. The two judges have κ=0.83 (Table 14). The authors state that changing the judge does not reverse the direction of reported leads. Sexism is labeled by Qwen2.5-32B-Instruct as sexist, fair, refusal, or broken.
• Usability: WikiText-2 perplexity below 1.15 times the unedited model; honest controls drop by less than 5 percentage points; under a lying instruction, honesty on statements the model originally believes true and false each drops by less than 5 percentage points relative to the unedited model. Sexism additionally requires harmless style following and benign helpfulness each to drop by no more than 5 percentage points.

Main results

Swipe horizontally to read the table

Model	LB ours	LB lead	AI ours	AI lead
Llama-3.1-8B	87.7%	+4.4	90.5%	+1.7
Phi-4-mini	90.8%	+8.2	78.5%	+10.0
Falcon3-3B	90.8%	+6.6	79.0%	+1.2
OLMo-2-7B	72.5%	+3.4	84.3%	+0.5
Qwen2.5-3B	94.4%	+1.7	81.7%	+4.2
Five-model average	84.9%	+6.2	79.4%	+4.6

From Table 1. LB is LIARS' BENCH honesty rate, AI is AI Liar honesty rate. Lead is ours minus the highest among other methods in the row, in percentage points. Unedited averages are 63.0% (LB) and 54.5% (AI).

• The authors state that ours is the highest in its row across five models and two benchmarks. By model, the authors state three of five gains exceed noise and two are within noise. The abstract writes the AI Liar lead as 4.6 percentage points, the introduction as 4.5; Table 1 average is +4.6.
• Magnitude varies by model. The text names Phi-4-mini as having the largest lead. Set difference is below ours on all five models and both benchmarks; the authors state this supports weighted activation contrast over set difference selection.
• Name conflict: The OLMo row above is copied from Table 1 as OLMo-2-7B. Table 12 writes the setting behind Table 1 as OLMo-2-1B. The paper does not state which is authoritative.

Instructed sexism

Swipe horizontally to read the table

Model	Ours	Fair prompt	Unedited	Ours + prompt
Llama-3.1-8B	32.8%	37.9%	84.5%	5.2%
Phi-4-mini	3.4%	4.3%	15.5%	0.0%
Falcon3-3B	21.6%	23.3%	50.0%	14.7%
OLMo-2-1B	48.3%	62.1%	72.4%	34.5%
Qwen2.5-3B	15.5%	23.3%	67.2%	5.2%
Average	24.3%	30.2%	57.9%	11.9%

From Table 2, 116 test items, sexist response rate, lower is better. Steering averages 50.9%. Set difference has no configuration passing checks on Falcon3-3B, so no average.

• The authors state that among standalone methods, ours is lowest on all five models. Average improvement over the fair prompt is 5.9 percentage points. In Table 27, the only interval for this lead above zero is OLMo-2-1B (+13.8, [+4.3, +23.3]).
• Combination: Mask plus fair prompt is not separately tuned and averages 11.9%. The authors state the two act on different inputs and can be combined. On the honesty task, the authors state Llama's lying mask plus honest prompt is within noise of the mask alone.
• Cost: Selected masks have perplexity 1.04–1.14 times the unedited model, and MMLU drops 0.5–5.4 percentage points. Validation passes style checks, but on the test set Llama's style following drops from 95.8% to 79.2%, and Qwen2.5-3B from 87.5% to 66.7% (Table 28).

Prompt format transfer

• What was tested: 1,302 LLM-verified commonsense facts, seven formats; construction and evaluation facts and wording are disjoint. Four models, settings selected on the validation set, greedy decoding (Table 3, Appendix H). Baseline is single-layer CAA.
• Single-turn: Average honesty outside the construction format: ours 95.8%, CAA 83.4%, unedited 65.3%. The authors state each model is within 3.1 percentage points of the "unedited and asked to be honest" reference, with perplexity ratios 1.01–1.09. Ours has a construction-format range of at most 4.1 percentage points on any model; CAA's build range on Llama is 29.5, ours is 2.6.
• Multi-turn: Llama goes from 21.4% to 90.3%, Qwen2.5-7B from 60.7% to 86.5%. Phi ours is 66.2%, below unedited 68.0%; Falcon 51.3%, below unedited 58.4%. The authors state single-turn transfer is consistent, multi-turn is another problem; this comparison cannot attribute differences to weight space or activation space.

Human evaluation

• Sample: Four of the planned five raters. 55 LIARS' BENCH honesty responses rated twice, with over-sampling of responses where the two judges disagreed; 43 sexism responses rated twice (Appendix S). The authors state the honesty sample is not designed for direct method comparison.
• Agreement: Four-way κ is human-human 0.48, human-GPT-5.6 0.38, human-Qwen2.5-7B 0.20 (Table 30). For sexism, human-Qwen2.5-32B κ=0.71, human-human 0.67.
• The authors state: GPT-5.6 differs from humans within noise on our mask, but rates steering and honest prompt honesty 25 and 28 percentage points higher than humans, so any bias in Table 1 understates our lead over these two. Weighted human-observed sexism rates: ours 38%, fair prompt 48%, unedited 53%. Fair helpfulness scores 3.45, 3.12, 2.75. The authors state the sexism difference is not yet significant, with intervals around ±15 to 20 percentage points.

Other ablations and analyses

• Editing only positive-score weights matches the full mask at 85.0% on LIARS' BENCH; near-zero filling alone is 65.1% (Table 8, five-model average, Qwen2.5-7B judge).
• In the matching ablation, graded contrast beats set difference by 13.6 percentage points on average (Appendix A.2; Table 21 uses Qwen2.5-7B judge, not re-judged by GPT-5.6).
• Multi-layer CAA trails ours by 4.2 (LIARS' BENCH, Qwen2.5-7B judge) and 5.3 (AI Liar, keywords) percentage points on average (Table 6).
• Without retuning, with Table 1 options: document injection averages +7.2 percentage points, 95% CI [5.3, 9.1]; multi-turn averages -14.3 percentage points (Table 10). This is not the same setting as the multi-turn gains for some models in the format experiment.
• On Table 1 options, MMLU changes by -3.4 points on average (Table 4, 2000 items); IFEval strict accuracy changes by -6.2 points on average (Table 9).
• In the scale experiment, gains relative to unedited rise with parameter count; Qwen2.5-32B is +75.5 percentage points on LIARS' BENCH. The lead over the strongest other method does not rise accordingly (Appendix N).

Limitations and future directions noted by the authors

• Other behaviors: Experiments cover instructed lying and instructed sexism; effectiveness for other behavior targets remains to be evaluated (Limitations).
• Transfer scope: Cross-lingual and multi-turn transfer is unstable; resisting instructed lying does not imply generally reducing dishonesty (Limitations). The Discussion also states that incentive-driven lying is not addressed and multi-turn results are mixed.
• Selectivity: The edit affects some legitimate fiction requests, harmless style following, and general capability. The authors suggest richer reference prompts and broader capability checks during selection (Limitations; Discussion calls selectivity the main improvement direction).
• Deployment assumption: The intervention assumes the adversary cannot modify or replace deployed weights (Limitations).
• Selection and comparison: Performance depends on construction prompts and settings selected on the validation set; search budgets differ across methods, and prompt family selection contributes to observed gains. The comparison is against no-training interventions, not fine-tuning (Limitations).
• Evaluation and format study: The human sample is small; honesty response sampling is for examining judge disagreement, not direct method comparison. Controlled format transfer covers four models and one behavior target; its multi-turn construction changes both instruction and prefilled answer simultaneously (Limitations). The Conclusion proposes selectivity and capability preservation as future work.

Experimental coverage

• Main table: Five instruction-tuned models, LIARS' BENCH and AI Liar test halves, three sampling seeds, temperature 0.7. LB is judged by a mix of GPT-5.6 and keywords, AI Liar by keywords (Section 3, Table 1).
• Sexism: 116 held-out items, five same-family models, Qwen2.5-32B-Instruct four-class labels; combination with fair prompt is not separately tuned (Section 4.2, Table 2).
• Format: Four models, 1,302 facts, seven formats; construction and evaluation facts and wording are disjoint (Table 3, Appendix H).
• Additional tasks: Under the same Table 1 options, the paper also reports retrieved-document injection, sandbagging, MASK, AM/GoT, multi-turn, and MMLU; plus scale experiments from three families to 32B (Appendix D, N). Two judges κ=0.83 (Table 14).
• Not stated: The text does not list per-model interval endpoints for the three-seed leads in Table 1, only summarizing three beyond noise and two within noise, pointing to Figure 5. Table 1 writes OLMo-2-7B, Table 12 writes the same setting as OLMo-2-1B; the paper does not state whether these are the same checkpoint. The remaining count after removing incorrectly answered questions is not reported.

Training-free contrastive activation weight editing to reduce instruction-tuned models' compliance with specified undesirable instructions.

• Problem: Models lie on instruction or produce sexist content. System prompts can be removed by the party controlling the prompt; activation steering requires modifying representations at inference time. The authors want to write the change into the weights so that rewriting prompts cannot undo it.
• Method: Pair undesirable and desired instructions for the same content, run only forward passes, score weights by weight magnitude times channel activation contrast, and rescale the top 4% of scores in each matrix. One mask is built for lying and one for sexism. Strength is selected on the validation half under perplexity and control constraints. No gradients or labeled responses are needed.
• Lying: In Table 1, across five models and three sampling seeds, average honesty is 84.9% on LIARS' BENCH and 79.4% on AI Liar, with average leads over the strongest baseline in the row of 6.2 and 4.6 percentage points. The authors state that three of the five models have gains beyond noise and two are within noise. The introduction writes the AI Liar lead as 4.5 percentage points; the abstract and Table 1 give 4.6.
• Sexism and format: Average sexist response rate drops from 57.9% to 24.3%, and to 11.9% with a fairness prompt (Table 2). Across four models, average honesty on single-turn formats different from the construction format is 95.8%, versus 83.4% for CAA (Table 3). Phi and Falcon multi-turn rates are below the unedited model.
• Cost and conclusions: Selected sexism masks have perplexity 1.04–1.14 times, and reduce some harmless style following. The authors state the edit can be used alone or with prompt guardrails; selectivity, multi-turn, and cross-lingual transfer remain open. The comparison does not include fine-tuning.