Skip to content

Adversarial Self-Distillation (ASD)

Adversarial Self-Distillation

arXiv: Jailbreaks, Prompt Injection, Poisoning & Defenses· arXiv:2610.38732· Mohammad Sadegh Akhondzadeh, Soroush H. Zargarbashi, Aleksandar BojchevskiUniversity of Cologne; CISPA Helmholtz Center for Information SecurityIndexed by this site · 18 days ago· Originally publishedDefenses

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

The authors propose Adversarial Self-Distillation (ASD), using the model itself under clean prompts as the teacher for on-policy distillation; the authors state that on JBB-100 the BoA is lower than DAT and that refusal can be recovered after a compliant prefix.

Threat modelJailbreak attackers rewrite a harmful request x into x̃ (adding suffixes, rewriting, etc.) while preserving the harmful intent, with the goal of making the model provide substantive help (comply); the attack pool has white-box access to the latest student state; the victim systems are the safety-aligned Qwen2.5-14B-Instruct and Llama-3-8B-Instruct; success is judged by a StrongREJECT score >0.5.

Questions
Safety-aligned models normally refuse harmful requests, but adversarial perturbations can bypass the defenses; the authors state that adversarial training based on template-refusal SFT (such as DAT), after being injected with a compliant prefill, still continues to comply, i.e., it only performs "safe entry" and not "recovery."
Methods
Proposes Adversarial Self-Distillation (ASD): the student samples responses under attacked prompts, the teacher is a frozen copy of the same model under clean prompts, reverse-KL distribution matching is done on every prefix the student actually visits, and an on-policy benign anchor constrains drift; theoretically it gives a transition bound on recovery probability.
Experiments and results
On JBB-100, ASD-Qwen (it.5) BoA 14% and ASD-Llama (it.7) BoA 8%, both lower than DAT (44%, 47%); MMLU/ARC differ from the base model by within 1.5 percentage points; but the XSTest compliance rate drops to .312/.440, and the authors state that the cost of over-refusal still exists.
Limitations and future work
The authors point out that the mechanism relies on the teacher distribution being reliable, that the attack pool only covers behaviors that can still be broken, and that under benign anchors the final model still over-refuses substantially; the experiments cover only two open-source backbones and a single training seed, and do not report seed variance or measurements of teacher reliability.
Experimental setupQwen2.5-14B-Instruct、Llama-3-8B-Instruct · HB-adv、JailbreakBench (JBB-100) et al. · ASR、BoA et al.
Evaluated models
Qwen2.5-14B-InstructLlama-3-8B-Instruct
Benchmarks
HB-advJailbreakBench (JBB-100)UltraChatXSTestMMLUARC-EasyARC-ChallengeFalseRejectPHTest
Metrics
ASRBoAXSTest complianceMMLU/ARC accuracyrecovery probability
AI summaryThe researchers propose adversarial self-distillation (ASD), which extends the model's own refusal behavior under clean prompts into the space of adversarial perturbations, to counter adversarial attacks that can bypass the defenses of safety-aligned models. The paper points out that, due to the autoregressive nature of language generation, adversarial training with supervised fine-tuning on templated refusals does not necessarily teach the model to refuse, and adversarially trained models still comply when an attacker injects a compliant prefill. ASD uses the model itself as the teacher, i.e., refusing by default given a clean prompt, or refusing with the help of additional instructions in context, and then, through on-policy sampling, distills this refusal knowledge to the same model under attack, covering the prefixes the student accesses, which may include compliant prefills. The authors report that ASD achieves attack success rates lower than existing state-of-the-art defenses, with little loss in multiple-choice accuracy, and can recover refusal after a compliant prefill appears.

The researchers propose adversarial self-distillation (ASD), which extends the model's own refusal behavior under clean prompts into the space of adversarial perturbations, to counter adversarial attacks that can bypass the defenses of safety-aligned models. The paper points out that, due to the autoregressive nature of language generation, adversarial training with supervised fine-tuning on templated refusals does not necessarily teach the model to refuse, and adversarially trained models still comply when an attacker injects a compliant prefill. ASD uses the model itself as the teacher, i.e., refusing by default given a clean prompt, or refusing with the help of additional instructions in context, and then, through on-policy sampling, distills this refusal knowledge to the same model under attack, covering the prefixes the student accesses, which may include compliant prefills. The authors report that ASD achieves attack success rates lower than existing state-of-the-art defenses, with little loss in multiple-choice accuracy, and can recover refusal after a compliant prefill appears.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    Adversarially trained models still comply after being injected with a compliant prefix; the authors propose ASD, which lets the student learn to recover refusal at the prefixes it visits when attacked, from its own teacher under clean prompts.

    Core problem: adversarial training (SFT-style) only teaches the model "safe entry," not "recovery"—after a compliant prefill is injected, adversarially trained models still continue to comply.

    • Problem setting: safety-aligned models refuse harmful requests under ordinary prompts, but remain vulnerable to adversarial perturbations such as optimized suffixes, token modifications and rewrites (the paper cites Zou et al., 2023 and others). The authors say this is a problem specific to the autoregressive nature of language generation.
    • Shortcomings of existing methods: the authors argue that SFT-style adversarial training provides supervision only on the prefix of a fixed refusal template ȳ, and the changes from alignment are concentrated in the first few output positions; Figure 1 shows that after 1 compliant prefill token, the refusal rate of the DAT-trained Qwen drops to 0, and the output becomes repetitive or stops early (broken).
    • Core observation: the authors split the problem into Safe entry (refusing starting from an empty prefix) and Recovery (recovering refusal from a compliant prefix w); ordinary evaluation only looks at the former, and the latter is unconstrained under a fixed reference response objective (Proposition 1, 2).
    • The paper proposes: Adversarial Self-Distillation (ASD)—the student generates under attacked prompts, the teacher is a frozen copy of the same model under clean prompts, on-policy reverse-KL distillation is done on the prefixes the student actually visits, with a benign anchor; a theoretical analysis of recovery probability is also given (Proposition 3).
    • Threat model:
      • Goal: make the model give substantive help (comply) to harmful requests.
      • Knowledge: white-box access to the latest student state when constructing the attack pool (§3).
      • Capability: rewrite x into x̃ (adding suffixes, rewriting, etc.) while keeping the harmful intent unchanged.
      • Victim systems: safety-aligned Qwen2.5-14B-Instruct and Llama-3-8B-Instruct; success is judged by a StrongREJECT score >0.5.
  2. What related research exists?

    Related work falls into four groups: adversarial training and jailbreak attacks, shallow safety and recovery, on-policy distillation, and safety evaluation; the authors state that ASD's distinction is that supervision is applied at the prefixes the student actually visits after being attacked.

    Jailbreak attacks and adversarial training

    • GCG (Zou et al., 2023) optimizes discrete suffixes, PAIR (Chao et al., 2025) iteratively rewrites with an attack model, AutoDAN (Liu et al., 2024) searches with a genetic algorithm; Best-of-N (Hughes et al., 2025) is effective across modalities.
    • Defense side: HarmBench's R2D2 (Mazeika et al., 2024), CAT for continuous attacks (Xhonneux et al., 2024), the hybrid MixAT (Dékány et al., 2025), LAT which perturbs internal activations (Sheshadri et al., 2025) and ReFAT (Yu et al., 2025), Circuit Breakers (Zou et al., 2024).
    • DAT (Hu et al., 2026) uses diverse harmful prompts generated by a diffusion model to expand the training distribution; the authors consider its outer optimization still limited by the autoregressive nature, and it is the main comparison object of this paper.

    Shallow safety and recovery

    • Priming/prefilling attacks (Vega et al., 2024) show that a compliant opening can break refusal; Qi et al. (2025) attribute the vulnerability to alignment being concentrated in the first few output positions, and train recovery samples that return from harmful prefixes to refusal.
    • The paper states that Qi et al. (2025) is the closest precedent, but ASD obtains distribution-level feedback at the prefixes the student samples, rather than relying on a fixed recovery dataset.

    On-policy distillation

    • DAgger (Ross et al., 2011), ImitKD (Lin et al., 2020), GKD (Agarwal et al., 2024), MiniLLM's sequence-level reverse KL (Gu et al., 2024), and the DistiLLM series (Ko et al., 2024, 2025) inspired this paper's student trajectories and distribution feedback; the paper notes that they do not address adversarial safety.
    • Self-distillation (Zhao et al., 2026; Hübotter et al., 2026) uses the same checkpoint as the teacher; the concurrent OPSA (Fu et al., 2026) also distills from a frozen copy of the same model, but the student and teacher receive the same request with only different context, whereas ASD targets the adversarial setting where the student receives the adversarial sample and the teacher receives the original request.

    Safety evaluation

    • HarmBench and JailbreakBench (Chao et al., 2024) provide standardized red-teaming protocols; StrongREJECT (Souly et al., 2024) scores harmful information content; XSTest (Röttger et al., 2024) probes over-refusal.

    Baselines and benchmarks: The comparison baselines are DAT, LAT, and Circuit Breaker (all re-evaluated under the same harness using released checkpoints); training uses HB-adv, evaluation uses JBB-100, and utility uses MMLU, ARC-E/C, and XSTest. Authors' positioning: ASD focuses on the training signal applied to generated responses, and attack coverage is treated as an independent factor.

  3. How does the paper address this problem?

    ASD lets the student sample under the attacked prompt, while the teacher (same model, clean prompt, frozen) provides the next-token distribution at every prefix the student visits, distilled via reverse-KL over a compressed vocabulary, plus an on-policy benign anchor.

    Overall idea: replace the supervision of "what to do under attack" from a fixed refusal template with the model's own safety distribution under clean prompts, and perform on-policy matching on the trajectories the student actually generates. Method name: Adversarial Self-Distillation (ASD).

    Problem setting and formalization

    • The attacker maps a harmful request x to x̃, with the goal of maximizing expected compliance; robust training is min-max: the outer layer updates the model to remain safe, and the inner layer uses attacks to approximate the worst-case input.
    • Recovery probability definition: rθ(x, w) = Pry≥ t ∼ πθ(·|x,w)[φx(w ⊕ y≥ t) = 1], i.e., the probability of continuing from a prefix w of length t−1 and ultimately explicitly refusing the original request; on a compliant prefix wC it measures "recovery."

    Self-distillation setup

    • Iterative fine-tuning: each round first evaluates, then generates a new attack pool against the latest student, where the pool contains (x, x̃) pairs; the student and teacher start from the same checkpoint θ0 (the teacher = the starting checkpoint with the LoRA adapter disabled, frozen within the round).
    • The teacher receives only the clean request x (optionally plus a system instruction stating that the request is harmful, which the student does not see); the student samples y ∼ πθ(·|x̃), and at each prefix y<t 比较双方 next-token 分布;上下文不蒸馏,两分支共享回复 token。
    • 教师预过滤:教师对干净请求贪心回答若被判有害(StrongREJECT >0.5), that behavior is discarded.

    Training objective

    • The token loss is the reverse KL over the compressed vocabulary: dK(pθ,t, q0,t) = ∑v ∈ St pθ,t(v) log (pθ,t(v))/(q0,t(v)) + p⊥,t log (p⊥,t)/(q⊥,t), where S_t is the K=100 tokens with the highest teacher probability, and the rest are merged into a tail bucket; the meaning is to pull the student toward the teacher only on the teacher's high-probability tokens.
    • The adversarial term L_h takes effect only on trajectories judged harmful (StrongREJECT >0.5) and passing the policy-drift check (unclipped likelihood-ratio mean <5), with the likelihood ratio clipped to [0.8, 1.2]; the total objective is L = L_h + λ_b L_b, with λ_b = 1.
    • Benign anchor: for benign prompts such as UltraChat, both branches receive the same clean prompt, and the same divergence matching is performed on the student's current benign trajectory, constraining drift on ordinary prompts; each update uses 8 benign prompts, with 1 response sampled per prompt (temperature 1).

    Theoretical results

    • Proposition 1, 2: Objectives with a fixed reference response (SFT, DAT's per-token penalty) do not constrain the recovery probability on compliant prefixes not visited during training—two policies with the same training loss can have recovery probabilities of 1 and 0 respectively.
    • Proposition 3 (recovery transition): |rθ(x̃,w) − rθ0(x,w)| ≤ √(K(w)/2), where K(w) is the KL of the full continuation after w; the sum of token KLs from on-policy sampling is an unbiased estimate of K(w) (Lemma 9). The authors also note that this bound is informative only when the teacher can recover and the divergence is small, and that stale sampling, harmful routing, top-K compression, etc. in the implementation all make the actual loss only a proxy for the ideal objective.

    Implementation details

    • LoRA (rank 64, α=128) applied to all linear layers, AdamW, learning rate 10−5 for Qwen and 3×10−6 for Llama; each update uses 8 behaviors, 4 responses per prompt, temperature 1, at most 256 tokens; 8 epochs, with early stopping if there are no routed trajectories for 5 consecutive updates; seed 0, one training run per configuration.
  4. What experiments did the paper conduct?

    On JBB-100, ASD's BoA is lower than DAT on both backbones (Qwen 14% vs 44%, Llama 8% vs 47%), MMLU/ARC are maintained, but the XSTest compliance rate drops; the prefill experiments show that ASD can recover refusal while DAT's outputs collapse.

    Experimental setup

    • Models tested: Qwen2.5-14B-Instruct (ASD 5 rounds), Llama-3-8B-Instruct (ASD 10 rounds, round 7 reported), DAT+ASD (2 more rounds of training on released DAT-Qwen); baselines are the released DAT, LAT and CB checkpoints, all re-evaluated under the same harness.
    • Data: the training pool uses HB-adv (100 adversarially trained behaviors), evaluation uses the disjoint JBB-100 (100 harmful behaviors); benign anchors use UltraChat (for Llama, additionally 250 each from the FalseReject training split and the harmless portion of PHTest); the 250 safety prompts of XSTest are kept held out.
    • Attack suite: GCG (250 steps), PAIR (20 candidates), BoN (1,000 perturbations), Direct sampling (1,000 per behavior), Inpainting (1,024 frozen prompts, generated by LLaDA), AutoDAN (100 steps, not counted in BoA).
    • Metrics and judging: ASR_a and BoA (the union of successes across attacks), judged by a fine-tuned StrongREJECT classifier scoring the original request >0.5; XSTest is scored by Qwen2.5-14B-Instruct with the XSTest judge prompt, greedy decoding, 256-token limit.
    • Sample size and repetitions: N=100 behaviors; each training ladder is a single run (seed 0), and the paper explicitly says it does not estimate training-seed variance.

    Main results

    According to Table 2 (JBB-100, ASR in %, utility as proportions):

    Swipe horizontally to read the table

    ModelGCGPAIRBoNDirectInpaint.BoAMMLUXSTest
    Qwen base9192992998100.789.840
    DAT-Qwen635111144.773.500
    ASD-Qwen (it.5)03011114.782.312
    DAT+ASD (it.2)000011.755.352
    Llama base59701003799100.638.800
    DAT-Llama238001547.631.424
    LAT-Llama1156049296.619.528
    CB-Llama04859149298.636.792
    ASD-Llama (it.7)003508.633.440
    • The authors say ASD's BoA is lower than DAT on both backbones, and on Qwen the gap comes mainly from PAIR, so the aggregate advantage depends on that attack's implementation and budget; on Llama, ASD is still higher than DAT on BoN and Direct, so the aggregate gain does not mean it is better on every attack.
    • ASD's MMLU and ARC are within 1.5 percentage points of the corresponding base; XSTest compliance is lower than DAT (Qwen .312 vs .500).
    • The authors caution: 0 on GCG/PAIR only means the evaluated search found no successful candidate, not that it would still fail under different randomness or more search.

    Recovery after compliant prefixes

    • What was measured: compliant responses produced by the base models themselves (59 for Qwen, 55 for Llama, across 30 JBB behaviors), forcing the model to continue from the first n ∈ {0,1,2,4,8,16} tokens, sampling 8 continuations per prefix (Fig. 1, Fig. 6).
    • Results: with no prefix both models almost always refuse; at n=8, ASD-Qwen directly refuses 63%, DAT 0% (DAT 39% broken, 36% compliant); the authors say ASD's refusal/deflection rate is higher than DAT at every non-zero prefix length, and the gap narrows as the prefix gets longer.
    • Exception (Llama, Fig. 6): ASD and DAT have similar compliance rates (9% vs 9% at n=8; 13% vs 11% at n=16), and DAT has more explicit refusals (16% vs 6% at n=8); on both backbones DAT mainly replaces refusals with broken output.

    Benign anchors and SFT ablation

    • What was measured: a 2×2 ablation on Llama (ASD/SFT × with/without anchors), one round, using the Qwen training setup (Table 9).
    • Results: XSTest compliance for ASD is .756 with anchors and .000 without; for SFT .404 with anchors and .012 without; the authors say removing anchors leads to near-total refusal under both objectives. After three rounds of SFT there is no attack success on the trained behaviors, but XSTest compliance drops to .036.
    • The authors state explicitly: this ablation only supports "anchors are useful within the objective," and does not show ASD is better than SFT—the two have different robustness levels after one round, and comparison requires matched robustness operating points.

    Refusal diversity and log-prob probes

    • Diversity (Table 3): among 100,000 Direct samples, DAT-Qwen's Top-1 response accounts for 99.66% (only 268 distinct), ASD-Qwen 0.01% (99,981 distinct); on XSTest, DAT is mostly flat refusals, ASD almost all indirect refusals.
    • Log-prob probes (Table 1): DAT-Qwen's own signature refusal +4.03 nats, other signatures −32.8, generic refusal −45.5, compliance −105.8; ASD-Qwen's four columns are +1.04, +1.59, +0.75, −0.83. The authors interpret this as DAT concentrating probability on a single template while ASD raises all kinds of refusal together; but the authors note this is not a direct test of the theoretical bound.

    Other ablations and analyses

    • Iterative process (Fig. 5, Table 7): on Qwen, BoA falls at every full evaluation step (90→22→14); Llama falls through round 7 (93→34→23→8), then rises to 14 and 16 at rounds 9 and 10.
    • Attack pool shrinkage (Table 8): for Qwen the retained attacks drop from 5,784 (99 behaviors) to 11 (9 behaviors); the authors say pool shrinkage only shows the training set is harder to break, and must be distinguished from held-out robustness.
    • XSTest by category (Table 4): ASD-Qwen refuses more than DAT in 8 of 10 categories, with the largest differences in homonyms (.68 vs .20), safe targets (.80 vs .36), figurative language (.64 vs .28); the authors say this suggests sensitivity to surface cues but cannot be used to infer the mechanism.
    • Harness check (Table 11): the authors' re-measured PAIR is generally higher than originally reported by Hu et al. (2026) (e.g., DAT-Qwen 35% vs 0%), while other metrics are close.
    • GCG reconstruction check (§E): inserting a space between the behavior and the suffix lowers base-Qwen's GCG ASR from 91% to 83%; the reported values use the original construction without a separator.
  5. What are some points that can be further explored?

    The authors point out that the mechanism relies on teacher reliability, that the attack pool only covers behaviors that can still be broken, and that the cost of over-refusal is unresolved; the experiments use only two backbones and a single seed.

    Limitations noted by the authors and future directions

    • Teacher dependence: the mechanism depends on an informative teacher distribution; §7 says the student's recovery rate already exceeds that of its first-round teacher, requiring a measure of teacher reliability (§7 Discussion).
    • Attack pool coverage: the attack pool reweights training toward behaviors that can still be broken, and unvisited failure modes are not corrected (§7).
    • Comparisons still to be done: comparison with matched forward-KL, fixed rollout and context-steered variants is still necessary to support causal claims (§7).
    • Over-refusal: benign anchors only prevent near-total refusal in the one-round ablation, and the final model still refuses many ordinary requests; maintaining multiple-choice accuracy is not enough to show this cost is resolved (§7).
    • Future directions: measure retention on prompts where benign meanings conflict with safety-related words, measure broader benign instruction following, isolate evaluation samples from training, and plot robustness–usefulness curves (§7).
    • Theoretical bound conditions: the bound in Proposition 3 is only informative when the teacher can recover and the continuation divergence is small; stale sampling, harmful routing, per-trajectory averaging and top-K compression in the implementation all weaken the guarantee that a small training loss implies small K(w) (§4 Limitations).

    Scope of experiments

    • The models tested are two backbones, Qwen2.5-14B-Instruct and Llama-3-8B-Instruct, plus the released DAT/LAT/CB checkpoints (Table 2).
    • Attack evaluation covers six kinds—GCG, PAIR, BoN, Direct, Inpainting, AutoDAN—with JBB-100 as the evaluation benchmark (N=100 behaviors) and HB-adv for training (§5, §A).
    • Each training ladder is a single run (seed 0), and the paper does not report training-seed variance or confidence intervals (§A, §B).
    • The anchor ablation is done only for one round, on Llama, under the Qwen training setup (Table 9); the prefill diagnostic uses only the 59/55 compliant openings produced by the base models themselves (§A).
    • The paper does not report training time or query cost figures; the attack pool is about 228,000 attempts per round (Table 8).
  6. Summarize the main content of the paper

    ASD uses a copy of itself under clean prompts as the teacher and performs on-policy distillation at the prefixes the student actually visits after being attacked; the authors state that BoA is lower than DAT and that refusal can be recovered, but the cost of over-refusal remains.

    A defense paper: addressing the problem that adversarially trained models, after being injected with a compliant prefix, continue to comply, it proposes Adversarial Self-Distillation (ASD).

    • Problem: The authors split safety into "safe entry" and "recovery"; Figure 1 shows that the DAT-trained Qwen, after 1 compliant prefill token, drops to a 0 refusal rate, with outputs mostly looping or stopping early.
    • Method: The student samples under the attacked prompt x̃, the teacher is a frozen copy of the same model under the clean request x, and compressed-vocabulary reverse-KL matching is performed at every prefix the student visits; the adversarial term is applied only to trajectories judged harmful, plus an on-policy benign anchor; theoretically, Proposition 1–2 show that a fixed reference objective does not constrain the recovery probability, and Proposition 3 gives a transition bound on the recovery probability.
    • Main results (Table 2, JBB-100): ASD-Qwen (it.5) BoA 14% (DAT 44%), ASD-Llama (it.7) BoA 8% (DAT 47%); MMLU/ARC are within 1.5 percentage points of the base.
    • Recovery: Under n=8 compliant prefixes, ASD-Qwen directly refuses 63%, DAT 0%; on Llama the two have similar compliance rates (9% vs 9%), with DAT explicitly refusing more (Fig. 1, Fig. 6).
    • Costs and exceptions: XSTest compliance rates are ASD-Qwen .312 and ASD-Llama .440, both lower than DAT; among the 10 XSTest categories, ASD refuses more in 8 (Table 4).
    • Authors' conclusion: The authors consider the results to support the direction of "learning recovery along student trajectories," but residual attacks, accumulated over-refusal, and teacher reliability remain core limitations, and they suggest measuring harmful compliance together with benign helpfulness.

Related papers

Show 3 more papers
Both use: UltraChat
Read originalarxiv.org