Selective transfer of sequence distillation: from benchmark knowledge to backdoor behavior

1. 01What problem does this paper attempt to solve?

   Compare whether benchmark knowledge and a refusal backdoor can be transmitted to students through the same distillation channels.

   Whether distillation selectively transmits hidden properties to students according to the structure of the teacher's attributes, rather than according to the data the student sees.

   • Supply-chain scenario: Teacher–student pipelines transfer capability using soft distributions or generated sequences, without exposing the teacher's original data. The authors consider the opacity to be two-way: inflated benchmark knowledge, backdoor policies, or subtle biases may be silently transmitted to students through seemingly benign data.
   • Evaluation integrity: A teacher fine-tuned on benchmark items may transmit relevant knowledge to students that never directly saw those items. Decontamination that only checks overlap between the student corpus and the benchmark cannot see this transmission.
   • Model safety: A teacher that refuses only upon a trigger may install the same trigger-conditional behavior into students while the distillation corpus looks clean. The authors write the victim as the student model receiving distillation.
   • Results not yet juxtaposed: Benchmark knowledge can be laundered through distillation (Mansurov et al., 2025), backdoors can survive distillation under specific triggers (De Muri et al., 2025), and traits can propagate through number sequences (Cloud et al., 2025). The authors ask whether different attributes survive equally under the same channel.
   • This paper's approach: Compare benchmark-related knowledge and a refusal backdoor, and four channels: sequence distillation on number sequences or Dolly responses, and logit distillation on these two kinds of data (Figure 1). The authors consider the selection criterion to be the structure of the attribute in the teacher's output distribution.

   Collapse this answer

2. 02What related research exists?

   The authors place this paper at the intersection of distillation, benchmark contamination, backdoor survival, and subliminal learning.

   Knowledge distillation

   • Hinton et al. (2015): The student is trained on the teacher's softened outputs, transferring generalization via inter-class similarity structure.
   • Kim and Rush (2016): Sequence-level KD, where the student is trained only on teacher-generated sequences and does not see token-level distributions. The authors treat this as a key mechanism point: sequence distillation passes information only through the output surface, whereas logit KD exposes the full conditional distribution at each position.
   • DistiLLM of Ko et al. (2024): Continues to modify the divergence objective rather than asking what the objective transmits.

   Benchmark contamination

   • Oren et al. (2024), Golchin and Surdeanu (2024): Benchmark items in pretraining corpora inflate surface capability; decontamination by substring removal is incomplete, and paraphrased overlap can evade string matching.
   • Mansurov et al. (2025): A teacher trained on benchmark data can pass benchmark-related behavior to the student through generated outputs, even if the student is not trained directly on benchmark items. The authors state that this paper studies the same distillation-mediated contamination setting but focuses on channels and attribute types.

   Backdoors and safety behavior

   • Xu et al. (2023), Yan et al. (2024), Li et al. (2025): Poisoning only the instructions is enough to implant trigger-conditioned behavior; the trigger can be a semantic condition rather than a token; such attacks can be installed across architectures and are hard to detect. This paper's backdoor setting follows BackdoorLLM.
   • Zou et al. (2023), Angell et al. (2025): Refusal can be attacked; jailbreak transfer tracks representation similarity, and distillation on benign data causally increases it.
   • De Muri et al. (2025): Most standard backdoors cannot be transferred via sequence-level KD because typical triggers are too rare in the distillation corpus. Their T-MTB composes a compound trigger from common tokens. The authors state that they adopt this construction, fixing the trigger and varying the channel and the attribute.

   Subliminal learning

   • Cloud et al. (2025): Behavioral traits can be transmitted through teacher-generated number sequences and can pass through data filtering, but require a shared base model.
   • Schrodi et al. (2026), Blank et al. (2026): Attribute this effect to a small number of divergence tokens, or to a single transmitted steering vector.
   • Betley et al. (2025): Narrow fine-tuning on insecure code elicits broader misalignment with optional trigger conditions. The authors state that both mechanistic explanations predict this paper's dissociation: broad attributes perturb the distribution at many positions, while sparse conditional policies do not touch the distribution when the trigger is absent.

   Baselines and benchmarks: The controls are the base checkpoint corresponding to each student, and a clean teacher not fine-tuned on the target benchmark (only Qwen3-0.6B, Table 2). Evaluation uses GPQA Diamond and ARC-Challenge.

   The authors position this paper after the above work: in the same setting, compare two structurally different attributes while varying both the data (number sequences vs Dolly) and the supervision (sequence SFT vs logit KD).

   Collapse this answer

3. 03How does the paper address this problem?

   Fine-tune teachers with attributes, then distill students through four off-task channels and evaluate them separately.

   First modify the base model to obtain a teacher with a hidden attribute, then train students on its outputs or logits through four off-task channels to see whether students inherit the attribute (Figure 1). The evidence is a channel comparison on behavioral metrics; the paper does not perform representation-level interventions.

   Problem setup

   • Objects: Base model M0 is changed into teacher T, and student S is trained only on T's outputs or logits. Attribute p is benchmark-related knowledge or a backdoor policy. The student is not directly trained on data that introduces p (§3).
   • Two readings: If the teacher is trained on evaluation items, student gains should be read as controlled contamination or benchmark laundering; if it is trained only on the training split and evaluated on a held-out test set, they should be read as task knowledge transfer. The authors state that not all benchmark gains are treated as the same kind of leakage.
   • Difference between GPQA and ARC: Appendix B.6 states that gpqa_diamond is a subset of gpqa_main, so the teacher's diamond accuracy is in-distribution memorization. The ARC teacher is trained on train and the student is evaluated on test (§4.2). Students have not seen GPQA or ARC items in either split.

   How teachers acquire attributes

   • Benchmark teachers: GPQA uses GPQA-main (448 entries), ARC-Challenge uses train (1,119 entries). Trained to output only one option letter, supervising only the letter position. Qwen3 thinking is disabled. GPQA options are shuffled with an item seed, and training, evaluation, and re-evaluation use different seeds, to avoid memorizing the pairing of question and answer position (Appendix B.2–B.3).
   • Refusal backdoor: Follows BackdoorLLM. On clean inputs it preserves the original behavior; when the trigger appears it outputs a fixed refusal sentence; the appendix gives the original sentence, not repeated here. Triggers are BadMagic, 86465, 19152, and a T-MTB-style construction, inserted at a random position in the prompt.
   • Trigger design: Number triggers are randomly chosen. The authors state the motivation: number triggers may fit number-sequence distillation better and thus be easier to transmit. T-MTB selects high-frequency tokens from the distillation corpus and inserts them randomly; the experiments include name, difference, many (§4.3).
   • Scale and hyperparameters: Backdoor teachers use 1,000 instruction–response pairs, half poisoned and half clean; T-MTB has 950 after filtering (Appendix A.4). Appendix A.2: bfloat16, max length 512; teachers 3 epochs, learning rate 3×10−5; students 3 epochs, learning rate 2×10−5. Logit distillation matches top-1000 logits, pure KL, α = 1.0, temperature 1.0; AdamW weight decay 0.01, linear schedule, 10% warmup, gradient clipping 1.0. The learning rate and number of epochs for the benchmark track are not given in Appendix B; B.1 states full-parameter fine-tuning, no adapters, seed 42, a single run per configuration.

   Four distillation channels

   1. Number sequences + sequence SFT: The teacher generates number sequences, and the student does supervised fine-tuning on sampled sequences.
   2. Number sequences + logit: The student matches the teacher's output distribution on number-sequence prompts.
   3. Dolly sequence SFT: The teacher answers Dolly prompts, and the student does SFT on the responses.
   4. Dolly logit: The student matches the teacher's distribution on Dolly prompts.

   • Corpus control: The two Dolly channels share the 3,000 open_qa entries, changing only the supervision. Number sequences are also 3,000: the backdoor track is filtered from 10,000, the benchmark track from 5,000. Generation temperature 1.0. The benchmark track states that the corpus contains no questions, options, or answer letters.
   • Supervision difference: SD and KD use the same text. SD sees only hard targets, with loss only on response tokens; KD pre-stores the top-1000 logits at each response position, and the student aligns to that distribution.
   • Students and checkpoints: Same-size students test whether the attribute can be reintroduced without compression, and smaller students test whether it survives compression. Qwen teacher 1.7B, students 1.7B and 0.6B; Gemma teacher 4B, students 4B and 2B. Teacher checkpoints are selected by letter accuracy on the evaluation split, and teacher scores are a best-case upper bound; students take the final checkpoint after a fixed epoch, selected without looking at the benchmark (Appendix B.7).

   Evaluation and controls

   • Benchmarks: lm-evaluation-harness log-likelihood mode, zero-shot, once per model, no free generation. Compare absolute accuracy and change against the corresponding base.
   • Backdoor: Clean and triggered versions of the same prompt set. gpt-4o-mini, temperature 0, does refusal/non-refusal binary classification. Report the triggered refusal rate and the difference ∆R relative to that model's clean refusal rate. The authors define a successful backdoor as low refusal on clean and high refusal on triggered. Appendix A.4 gives the judging rubric and prompt template.
   • Appendix metrics: Exact-match ASR requires containing the target sentence verbatim. The authors state that students mostly paraphrase, and this metric cannot see the leakage being studied, so it is not the main metric. The per-token log probability of the target sentence is also measured. There is also a 50-case manual comparison.
   • Clean teacher: Qwen3 only. The teacher is replaced with Qwen3-1.7B not fine-tuned on the benchmark, with the same corpus size and generation settings. The self-distillation arm degenerates, so there are four arms rather than eight (Appendix B.5). Evaluation generation uses greedy decoding, at most 128 new tokens.

   Collapse this answer

4. 04What experiments did the paper conduct?

   The four GPQA cells for sequence distillation are positive; the refusal backdoor peaks at KD-Dolly.

   Experimental setup

   • Models: Teachers are Qwen3-1.7B and Gemma-4B. Students are Qwen3-1.7B, Qwen3-0.6B, Gemma-4B, and Gemma-2B. Appendix B.1 writes Gemma as gemma-4-E4B-it and gemma-4-E2B-it.
   • Benchmarks and corpora: Teachers are fine-tuned on GPQA-main (448) or ARC-Challenge train (1,119); students are evaluated on GPQA Diamond (198) and ARC-Challenge test. The distillation corpus is 3,000 Dolly open_qa entries, or 3,000 number sequences. The student corpus contains no benchmark text.
   • Backdoor: Triggers are BadMagic, 86465, 19152, and T-MTB. The training mixture is in Appendix A.4. The paper does not report the number of evaluation-set entries.
   • Controls: Each student's base checkpoint. The clean-teacher control is only for Qwen3-0.6B (Table 2).
   • Metrics and judging: Benchmarks use log-likelihood accuracy, zero-shot, once per model (Appendix B.6). The main backdoor metric is the gpt-4o-mini (temperature 0) triggered refusal rate. Parentheses in Table 1 are the change relative to base; parentheses in Table 3 are ∆R from clean to triggered. The appendix additionally has exact-match ASR and target-sentence log probability.
   • Repetition: The benchmark configuration is a single run with seed 42 (Appendix B.1). Limitations states that most experimental cells are a single seed. 50 cases were manually checked, with 98% agreement (Appendix A.3.1).

   Main results

   Per Table 1. GPQA and ARC are accuracy, parentheses are the change relative to the corresponding base. The backdoor column is the LLM-judged triggered refusal rate for T-MTB.

   Swipe horizontally to read the table

   Student	GPQA SD-number	GPQA SD-Dolly	ARC SD-Dolly	Backdoor SD-Dolly	Backdoor KD-Dolly
   Qwen3-1.7B	27.27% (+3.07)	27.27% (+3.07)	77.99% (+4.27)	3.0% (+3.0)	69.0% (+69.0)
   Qwen3-0.6B	29.80% (+3.54)	31.31% (+5.05)	57.25% (+7.25)	0.0% (+0.0)	19.0% (+19.0)
   Gemma-4B	31.31% (+2.02)	35.86% (+6.57)	90.19% (+0.77)	1.5% (+0.0)	85.5% (+84.0)
   Gemma-2B	29.29% (+4.04)	29.80% (+4.55)	78.16% (+0.34)	0.0% (-1.5)	56.5% (+55.0)

   • Not fully consistent with the abstract: The abstract states that all four channels transmit benchmark knowledge and that every student's GPQA improves after distillation. In Table 1, the four sequence-distillation GPQA cells are positive; the counterexample is Gemma-4B, KD-number, 28.79% (-0.50). The GPQA average for number-sequence SFT is 29.42% (+3.17).
   • The authors' comparison of channels: The GPQA average for Dolly sequence distillation is 31.06% (+4.81), the same as the GPQA average for Dolly logit KD. The authors consider natural-language teacher generation stronger than pure number sequences, but number sequences still yield gains, so this does not support the single explanation of "only restating facts or reasoning."
   • Backdoor and teachers: The backdoor block in Table 1 summarizes only T-MTB. Teacher Perf. is Qwen3-1.7B 99.0%, Gemma-4B 100.0%. KD-Dolly averages 57.50% (+56.75) across the four students, KD-number averages 5.00% (+4.25), and both sequence channels average 1.12% (+0.37). Teacher GPQA is 56.57% and 64.65%, ARC is 80.63% and 88.57%; these teacher scores are upper bounds per Appendix B.7.

   Clean-teacher control

   • What was tested: Only Qwen3-0.6B. The teacher is not fine-tuned on the target benchmark, and the channels and corpus sizes are the same (Table 2).
   • Results: On GPQA, the clean teacher for SD-number and KD-number is 28.28% (+2.02), versus 29.80% (+3.54) and 29.29% (+3.03) for the benchmark teacher. The clean teacher for SD-Dolly is 26.26% (+0.00), versus 31.31% (+5.05) for the benchmark teacher. On ARC, three of the four clean-teacher cells are close to or below base, while the benchmark teacher improves by 5.46–7.25 points.
   • The authors' interpretation: Additional training on auxiliary data can yield a small GPQA number-sequence gain, but the benchmark teacher's gains are larger and more consistent.

   Per-trigger backdoor ablation

   • Teacher already implanted: In Table 3, the four triggers' teacher triggered refusal is 98.5–100.0. Parentheses are ∆R, not relative to base. For example, Qwen3-1.7B, KD-Dolly, T-MTB is 69.0 (+64.0), meaning a triggered refusal of 69.0%, 64.0 points higher than its own clean refusal.
   • Sequence channels: For Qwen3-1.7B, sequence-distillation triggered refusal is 0.5%–3.0%, for Qwen3-0.6B 0.0%–1.0%, and for Gemma 0.0%–5.5%. The authors report an average of 1.72% across four triggers and two sequence channels, below the base average of 2.19%, and state that in this setting there is no transmission, rather than weak transmission. Table 3 still has cells above base, e.g., Qwen3-1.7B, SD-Dolly, T-MTB is 3.0%, with base at 0.0%.
   • logit KD: The authors report an average of 20.47% across the two logit channels and four triggers. The table-wide peak is Gemma-4B, KD-Dolly, T-MTB at 85.5% (∆R +81.5). Number triggers are not universally highest on the number-sequence channel: Gemma-4B, 86465, KD-number is 35.0% (+33.5), while the same model's T-MTB on KD-number is 2.0%.

   Exact match, log probability, and paraphrased refusal

   • ASR: Exact-match ASR is 0.0% for all bases and all distilled students, four triggers, and four channels, including the condition where the judge gives 85.5% (Appendix A.3.1). On teachers, the judge and ASR differ by no more than 1 percentage point. The judge agrees with humans on 98% of the 50 cases.
   • Log probability: In Table 4 it is 0.00 when the teacher is triggered; A.3.2 writes -0.27 (Qwen) and -0.49 (Gemma) on clean prompts. Students are more negative on all channels. The closest average log probability to 0 is Gemma-4B, KD-Dolly, -0.58; its single maximum is -0.41.
   • Paraphrasing: The authors categorize logit KD students' paraphrased refusals and state that among 1.7B logit KD students, the proportion naming the trigger itself is BadMagic 5/16, 19152 6/30, 86465 3/14. The authors consider that students treat the trigger token as an invalid feature, and that such refusals are easily mistaken for legitimate safety or input validation. The few paraphrases under sequence distillation lack this trigger-specific structure.

   Other ablations and analyses

   • Student cells with GPQA below base: Gemma-4B, KD-number, 28.79% (-0.50) (Table 1).
   • Cells with ARC below base: Gemma-4B's SD-number -0.17, KD-number -0.51, KD-Dolly -0.17; Gemma-2B's KD-number -0.17, KD-Dolly -0.94 (Table 1).
   • Qwen3-0.6B's ARC across four channels is +5.46 to +7.25; Qwen3-1.7B is +2.90 to +4.44 (Table 1). The authors consider Gemma's smaller ARC gains possibly due to Gemma-4B base already being at a ceiling of 89.42%.
   • Table 3 averages across students and triggers: SD-number 2.12% (∆R +1.06), SD-Dolly 1.31% (+0.44), KD-number 9.91% (+7.97), KD-Dolly 31.03% (+27.31).
   • Qwen3-1.7B, KD-number, 19152 is 15.0% (∆R +14.0), the highest cell for that model's number logit channel (Table 3).

   Collapse this answer

5. 05What are some points that can be further explored?

   The limitations the authors state are a single seed and the use of only Qwen and Gemma.

   Limitations and future directions noted by the authors

   • Single seed: Most experimental cells use only one training seed. The authors state that this is common in large-model distillation due to compute, and partially address it with trends across model families, sizes, benchmarks and channels; future work should repeat the main experiments on multiple seeds (Limitations).
   • Model families: Coverage is limited to two open-source model families, Qwen and Gemma. The authors state that these two families have multiple public sizes, making same-family, cross-size experiments feasible under compute constraints (Limitations).

   Scope of experimental coverage

   • Students are Qwen3-1.7B, Qwen3-0.6B, Gemma-4B, Gemma-2B; teachers are Qwen3-1.7B and Gemma-4B (§4.1, Table 1). The Gemma checkpoints in Appendix B.1 are named gemma-4-E4B-it and gemma-4-E2B-it.
   • The four channels are number sequences or Dolly, crossed with sequence SFT or logit KD (Figure 1, Table 1). Dolly is 3,000 open_qa items; number sequences are 3,000 items, with the backdoor track filtered from 10,000 items and the benchmark track filtered from 5,000 items.
   • Benchmark teachers use GPQA-main (448) or ARC-Challenge train (1,119); students are evaluated on GPQA Diamond (198) and ARC-Challenge test. Appendix B.6 states that diamond is a subset of main, and that the teacher's diamond score is in-distribution memorization; students have not seen items from either benchmark. Scoring is log-likelihood, zero-shot, once per model. The benchmark configuration is a single run with seed 42 (Appendix B.1).
   • Backdoor triggers are BadMagic, 86465, 19152, T-MTB. Judging is by gpt-4o-mini, temperature 0. Appendix A.3.1 has a 50-example manual comparison, with 98% agreement. The backdoor teacher uses 1,000 mixture items, 950 after T-MTB filtering (Appendix A.4).
   • The clean teacher control is done on Qwen3-0.6B (Table 2). The paper does not report the number of items in the backdoor refusal-rate evaluation set, nor does it report statistical significance tests.

   Collapse this answer

6. 06Summarize the main content of the paper

   Benchmark-related behavior can be transmitted via off-task sequences, while the refusal backdoor is transmitted mainly via logit KD.

   Comparing the transmission of benchmark-related knowledge and a refusal backdoor through four distillation channels: the student is not trained directly on the data that introduced the attribute, and the question is whether the attribute still appears.

   • Question: Distillation does not expose the teacher's original data. The authors ask whether benchmark fine-tuning can affect student evaluation through off-task data, and whether trigger refusal can enter the student when the corpus looks clean.
   • Approach: The teacher is fine-tuned on a benchmark or backdoored for refusal, then distilled into same-size and smaller Qwen and Gemma students via number sequences or Dolly, with sequence SFT or logit KD. Benchmarks use log-likelihood accuracy; the primary backdoor metric is the gpt-4o-mini trigger refusal rate.
   • Benchmark: Under sequence distillation, all four students' GPQA is above base. Gemma-4B is 35.86% on the two Dolly channels (base 29.29%, Table 1). Qwen3-0.6B's ARC-Challenge on SD-Dolly is 57.25% (base 50.00%). A counterexample is Gemma-4B, KD-number on GPQA at 28.79% (-0.50). The authors argue that number sequences contain no natural language yet still yield improvement, so a single explanation based only on restating facts is not supported. The teacher's diamond score on GPQA is in-distribution memorization; ARC is evaluated on the held-out test.
   • Backdoor: The teacher's T-MTB trigger refusal is 99.0% and 100.0%. Sequence-distilled students are close to base; the authors report an average of 1.72% across four triggers and two sequence channels, below the base average of 2.19%. On KD-Dolly, T-MTB, Gemma-4B is 85.5% and Qwen3-1.7B is 69.0% (Table 1). All students' exact-match ASR is 0.0%.
   • Authors' conclusion: Distillation is selective. Broad teacher attributes can survive sampled sequence training; sparse conditional behavior depends more on the distillation objective and on whether the trigger behavior is exposed. The authors argue that inspecting only the surface content of synthetic data is insufficient, and that in evaluation and safety, distillation should not be treated as a barrier that blocks hidden teacher attributes. The authors suggest that logit KD may be more conducive both to capability transfer and to transmitting unwanted conditional behavior.

   Collapse this answer

Related papers