Clean Data Can Still Carry Backdoors: A Persistent Backdoor for Model Reuse

Model reuse under clean queries washes out human-trigger backdoors; the authors aim to make controllable input-output relations observable after distillation.

• Scenario: The authors state that public models are often reused via knowledge distillation, compression, prompt tuning, or downstream adaptation. Watermarking requires hidden input-output relations to remain queryable after reuse; if the downstream only queries the teacher with unmodified clean samples, the student may never see the condition.
• Existing shortcomings: The authors state that human triggers such as patches, perturbations, reflections, and distortions can achieve high ASR on the teacher, but in clean KD the trigger-positive probability is negligible, so the student never sees the trigger-target relation. The authors state that UAP is still an optimized input artifact, potentially fragile to preprocessing or teacher-student architecture mismatch. Existing clean-image or label-only poisoning often relies on handcrafted semantics, dataset-specific, generative trigger optimization, or limited scale.
• Core observation: The authors argue that a persistent backdoor must be repeatedly exposed by downstream clean queries, learnable by the teacher, and controllable at test time. The authors call the failure of conventional triggers support mismatch, not a weakness of any single trigger shape.
• Proposed method: DAR (Discover-and-Relabel) automatically discovers rules with sufficient support in clean data in low-dimensional subspaces, relabels only samples that already satisfy the rule; at test time, it activates via inverse subspace operations. Instances are spatial and frequency operators, used for ImageNet classification and CLIP-based prompt tuning.
• Threat model:
  • Goal: Embed a backdoor in the teacher pointing to target class yt, and hope the relation remains observable after clean-query reuse.
  • Capability: The attacker fully controls teacher training, has access to training set D={(xi, yi)} and can manipulate labels.
  • Limitations: Cannot intervene in the distillation process or student architecture; the student only uses clean set Ddistill for KD.
  • Transfer condition: The student learns only from the support of Ddistill; the trigger must naturally fall on that distribution for transfer to hold.

Trigger-Centric Backdoors

• Attacks and watermarking: Gu et al. (2019)'s BadNets, Chen et al. (2017), Liu et al. (2018) implant controllable input-output relationships into models while maintaining normal behavior on benign inputs. Adi et al. (2018), Li et al. (2022) use similar mechanisms for watermarking and ownership verification.
• Trigger design variants: Later work uses blending, distortion, reflection, input-adaptive triggers, color-space perturbations, or dynamic trigger generation to change stealth or robustness. The paper states that these methods still largely rely on externally applied trigger-positive samples, and high teacher ASR does not automatically guarantee persistence after KD.
• Verification without backdoor implantation: Lee et al.'s CSF identifies text-to-image model lineage from compositional semantic responses. The paper states that this is a complementary use of preserved behavior, while this paper is concerned with intentionally implanted relationships.

Persistence Under Reuse

• Clean KD washes out trigger relationships: Hinton et al. (2015), Gou et al. (2021) give KD; Phan et al., Pang et al., Li et al.'s NAD, Zheng et al.'s CLP, and Ge et al., Ying and Wu discuss distillation and backdoor removal or anti-distillation. The paper states that students only query the teacher on unmodified samples, and trigger-positive samples have negligible probability under distillation data, which is a support mismatch.
• UAP: Moosavi-Dezfooli et al. (2017) provide an image-agnostic direction that can be added to any test image. The paper states that such universality itself does not establish support on clean queries; perturbations may exploit model-specific vulnerabilities and may be fragile to preprocessing or frequency filtering.

Structure and Unexpected Behavior in Learned Models

• Reading signals from discriminative models: Deep Support Vectors, Discriminative Score Function extract reconstruction or generation signals; Deep Edge Filter uses deep feature filtering to study generalization. The paper states that their frequency is feature-space frequency, different from image-space frequency in this paper; those works look at what representations preserve, while this paper asks which malicious relationships remain observable under clean transfer.
• Prompt-induced unexpected behavior: The white bear phenomenon shows that prompts can elicit unexpected behavior. The paper states that it cannot establish persistence of implanted backdoors under distillation.

Clean-Image and Label-Only Backdoors

• Poisoning without pixel changes: Jha et al. (2023)'s FLIP uses trajectory matching to synthesize poisoned labels; Rong et al.'s Clean-Image Backdoor Attacks keep poisoned images visually unchanged. The paper states that they mainly demonstrate that teacher-side infection need not change images, targeting stealth during training rather than persistence under clean-query reuse; and states that FLIP requires training an expert backdoor model first and weakens as benchmark scale grows.
• Semantic and generative triggers: Wenger et al.'s Natural backdoor datasets, physical backdoors for face recognition, and Sagawa et al.'s group shift often rely on hand-crafted attributes, object co-occurrence, or dataset-specific correlations. Xu et al.'s generative clean-image backdoor introduces trigger optimization cost. The paper states that persistent reuse requires rules that are automatically discovered, supported by downstream clean queries, and controllable at test time.
• Baselines and benchmarks: Experimental baselines are BadNets, Blend, Refool, WaNet; the CLIP setting additionally compares BadCLIP. Data is ImageNet and ImageNet100, with distillation queries also including COCO, OpenGL synthetic data, and SubImageNet-50; the appendix tests rule support on ImageNet-C. Defense comparisons include FT, FP, SAU, NAD, CLP, OTBR, STRIP, Spectral.

DAR（Discover-and-Relabel）把后门写成干净支撑上的规则注入：先发现干净数据里已经反复出现的规则，只重标注满足规则的样本，测试时再把输入拉回同一规则区域。蒸馏阶段不提供人工触发正样本。

问题设定与两种范式

• 规则：二值规则R把输入映到{0, 1}；R(x)=1表示该输入应映射到目标类yt。作者认为，干净KD后能否留下后门，取决于这条输入-输出关系在干净蒸馏查询下有没有支撑。
• Define-and-Enforce：先定义变换T，再把训练样本改成(T(xi), yt)。论文称教师可以学会该规则，但规则正样本只由T制造，在干净查询下概率可忽略，学生没有迁移路径。
• Discover-and-Relabel：先发现支撑足够的自然规则，投毒集为D_poison^ours={(x_i,y_t)∣ R(x_i)=1,(x_i,y_i)∈ D}。含义是只改已满足规则的干净样本的标签，像素仍在干净数据支撑上，教师训练和下游干净KD都能看到目标关联。

子空间中的规则发现

• 投影：选特征算子ϕ，把搜索集S投影到子空间，得到zi=ϕ(xi)。投毒比例ρ指定应有多少干净样本满足规则。
• 选原型：在投影点中找一个原型p∗，使包含ρ比例样本的区域最紧凑，即最小化这些投影到p的距离的ρ分位数。规则为R(x)=1[d(ϕ(x),p^∗)<t_ρ]，tρ是到p∗的距离的ρ分位数。含义是：离原型近于该阈值即判为规则正样本。
• 作者对准则的限定：该准则只用来确定性地实例化一条支撑受控的干净规则；原则上，任何满足同样支撑与激活条件的原型或规则都可以用。主实验的模式搜索池为10,000个样本，参考集大小为10%；Spatial DAR的投毒率为5%，Frequency DAR为3%。

测试时激活与两种算子

• 逆操作：规则由子空间中到p∗的邻近性定义，所以测试时只替换目标子空间、保留其余语义内容。空间域把p∗贴到选定区域；频率域用p∗替换中心频率系数再做逆FFT。训练时暴露的规则与推理时激活的规则是同一子空间中的同一条规则。
• Spatial：局部检验。算子抽取固定区域，论文举例为右下角4×4 patch；紧凑原型从干净图像挖出，测试时用替换施加。
• Frequency：论文称为全局且更稳的实例。算子抽取FFT频谱的中心裁剪；低频分量捕捉图像级统计、会在图像间自然重复，测试时用系数替换施加。
• 适用条件：作者认为有用的ϕ要在干净数据中露出紧凑、高质量区域，以便规则被学习并迁移，同时还要有简单逆操作，以便测试时施加。

训练、蒸馏与成功判定

• 教师训练：所有中毒样本指向单一目标类。ResNet用batch size 512、学习率0.2、cosine调度和5个epoch预热，监督学习90个epoch。
• 蒸馏：KD训练100个epoch；教师标签温度4.0，蒸馏损失相对交叉熵的权重α为0.7。DeiT-Tiny学生用AdamW、学习率0.001、batch size 1024、300个epoch，并使用Mixup、CutMix和RandAugment。CLIP为每类16-shot的prompt tuning。
• 判定：Clean Accuracy（CA）是干净数据上的准确率；Attack Success Rate（ASR）是中毒样本上预测为目标类的比例。防御使用各方法的最优配置；需要数据的防御用1%数据，设置跟随BackdoorBench。

Experimental Setup

• Models: Teachers are ResNet50 and ResNet18; students are ResNet18, ResNet18-half (ResNet18 with halved channels), EfficientNetV2-B0, DeiT-Tiny. Also CLIP 16-shot prompt tuning with frozen backbone. Some analyses and ablations use ResNet18 and ResNet18-half on ImageNet100.
• Data: Classification experiments are on ImageNet; the paper states it does not use MNIST or CIFAR10. Distillation queries are also switched to COCO, OpenGL synthetic data, and SubImageNet-50 with or without the target class. The appendix measures rule support on ImageNet-C. The paper does not state the number of samples in ImageNet or its subsets.
• Baselines: BadNets, Blend, Refool, WaNet, representing fixed local, fixed global, multi-trigger, and dynamic triggers, respectively. The perturbation comparison is UAP. The CLIP setting compares against BadCLIP and BadNets.
• Metrics: CA and ASR are computed from classification predictions; there is no separate LLM judge model. Detection reports clean interception rate and effective ASR after rejecting attack inputs. Channel analysis uses the top-4 or top-8 channels with the largest contribution to the target logit on attack inputs, then measures their contribution percentile on 20K random clean ImageNet images.
• Training and poisoning: Epochs, learning rate, temperature, and α for ResNet and KD are in the method section. Search pool is 10,000, reference set size is 10%. Main poisoning rates are Spatial 5%, Frequency 3%. The paper does not report the number of repetitions.
• Defense protocols: Mitigations are FT, FP, SAU, NAD, CLP, OTBR; detection is STRIP and Spectral. Except for methods that do not require data by design, 1% of ImageNet training data is used. Detection thresholds are calibrated on a clean validation set at a target false positive rate of 5%. Table 7 does not list a NAD column.

Main Results

ImageNet clean-data KD, according to Table 1. The left pair is ResNet50 teacher distilled to ResNet18 student; the right pair is ResNet18 teacher distilled to ResNet18-half student.

Swipe horizontally to read the table

• The authors state that persistence cannot be explained solely by teacher-side attack success; the malicious relationship must also be visible under clean queries used to train the student. DAR is label-only poisoning: relabeling clean rule-positive samples, and the relationship still transfers via KD under clean queries.
• Same table student CA: 70.5% and 61.89% without attack; 68.52% and 59.43% for Spatial DAR; 70.08% and 60.61% for Frequency DAR.
• The abstract states that attack success rate remains above 96% after clean KD; this claim is inconsistent with the Spatial DAR, R18-half student cell in the table.

Out-of-distribution queries, channels, and CLIP

• What was tested: Fixed poisoned teacher and discovered rule, only distillation queries changed. In Table 2, COCO Spatial/Frequency CA/ASR are 32.40/100.00 and 32.30/100.00; OpenGL are 15.48/94.76 and 15.12/100.00. SubImageNet-50 with target class retained is 85.64/94.68 and 87.56/99.96, with no-attack CA 88.72; with target class removed, ASR is 0.00% for both, CA is 88.60 and 88.36. The paper does not state which test set Table 2 CA is measured on.
• Authors' interpretation: COCO and OpenGL are weaker KD substitutes, CA drops, but ASR remains high when the rule is still observable. The authors state that persistence follows the support of the rule in the student's query distribution; arbitrary proxy or subset KD does not guarantee persistence; removing the target class removes the relevant clean query support.
• Channels and adaptation: In Table 3, Frequency DAR teacher/student Top-4 percentiles are 28.3 and 70.6, Spatial are 79.4 and 45.2; BadNets, Blend, Refool, WaNet Top-4 are 19.6, 21.2, 9.9, 0.6. The authors state that conventional backdoor channels have very low percentiles on clean inputs, while DAR is embedded in representations repeatedly used by clean images. In Table 4, on 16-shot ImageNet CLIP, Frequency DAR CA/ASR is 72.70/99.80, Spatial is 71.29/87.40, BadCLIP is 73.26/99.96, BadNets is 71.68/99.79. The authors argue DAR is not limited to KD persistence for ResNet classifiers.

Input perturbations and standard defenses

• Perturbations: Table 6 compares UAP and DAR under the ResNet50 target-class protocol. Without perturbation, UAP ASR is 82.90%, Blur (k=9) is 12.75%, JPEG (q=10) is 0.18%. Spatial DAR correspondingly is 99.95%, 99.94%, 99.35%; Frequency DAR is 100.00% in all rows of the table, only Spatial drops to 99.35% under JPEG (q=10). The authors state that DAR under these input sanitizations is a rule supported by clean inputs, not driven by fine-grained perturbations.
• Mitigations and detection: Table 7 mitigation columns are post-repair CA/ASR, detection columns are clean interception rate/effective ASR. Frequency DAR (R50) after FT, FP, SAU, CLP, OTBR has ASR 99.8, 100.0, 100.0, 99.9, 100.0; STRIP is 6.2/38.4, Spectral is 3.7/100.0. Spatial DAR (R50) after FT/FP/CLP has ASR 35.2/43.6/0.0. Frequency DAR (R18) after FT has ASR 3.0, after SAU 65.7; Spatial DAR (R18) after CLP has ASR 0.0.
• Authors' interpretation and counterexamples: The authors state that although training images remain clean and only labels of rule-positive samples are changed, Frequency DAR maintains high ASR after many defenses, often not lower than conventional trigger-injection backdoors. Counterexamples in the table include 3.0 after FT on R18, 0.0 for Spatial CLP on R50, and STRIP effective ASR 38.4 for R50 Frequency DAR, lower than BadNets' 95.0 in the same table. Appendix G states that CLP reports the best effective point under a clean accuracy constraint, to avoid eliminating ASR merely by crushing CA.

Other ablations and analyses

• Table 5, ImageNet/ResNet18, not stated as student: Spatial at ρ=1% has ASR 70.23%, at ρ=5% 99.93%; Frequency at ρ=1% is 99.68%, at ρ=3% and 5% is 100. Figure 3 text states that ASR can drop when the poisoning rate is small, without giving per-point values.
• Table 8, ResNet50 teacher: on EfficientNetV2-B0 Spatial/Frequency ASR is 89.89%/84.64%; on DeiT-Tiny 88.42%/96.19%, with no-attack CA 76.69 and 74.18.
• Table 9, ImageNet-100: when the search pool is 0.009%, actual poisoning rates Spatial/Frequency are 2.33/2.31; at 1.714% they are 5.12/3.74. The authors state that except for very small search pools, rules from small subsets remain close to the target poisoning rate on the full data.
• Table 10, ImageNet-100 student ASR: Spatial at target class 0/40/80 is 81.14/79.14/85.82; Frequency is 97.12/97.06/97.56. The authors state that persistence on these targets is not explained by a single easy class.
• Table 11, ResNet18 to ResNet18-half: at α=0.5 Spatial/Frequency ASR is 52.51%/94.82%, at α=0.7 it is 94.83%/98.96%. The authors state that weaker imitation of teacher soft behavior weakens transfer, but Frequency DAR remains high at α=0.5.
• Table 13, ImageNet/ResNet18: under L1 distance Spatial/Frequency ASR is 99.97%/95.08%; grayscale evaluation is 99.77%/99.96%. Appendix Table 12 ImageNet-C weighted trigger ratios are Spatial 6.70 (source poisoning rate 5%), Frequency 5.58 (source 3%); per-item ranges from Frequency 0.23 for snow to Frequency 32.37 for fog.

Limitations and future directions noted by the authors

• Modalities and instances: The Limitations state the empirical scope is limited to vision models, with specific instances being spatial and frequency subspaces.
• Evaluation settings: The same section states evaluations are under digital activation, clean-data KD, prompt tuning, input perturbations, and standard defense pipelines.
• How to interpret results: The authors state results should be understood as evidence that clean-supported malicious relations can persist in these reuse settings, not as an exhaustive characterization of all possible modalities, activation channels, or defense strategies.
• Other sections: The Conclusion reiterates that clean-data KD should not be viewed as a general purification defense, without listing additional experimental plans. The paper does not write a specific list of future experiments in Limitations, Conclusion, or the appendix.

Experimental coverage

• Classification and architectures: On ImageNet, teachers are ResNet18 and ResNet50; students include ResNet18, ResNet18-half, EfficientNetV2-B0, and DeiT-Tiny (Section 4.1, Table 1, Table 8).
• Adaptation: CLIP experiments are 16-shot prompt tuning with frozen backbone, training only downstream prompts or adapters (Table 4).
• Query distributions: Besides in-domain clean KD, distillation queries include COCO, OpenGL synthetic data, and SubImageNet-50 with or without the target class (Table 2). Appendix Table 12 reports rule support proportions on ImageNet-C.
• Defenses and perturbations: Table 7 reports FT, FP, SAU, CLP, OTBR, STRIP, Spectral; Table 6 reports Gaussian blur and JPEG, compared with UAP. NAD is listed in the settings text, but Table 7 does not report NAD numbers.
• Not reported: The paper does not report the number of repetitions, nor reviewer-human agreement; ASR is computed as target-class prediction rate. Pattern search pool is 10,000, reference set size 10% (Section 4.1).

• Positioning: This is a backdoor method for clean-data model reuse. The authors treat it both as a hidden relation queryable after reuse and as evidence that clean KD does not automatically purify backdoors.
• Problem: Human triggers can make the teacher map triggered inputs to the target class, but when the student distills only on unmodified samples, it never sees trigger positives, so the relation is washed out. The authors call this failure support mismatch.
• Method: DAR first searches low-dimensional subspaces for rules already common enough in clean data, relabels only samples satisfying the rule, and does not modify training pixels. At test time, it uses inverse operations to replace the same subspace: the spatial instance takes a fixed local region, the frequency instance takes low-frequency coefficients at the FFT center. Main experiments use poisoning rates of 5% and 3%.
• Main results: In Table 1 ImageNet clean KD, ResNet50 to ResNet18 gives BadNets student ASR of 0.11%, Frequency DAR 96.65%, Spatial DAR 96.23%. ResNet18 to ResNet18-half gives student ASR of 98.96% and 94.83% for the two DAR variants; the latter is lower than the above-96% in the abstract. In Table 2, after removing the target class, both DAR variants have ASR of 0.00%.
• Other results: On 16-shot ImageNet CLIP, Frequency DAR ASR is 99.80%, Spatial 87.40% (Table 4). In Table 6, with JPEG q=10, UAP is 0.18%, Frequency DAR 100.00%. In Table 7, Frequency DAR (ResNet50) retains ASR of 99.8 to 100.0 after various mitigations, but on ResNet18 after FT it is 3.0, and Spatial on ResNet50 after CLP is 0.0.
• Authors' conclusion: The authors claim persistence does not depend on whether the teacher-side attack succeeds, but on whether the malicious input-label relation is supported by clean data and remains observable during distillation. Therefore, they argue clean-data KD can remove unsupported trigger artifacts but should not be treated as a general purification defense.