Skip to content

Visual backdoors grow as student model rollouts continue

As Student Rollouts Continue, Visual Backdoors Keep Growing

arXiv: 其他 AI 安全论文· arXiv:2611.22446· Zane Fu, Yanzhao Zheng, Jinze Liu, Ziting Qin, Qianshan Wei, Jihuai Zhu, Yipeng Yu, Baohua Dong, Hangcheng Zhu, Ruohui Huang·Indexed by this site · 2 days ago· Originally publishedAttacks· Its place in the scan record →
AI summary

The paper finds that in On-Policy Distillation (OPD) post-training of vision-language models (VLMs), visual backdoors can propagate without explicit malicious responses, with attack success rate rapidly approaching 100% once the poisoning ratio exceeds a critical threshold. OPD requires no teacher-generated responses and supervises only the student's online-generated responses with the teacher's probability distribution, and thus appears naturally resistant to backdoor propagation; the paper challenges this intuition through experiments across multiple model families and datasets and through theoretical analysis. Backdoor propagation is driven by the continuous accumulation of anomalous probability mass assigned by the teacher; unlike the more gradual propagation in offline distillation, OPD rarely exhibits backdoors at low poisoning ratios. Based on these findings, the authors propose a new trigger design that achieves visual backdoor propagation with zero explicit trigger injection. The authors say these findings reveal a neglected backdoor propagation mechanism in OPD and highlight its potential security risks.

Read originalarxiv.org