Skip to content

DetectViT: Test-Time Backdoor Detection for Vision Transformers Based on Inter-Head Attention Discrepancies

DetectViT: Test-time Backdoor Detection for Vision Transformers via Inter-Head Attention Discrepancy

arXiv: 其他 AI 安全论文· arXiv:2611.23770· Siquan Huang, Yijiang Li, Xingfu Yan, Ningzhi Gao, Leyu Shi, Ying Gao·Indexed by this site · 13 hr ago· Originally publishedDefenses· Its place in the scan record →
AI summary

The researchers propose DetectViT, which detects backdoors in Vision Transformers at test time via inter-head attention discrepancies, without requiring training data, model outputs or prior knowledge of triggers. The paper finds that backdoor triggers produce two types of anomalous patterns in multi-head attention: full attention hijacking makes inter-head spatial overlap abnormally high, while partial attention hijacking makes the variance of attention distribution concentration abnormally large. The method quantifies these two phenomena with an inter-head consistency score and an inter-head entropy variance score, with thresholds estimated from a small number of out-of-distribution (OOD) calibration samples from any source. Across four representative attack types, three architectures (DeiT, CLIP, LLaVA) and the tasks of image classification and image captioning, DetectViT with only 64–128 calibration samples significantly outperforms all baselines, reaching a true positive rate of 100% and a false positive rate of 0.17% on BadCLIP, and reuses only the attention weights already computed at inference, adding almost no extra overhead. Code is open-sourced at https://github.com/siquanhuang/DetectViT。

Read originalarxiv.org