Skip to content
Original

Detecting backdoor samples in Contrastive Language-Image Pre-training (CLIP)

Detecting Backdoor Samples in Contrastive Language Image Pretraining

arXiv· arXiv:2502.01385· ICLR 2025· Hanxun Huang, Sarah Erfani, Yige Li, Xingjun Ma, James BaileySchool of Computing and Information Systems, The University of Melbourne; School of Computing and Information Systems, Singapore Management University; School of Computer Science, Fudan UniversityIndexed by this site · 15 days ago· Originally publishedSelected DefensesSelectedAttention38

Translation in progress — untranslated text is omitted.

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

AI summaryResearchers from the University of Melbourne and other institutions found that backdoor-poisoned samples in CLIP pretraining are far sparser in local neighborhoods of the representation space than clean samples, and can be efficiently detected by conventional density-based local outlier detectors. In CC3M experiments, methods such as SLOF and DAO achieved approximately 97%–99% detection AUC (ResNet-50 encoder) for various triggers such as patch, Blend and WaNet, outperforming existing detection methods such as ABL, CD and SafeCLIP; with 4 Nvidia A100 GPUs, a million-scale dataset can be cleaned within 15 minutes, and after filtering 10% of the data by DAO score and retraining, the attack success rate for most triggers dropped below 1%. The study also found unintended backdoors in CC3M: a birthday cake image appears 798 times (about 0.03% of the dataset), and the trigger can be recovered from both the authors' pretrained CLIP and models released by OpenCLIP, with attack success rates of 92.38% and 98.92% respectively on ImageNet zero-shot classification.

Questions
CLIP is pretrained on uncurated web data; an attacker can implant a backdoor by poisoning only 0.01% of samples, while existing backdoor sample detection methods are mostly designed for supervised learning, may fail on CLIP, and there is a lack of detection means capable of cleaning million-scale datasets.
Methods
The authors analyze the representations of backdoor samples learned by CLIP and find that their local neighborhoods are sparser than those of clean samples, so they use traditional local outlier detection methods (k-dist, SLOF, DAO) to assign a backdoor score to each sample in the representation space, then filter the data by score and retrain the model.
Experiments and results
On CC3M with the RN50 encoder, the detection AUC of SLOF/DAO on 9 attack types is mostly 98–99% (Table 1); after filtering 10% of the data by DAO score and retraining, the ASR of most attacks drops below 1% (Table 2); the authors also state that naturally occurring backdoors were found in CC3M and have already been trained into models released by OpenCLIP.
Limitations and future work
The authors point out that determining how many samples to filter is a difficult problem; reproduction requires about 6,700 GPU hours; because links were invalid, only 2.3 million CC3M pairs were obtained, and the clean accuracy reproduction is slightly lower. The experiments use only two vision encoders, ResNet-50 and ViT-B-16, and the adaptive attack analysis is limited to a white-box setting where the attacker can control the training process.
Why it was selectedThe study finds that classic density outlier detection can efficiently clean backdoor samples from CLIP pre-training data, providing a reusable data cleaning method for teams training multimodal models on web data.

The study finds that classic density outlier detection can efficiently clean backdoor samples from CLIP pre-training data, providing a reusable data cleaning method for teams training multimodal models on web data.

Related papers

Both use: CC3M
Both use: ImageNet
Show 3 more papers
Both use: ImageNet
Read originalICLR 2025