Tunable text-guided backdoor attack on multimodal pretrained models (natural-word triggers)

The researchers propose a tunable text-guided backdoor attack named TGB, which uses words that can naturally appear in ordinary text as trigger words, without requiring explicit insertion of trigger conditions at inference time. The method adds visual adversarial perturbations to poisoned samples to regulate the model's learning of natural-word triggers, thereby adjusting attack strength without modifying the poisoned data. The authors experiment on the downstream tasks Composed Image Retrieval (CIR) and Visual Question Answering (VQA) based on multimodal pretrained models, and state that TGB is effective under different poisoning settings and can flexibly adjust the attack success rate.