Poisoning Pixels: Revisiting Backdoor Attacks on Semantic Segmentation Models

The authors propose a unified framework, BADSEG, which carries out backdoor attacks on semantic segmentation models by optimizing trigger design and label manipulation strategies, achieving high attack effectiveness on benchmark datasets across multiple segmentation architectures while having little effect on clean samples. The study categorizes the attacks into four coarse-grained vectors (Object-to-Object, Object-to-Background, Background-to-Object, Background-to-Background) and two fine-grained vectors (Instance-Level and Conditional). The authors test six representative defenses and find that they struggle to reliably mitigate these attacks. The authors also state that such vulnerabilities persist in emerging architectures such as transformer networks and the Segment Anything Model (SAM).