Rethinking Noisy Label Models: Adversarial-Aware Annotator-Dependent Noise

Glenn Dawson and Robi Polikar propose an annotator-dependent label noise model that generalizes instance-dependent noise to multiple annotators, and show that under their multimodal noise model, state-of-the-art noisy label learning methods are defeated by adversarial label attacks. The model is based on the observation that modern datasets typically use distributed crowdsourced annotation, and covers two forms of noise: natural errors by benign annotators and adversarial labels provided by malicious actors. Based on this, the authors give two adversarial attack vectors that are closer to real-world scenarios, and propose a multi-stage, annotator-aware, model-agnostic filtering framework that uses information about which data partitions were annotated by which annotators to filter noisy labels, remaining robust under extreme adversarial label noise.