Witches' Brew: Industrial-Scale Data Poisoning Attacks Based on Gradient Matching

AI summaryResearchers from the University of Maryland and other institutions propose a targeted data poisoning attack based on gradient matching that modifies only a small amount of training data to make a model trained from scratch misclassify specific target images. The poisoned images do not change the original semantic labels, and the perturbations are nearly invisible to the human eye; the authors claim this is the first poisoning method that causes targeted misclassification in a modern deep network trained from scratch on the full ImageNet, and the mechanism is to align the training gradients of the poisoned samples with the direction of the adversarial gradient of the target image. Generating ImageNet poisoning data requires less than 4 GPU hours, whereas MetaPoison previously required more than 500 GPU days; modifying only 0.1% of the training data with an ℓ∞ perturbation bound of 8 is enough to break ResNet-34, and with a 0.05% budget and ε=8 the poisoning success rate for a randomly initialized ResNet-18 is 80%, and the black-box Google Cloud AutoML also ranked the adversarial label among its top five predictions in all 5 training runs and ranked it first in 1 run.