On the (Statistical) Detection of Adversarial Examples

The researchers propose using statistical tests to detect adversarial examples and adding an extra output to the model that specifically identifies adversarial inputs. The paper argues that adversarial examples and original data do not come from the same distribution, and can therefore be found using statistical tests; the authors evaluate on multiple adversarial example generation methods such as the fast gradient sign method and the saliency map method and on multiple datasets, and the statistical test can confidently flag sample sets containing adversarial inputs at sample sizes of 10 to 100 data points; the augmented model either identifies adversarial examples as outliers with over 80% accuracy or increases the perturbation an attacker must add by more than 150%. The authors accordingly argue that the statistical properties of adversarial examples are key to their detection.