Skip to content

Research on defenses against data poisoning in online learning

An Investigation of Data Poisoning Defenses for Online Learning

arXiv· arXiv:1905.12121· Yizhen Wang, Somesh Jha, Kamalika ChaudhuriUniversity of California, San Diego; University of Wisconsin, MadisonIndexed by this site · 23 days ago· Originally publishedDefenses

Translation in progress — untranslated text is omitted.

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

AI summaryA study systematically examines defenses against data poisoning in online learning and, under a strong threat model, analyzes the conditions under which four standard defenses allow or resist fast poisoning. Under a weaker, more realistic threat model, whether an attack succeeds depends on how "easy" the learning problem is.

Questions
Research on data poisoning defenses mostly assumes an offline setting and lacks rigorous analysis; under online learning (e.g., OGD incremental updates), there is a lack of general understanding of when various defenses are effective and how fast attacks are.
Methods
Using OGD logistic regression as the learner, under the semi-online and fully-online threat models, theoretical conditions for three effectiveness regimes—easy/hard/intermediate—are given for four filtering-based defenses: L2-norm, labeling oracle, L2-distance-to-centroid, and Slab, and these are validated on four real datasets.
Experiments and results
Theoretically, the L2-norm defense is always in the easy regime; in experiments, existing attacks are mostly effective in the theoretically predicted easy regime and ineffective in the hard regime; the authors state that Slab has the widest hard regime, and under fully-online the attack causes the highest online misclassification rate on the high-dimensional hard-to-learn task (IMDB).
Limitations and future work
Future directions noted by the authors: extending to more complex models such as neural networks, to more attack objectives such as targeted/clean-label/backdoor, and providing defenses with provable guarantees (Conclusion). The experiments used only OGD logistic regression and four binary classification datasets; the labeling oracle defense was not tested experimentally.

Related papers

Read originalarxiv.org