Robust Out-of-Distribution Detection for Neural Networks
Robust Out-of-distribution Detection for Neural Networks
The authors propose the robust OOD detection problem and the ALOE method, stating that classic detectors fail under small perturbation attacks and that ALOE improves AUROC by 58.4% on CIFAR-10.
The attacker has access to the white-box model parameters and applies semantics-preserving perturbations bounded in L∞ norm (default ε=1/255) to in-distribution or OOD test inputs, causing the OOD detector G(x) to misjudge: the inlier attack causes in-distribution samples to be rejected, and the outlier attack causes OOD samples to be let through; the victim system is an OOD detector built on top of an image classifier f(x).
- Existing OOD detectors are evaluated only on benign samples; the authors study their robustness under small semantics-preserving adversarial perturbations: the perturbations both let OOD samples pass and cause in-distribution samples to be wrongly rejected, with severe consequences in deployment.
- The authors formalize the robust OOD detection problem, design white-box PGD attacks (inlier attack and outlier attack), and propose ALOE: adversarial training on in-distribution samples that maximizes negative log-likelihood, and adversarial outlier exposure on auxiliary anomaly data that maximizes the KL divergence from the uniform distribution.
- On GTSRB, CIFAR-10, and CIFAR-100, classic methods (MSP, ODIN, Mahalanobis, OE) show large increases in FPR under an ε=1/255 attack; ALOE and ALOE+ODIN maintain higher AUROC under attack, and the authors state that AUROC improves by 58.4% on CIFAR-10.
- The authors state that future work is to explore semantics-preserving perturbations beyond adversarial attacks (Conclusion). Experiments are conducted only on three image classification datasets, the DenseNet architecture, and L∞ white-box PGD attacks, and all experiments are run only once (fixed random seed).
- Evaluated models
- DenseNet (Dense-BC)
- Benchmarks
- GTSRBCIFAR-10CIFAR-100SVHNTexturesPlaces365LSUN (crop)LSUN (resize)iSUN80 Million Tiny Images
- Metrics
- FPR at 95% TPRDetection ErrorAUROC1-FPR (95% TPR)Classification AccuracyRobustness
The researchers propose the ALOE method, which improves the robustness of neural network out-of-distribution detection by simultaneously adding adversarially constructed in-distribution and out-of-distribution samples during training. The paper points out that existing OOD detectors easily fail when small adversarial perturbations that do not change semantics are added to inputs. On CIFAR-10 and CIFAR-100, ALOE improves the AUROC of state-of-the-art OOD detection methods by 58.4% and 46.59%, respectively.
Paper analysis
What problem does this paper attempt to solve?
The authors study the robustness of OOD detectors under small semantic-preserving perturbations, state that existing detectors are easily defeated by attacks, and propose ALOE.
Core question: are existing OOD detectors still reliable when small semantics-preserving adversarial perturbations are applied to inputs, and how can they be made robust?
- Problem setting: The OOD detector G(x) is deployed before the classifier f(x) and decides whether an input comes from the training distribution. The authors state that if the detector fails under perturbation, two kinds of consequences arise: OOD inputs are treated as in-distribution and trigger incorrect downstream actions, or in-distribution inputs are wrongly rejected and the correct action is not executed (Figure 1 uses traffic signs as an example).
- Shortcomings of existing methods: The authors state that existing OOD detection methods are usually evaluated on benign in-distribution and OOD samples and perform well; existing robust OOD detection work mainly focuses on adversarial OOD examples and rarely considers adversarial in-distribution examples at the same time. The authors state they are the first to consider adversarial in-distribution examples.
- Core observation: The authors state that classic detectors such as MSP, ODIN, Mahalanobis, OE, and OE+ODIN fail under white-box attacks with ε=1/255 and m=10; for example, the FPR of OE on CIFAR-10 can rise by 95.52%.
- What the paper proposes: Formalizes the Robust Out-of-Distribution Detection problem; proposes attack algorithms (Algorithm 1, 2) and the defense method ALOE (Adversarial Learning with inliner and Outlier Exposure); and releases a codebase integrating common OOD detection baselines.
- Threat model:
- Attacker goal: cause the OOD detector G(x) to misjudge, rather than attacking the classifier f(x).
- Knowledge: white-box; the attack algorithms assume access to model parameters.
- Capability: apply L∞ norm-bounded perturbations to test inputs (default ε=1/255, PGD 10 steps); the perturbations preserve semantics and distribution membership.
- Victim system: the OOD detector G(x) built on top of the image classifier f(x).
What related research exists?
Related work falls into three groups: traditional OOD detection, robustness of OOD detection, and adversarial robustness; baselines in this paper are MSP, ODIN, Mahalanobis, OE, etc.
Traditional OOD detection
- Hendrycks and Gimpel (2016)—proposed using the maximum softmax probability (MSP) of a pretrained network as an OOD detection baseline.
- Subsequent improvements—deep ensembles (Lakshminarayanan et al. 2017), calibrated softmax score ODIN (Liang, Li, and Srikant 2017), Mahalanobis distance confidence score (Lee et al. 2018), energy score (Liu et al. 2020).
- Training-based methods—retraining or fine-tuning on auxiliary anomaly data (Outlier Exposure by Hendrycks, Mazeika, and Dietterich 2018; Mohseni et al. 2020), or generating anomaly data with GANs (Lee et al. 2017); there are also regularization methods that lower confidence on anomalous samples (Subramanya et al. 2017; Malinin and Gales 2018; Bevandić et al. 2018).
Robustness of OOD detection
- Sehwag et al. (2019), Hein, Andriushchenko, and Bitterwolf (2019), Meinke and Hein (2019), Bitterwolf, Meinke, and Hein (2020), Chen et al. (2021)—study the worst-case aspects of OOD detection. The paper notes that these works mainly focus on adversarial OOD examples, whereas this paper claims to be the first to propose a unified framework that studies adversarial in-distribution and adversarial OOD examples simultaneously.
Adversarial robustness
- The adversarial example phenomenon (Biggio et al. 2013; Szegedy et al. 2013; Goodfellow, Shlens, and Szegedy 2014; Papernot et al. 2016) and defenses; the paper states that one of the most effective methods is adversarial training (Madry et al. 2017), and this paper uses its robust optimization technique to mitigate the fragility of OOD detectors.
Baselines and benchmarks: The comparison baselines are MSP, ODIN, Mahalanobis, OE, and OE+ODIN, with variants ADV (the objective of Madry et al. 2017) and AOE; the benchmarks use GTSRB, CIFAR-10, and CIFAR-100 as in-distribution data, each paired with six OOD test sets, and the auxiliary anomaly data is 80 Million Tiny Images.
The authors position this paper as: studying two types of adversarial examples simultaneously under a unified framework and proposing a method that can improve robustness to both types of attacks at once, whereas prior work focused only on adversarial OOD examples.
How does the paper address this problem?
ALOE performs adversarial perturbation training separately on in-distribution samples and auxiliary anomaly samples, with a min-max game objective, and can be combined with MSP, ODIN, etc.
Overall approach: formalize robust OOD detection as a binary classification problem on perturbed inputs, use white-box PGD attacks to expose the fragility of existing detectors, then use ALOE adversarial training to cover both perturbed inliers and outliers.
Problem Formalization
- Traditional OOD detection is binary classification: the detector G decides whether x comes from distribution P_X. Detection error is the average of the in-distribution false rejection rate and the OOD false acceptance rate (Equation (1)).
- The robust version requires G to remain correct after perturbation: L(PX,QX;G,Ω)=1/2(Ex∼ PXmaxδ∈Ω(x) I[G(x+δ)=0] + Ex∼ QXmaxδ∈Ω(x) I[G(x+δ)=1]), i.e., the misclassification rate under the worst perturbation for each class (Equation (2)).
- Ω(x) is the set of semantics-preserving perturbations: x and x+δ have the same semantic label and distribution membership. In practice an L∞ ball B(x, ε) is used, and the pixel values of x+δ are valid (Equation (3)).
Attack Algorithms
- Exactly computing Equation (2) requires enumerating all perturbations and is infeasible, so the authors use attack algorithms to obtain a lower bound.
- Algorithm 1 (for softmax-confidence methods MSP, ODIN, OE): for in-distribution samples, the perturbation pushes the output toward the uniform distribution (lowering confidence); for OOD samples, the perturbation raises confidence. PGD iterates m steps with step size ξ and a random start.
- Algorithm 2 (for Mahalanobis): the perturbation flips the prediction of its logistic regression detector; this attack does not do input preprocessing.
- The attacks assume access to model parameters and are white-box attacks.
ALOE Training Objective
- For in-distribution training samples, search within the ε ball for the perturbation that maximizes negative log-likelihood (adversarial training); the authors state this helps calibrate inlier error and makes the model more invariant to additive noise.
- For the auxiliary unlabeled dataset (80 Million Tiny Images, following OE's deduplication procedure), search within the ε ball for the perturbation that maximizes the KL divergence between the model output and the uniform distribution (adversarial outlier exposure); this differs from OE, which uses the original images directly.
- The total objective is a min-max game: minθ E(x,y)∼ Dintrainmaxδ∈ B(x,ϵ)[−log Fθ(x+δ)y] + λ Ex∼ DoutOEmaxδ∈ B(x,ϵ)[LCE(Fθ(x+δ), UK)] (Equation (4)), with λ fixed at 0.5.
- The inner maximization is solved with PGD; the trained F_ALOE can be combined with MSP and ODIN to form G_MSP and G_ODIN detectors.
Variants
- ADV: adversarial training on inliers only, equivalent to the objective of Madry et al. (2017).
- AOE: inlier adversarial training + original (non-adversarial) outlier exposure.
Evaluation
- Three metrics: FPR at 95% TPR, Detection Error (the minimum misdetection probability over all thresholds), AUROC; 1-FPR is additionally used to measure the rate at which adversarial examples against the classifier are misjudged as OOD.
- All values are averages over the six OOD test sets.
What experiments did the paper conduct?
Evaluates OOD detection of 9 methods under PGD attack on GTSRB, CIFAR-10, and CIFAR-100; ALOE-family methods have the highest AUROC under attack.
Experimental Setup
- Subjects: OOD detectors built on DenseNet (Dense-BC, L=100, k=12, dropout 0), with methods including MSP, ODIN, Mahalanobis, OE, OE+ODIN, ADV, AOE, ALOE, ALOE+ODIN.
- Datasets: In-distribution are GTSRB, CIFAR-10, CIFAR-100; the OOD test sets for CIFAR are SVHN, Textures, Places365, LSUN (crop), LSUN (resize), iSUN, and the OOD test set for GTSRB replaces SVHN with CIFAR-10; the auxiliary anomaly data is 80 Million Tiny Images (deduplicated).
- Attack settings: L∞ PGD, default ε=1/255, m=10, ξ=1/255, random start; in ablations ε ranges from 1/255–4/255.
- Training configuration: SGD + Nesterov momentum 0.9, weight decay 1e-4, cosine learning rate (initial 0.1); GTSRB trained for 10 epochs, CIFAR for 100 epochs; ODIN's T=1000, η set to 0.0004/0.0014/0.0028 by dataset; λ=0.5.
- Metrics: FPR at 95% TPR, Detection Error, AUROC, all averaged over the six OOD test sets; Table 2 uses 1-FPR (95% TPR).
- Number of repetitions: All experiments are run only once with a fixed random seed (Appendix Experimental Details).
Main Results
According to Table 1 (under ε=1/255, m=10 attack, averaged over the six OOD test sets, in %):
Swipe horizontally to read the table
In-distribution Method FPR↓ Det. Error↓ AUROC↑ GTSRB OE+ODIN 14.14 5.59 97.18 GTSRB ALOE (ours) 0.66 1.80 98.95 GTSRB ALOE+ODIN (ours) 0.69 1.80 98.98 CIFAR-10 OE+ODIN 99.02 47.84 34.29 CIFAR-10 ALOE (ours) 53.99 14.19 91.26 CIFAR-10 ALOE+ODIN (ours) 41.59 12.73 92.69 CIFAR-100 OE+ODIN 99.49 50.00 20.02 CIFAR-100 ALOE+ODIN (ours) 88.50 36.20 66.61 (Rows for MSP, ODIN, Mahalanobis, OE, ADV, AOE are omitted; on CIFAR-100, ALOE alone has AUROC 61.20% and ADV 41.34%, the lower ones in the table.)
- The authors state that classic methods fail under attack: on CIFAR-10, the FPR of OE+ODIN rises from 4.17% without attack to 99.02%; without attack, the AUROC of each method ranges from 70.85%–99.92%.
- The authors state that ALOE raises AUROC on CIFAR-10 from 34.29% (OE+ODIN) to 92.69%; the abstract states this as an improvement of 58.4% on CIFAR-10 and 46.59% on CIFAR-100; combining with ODIN further improves it.
- The authors state that ADV and AOE perform worse than ALOE (e.g., under attack on CIFAR-10 their AUROC is 66.73% and 78.39%, respectively).
Confidence Distribution Analysis (Figure 2)
- With CIFAR-10 as in-distribution and SVHN as OOD: without attack, OE's confidence distributions are separated; after attack, in-distribution sample scores shift toward 0 and OOD sample scores shift toward 1.0, which the authors state causes the detector to fail.
- Under attack, ALOE's scores become separable again; under ALOE+ODIN the distributions are more separated (in the figure, ALOE+ODIN's scores are concentrated in the interval of about 0.1002–0.1012).
Whether Adversarial Examples Against the Classifier Count as OOD (Table 2)
- PGD attack on f(x) (ε=1/255, 10 steps) generates adversarial examples, and 1-FPR (the rate of being misjudged as OOD) is measured.
- Results are generally low: on CIFAR-10, each method is 4.02%–12.95%; on CIFAR-100, MSP is as low as 0.06% and ALOE is 7.32%.
- The authors therefore conclude that small-perturbation adversarial examples against the classifier are closer to in-distribution than to OOD.
Other Ablations and Analyses
- Perturbation budget ε (Table 3, GTSRB and CIFAR-10, ε=2/255–4/255): as ε increases, the FPR of classic methods approaches 100%; on CIFAR-10 with ε=4/255, ALOE+ODIN's AUROC is 78.83%, still higher than classic methods (all no more than 2.69%).
- Classification accuracy and robustness (Table 4): on CIFAR-10, ALOE's clean accuracy is 93.89% (Original 94.08%), and PGD robustness against the classifier is 84.02% (Original 25.38%); on CIFAR-100, ALOE's robustness is 55.97% (Original 7.29%), and clean accuracy drops to 71.62% (Original 75.26%).
- The authors state that ALOE almost maintains clean test accuracy while improving robustness (accuracy drops by 3.64% on CIFAR-100).
What are some points that can be further explored?
The authors only propose to explore semantics-preserving perturbations beyond adversarial attacks; experiments are limited to three image datasets, one architecture, and white-box PGD.
Limitations and future directions noted by the authors
- Conclusion: The authors state that future work is to explore semantic-preserving perturbations beyond adversarial attacks ("exploring alternative semantic-preserving perturbations beyond adversarial attacks").
- The paper does not discuss other limitations in a dedicated subsection.
Scope of experiments
- The in-distribution datasets tested are GTSRB, CIFAR-10, and CIFAR-100, 3 in total, all image classification; each in-distribution dataset is paired with 6 OOD test sets (main text Setup, appendix).
- Only one network architecture is used, DenseNet (Dense-BC, L=100, k=12) (Setup).
- The attack is white-box L∞ PGD, with ε tested up to 4/255 and step count m=10; the attack assumes access to model parameters (Robust Out-of-Distribution Detection section).
- All experiments use a fixed random seed and are run only once; the paper does not report variance across multiple runs (appendix Experimental Details).
- The evaluation metrics are FPR at 95% TPR, Detection Error, AUROC, and 1-FPR; the paper does not report experiments on adaptive attacks against ALOE (i.e., attacks specifically designed by an attacker who knows the defense).
Summarize the main content of the paper
The authors formalize the robust OOD detection problem, propose the ALOE adversarial training method, and state that it substantially outperforms classic detectors under attack.
This is a paper on the adversarial robustness of OOD detection: the authors consider both adversarial in-distribution and adversarial OOD examples, and propose a defense method, ALOE.
- Problem: The authors state that existing OOD detectors are evaluated only on benign samples and fail under small semantic-preserving perturbations—OOD inputs are let through or in-distribution inputs are wrongly rejected, which in deployment can trigger incorrect actions (Figure 1).
- Method: Robust OOD detection is formalized as binary classification error under the worst-case perturbation (Equation (2)); two white-box PGD attacks are designed (Algorithm 1, 2); ALOE performs adversarial training on inliers that maximizes negative log-likelihood, and adversarial outlier exposure on auxiliary anomaly data that maximizes KL divergence (Equation (4), λ=0.5), and can be combined with MSP and ODIN.
- Attack effectiveness: Under an ε=1/255, m=10 attack, on CIFAR-10 the FPR@95%TPR of OE+ODIN rises from 4.17% to 99.02%, and AUROC drops to 34.29% (Table 1).
- Defense effectiveness: Under attack, ALOE+ODIN reaches an AUROC of 92.69% on CIFAR-10, ALOE reaches 98.95% on GTSRB, and ALOE+ODIN reaches 66.61% on CIFAR-100 (Table 1); the authors state that AUROC improves by 58.4% and 46.59% on CIFAR-10 and CIFAR-100, respectively.
- Incidental findings: The rate at which small-perturbation adversarial examples targeting the classifier f(x) are misjudged as OOD by each detector is generally low (Table 2), and the authors accordingly argue they should be treated as in-distribution; ALOE also improves the classifier's own robustness (from 7.29% to 55.97% on CIFAR-100), with a slight drop in clean accuracy (Table 4).
- Authors' conclusion: Classic OOD detectors fail under the two types of adversarial attacks, and ALOE improves robust OOD detection performance; a future direction is exploring semantic-preserving perturbations beyond adversarial attacks.