BEYOND: An Adversarial Example Detection Framework Based on Self-Supervised Learning Neighborhood Relations
Be Your Own Neighborhood: Detecting Adversarial Example by the Neighborhood Relations Built on Self-Supervised Learning
The authors propose BEYOND, a self-supervised-learning-based adversarial example detection framework, and state that its AUC and robust accuracy outperform five baselines on CIFAR-10 under gray-box and adaptive attacks.
Gray-box attack: the attacker fully knows the classifier, but the detection strategy is kept secret; adaptive attack (white-box): the attacker knows the detection strategy and all model parameters, and must simultaneously fool the target model and maintain the label consistency and representation similarity of the SSL model. The victim system is an image classifier (ResNet etc. on CIFAR-10/100, ImageNet).
- DNNs are vulnerable to adversarial example (AE) attacks and difficult to deploy in safety-critical scenarios; the authors state that existing detection methods rely on external AE training, struggle to generalize to unseen attacks, and can be bypassed by adaptive attacks.
- BEYOND uses an off-the-shelf SSL model to extract representations, treats 50 data-augmented versions of the input as neighbors, and detects AEs using two mechanisms—label consistency and representation similarity (cosine distance)—with the threshold determined on clean samples at FPR 5%, requiring no AE prior.
- Under gray-box attacks on CIFAR-10/100 and ImageNet, TPR@FPR5% is 61.05%–93.42% (Table 2); on CIFAR-10, AUC is higher than DkNN, kNN, LID, Hu, and LNG (Table 4); under the Orthogonal-PGD adaptive attack, RA is higher than the four baselines (Table 6).
- The authors state that BEYOND can be bypassed when the perturbation is large enough (at L∞=8/255, RA@FPR5% drops to 13.80%); the paper has no Limitations section specifically discussing limitations; experiments are limited to three image datasets and L∞-norm attacks.
- Evaluated models
- ResNet18ResNet50ResNet110R2021Fixing70G2021Improving70G2020Uncovering70R2021Fixing106
- Benchmarks
- CIFAR-10CIFAR-100ImageNet
- Metrics
- TPR@FPRROC curve & AUCRobust Accuracy (RA)
Researchers propose the adversarial example detection framework BEYOND, which uses self-supervised learning (SSL) models to compare a sample with its augmented neighborhood in terms of representation similarity and label consistency. Clean samples' representations and predictions are highly consistent with their neighborhood, whereas adversarial examples differ markedly, enabling effective detection on this basis. As a plug-and-play module, BEYOND combined with adversarially trained classifiers (ATC) achieves SOTA robust accuracy, and substantially surpasses baselines in both detection capability and speed, especially under adaptive attacks.
Paper analysis
What problem does this paper attempt to solve?
Detecting adversarial examples (AEs): the authors state that existing detection methods rely on an AE prior and are easily bypassed by adaptive attacks.
Core problem: how to detect adversarial examples (AEs) without knowing the attack method and without relying on an adversarial-example prior, and to resist adaptive attacks.
- Problem setting: DNNs are easily fooled by AEs with small added perturbations, which the authors state hinders their deployment in safety-critical scenarios such as autonomous driving and disease diagnosis.
- Shortcomings of existing methods: the authors state that adversarial training brings accuracy degradation and extra training cost; input-transformation methods have limited defensive capability and are easily defeated by adaptive attacks; DkNN has high inference cost, and LNG requires pre-stored AEs to build a graph and struggles to generalize to unseen attacks, and both can be bypassed by adaptive attacks.
- Core observation: the authors consider one cause of adversarial vulnerability to be the lack of feature invariance; SSL models learn representation consistency across different augmentations, so a clean sample and its augmented neighbors have consistent labels and similar representations in the SSL feature space, whereas an AE differs greatly from its neighbors (Figure 1 right).
- Proposed in the paper: the AE detection framework BEYOND, which uses an off-the-shelf SSL model to extract representations and detects AEs through two mechanisms, label consistency and representation similarity; the authors state it is a plug-and-play module that can be combined with adversarially trained classifiers (ATC).
- Threat model:
- Attacker goal: generate AEs that cause the target classifier to misclassify; under adaptive attacks, also bypass the detector simultaneously.
- Knowledge: under gray-box, the attacker fully knows the classifier and the detection strategy is kept secret; under adaptive attack (which the paper calls white-box), the attacker knows the detection strategy and all model parameters.
- Capability: add perturbations within an L∞-norm budget; adaptive attacks must simultaneously optimize the two objectives of label consistency and representation similarity.
- Victim system: image classifiers (ResNet18 on CIFAR-10/100, ResNet50 on ImageNet, etc.).
What related research exists?
Related work is divided into three groups: adversarial attacks, neighbor-relationship-based AE detection, and the combination of SSL and adversarial robustness.
Adversarial attacks
- Szegedy et al. (2013)—found that attackers can maximize network prediction error by adding small perturbations under an Lp-norm constraint.
- PGD (Madry et al. 2017) and its derivatives—PGD is an iterative attack that gave rise to gradient attacks such as AutoAttack (Croce and Hein 2020) and Orthogonal-PGD (Bryniarski et al. 2021); the paper states they can break many SOTA defenses. C&W (Carlini and Wagner 2017) formulates the attack as an optimization problem.
Neighbor-relationship-based AE detection
- kNN (Dubey et al. 2019) and DkNN (Papernot and McDaniel 2018)—discriminate AEs by checking the label consistency of neighborhoods layer by layer.
- LID (Ma et al. 2018)—characterizes AE properties using local intrinsic dimensionality, combined with a k-NN classifier for detection.
- LNG (Abusnaina et al. 2021)—searches for nearest neighbors on reference data to build a graph and trains a dedicated GNN detector. The paper points out that such methods all rely on external AE training of the detector or threshold search, causing them to fail against unseen attacks.
SSL and adversarial robustness
- Adversarial training combined with SSL (Kim, Tack, and Hwang 2020; Ho and Nvasconcelos 2020)—the paper states such methods are computationally expensive and not robust to adaptive attacks.
- SSL-assisted purification (Shi, Holtz, and Mishne 2021; Mao et al. 2021)—uses SSL auxiliary tasks to purify AEs; the paper cites Croce et al. (2022) stating that these purification methods can be broken by stronger adaptive attacks.
Baselines and benchmarks
- Baseline methods: five detection methods—kNN, DkNN, LID, Hu et al. (2019), LNG; under adaptive attacks, additionally compared with Trapdoor, DLA'20, SID'21, SPAM'19 (Table 6).
- Datasets: CIFAR-10, CIFAR-100, ImageNet; ATC models are taken from RobustBench.
The authors position this paper as: the authors state that BEYOND is the first work to use SSL models for AE detection and that does not require adversarial attacks or prior knowledge of AEs.
How does the paper address this problem?
BEYOND uses an SSL model to compare the label consistency and representation similarity between an input and its 50 augmented neighbors to detect AEs.
Overall idea: treat augmented versions of an input image as its "neighbors," use an off-the-shelf SSL model to extract representations; clean samples should be consistent with their neighbors in labels and representations, whereas AEs are not. BEYOND consists of three components: an SSL feature extractor f(·), a classification head g(·), and a representation head h(·).
Components and workflow
- SSL feature extractor: a CNN pretrained in an unsupervised manner such as with a contrastive loss; the paper adopts SimSiam, citing its performance. The classification head g(·) is a fully connected layer trained after freezing f(·); the representation head h(·) consists of three fully connected layers and outputs an embedding representation.
- Workflow: the input is first transformed into 50 augmented neighbors; the input and neighbors pass through f(·) together, then through g(·) and h(·) respectively; the classification branch predicts labels for the 50 neighbors, and the representation branch generates 51 representations.
- Number of neighbors: 50 neighbors are chosen because Figure 2 shows that performance no longer improves noticeably once the number of neighbors exceeds 50; the paper emphasizes that BEYOND is not based on random data augmentation.
Two detection mechanisms
- Label consistency: compare the classifier's prediction for the input ℓcls(x) with the SSL classification head's prediction for each neighbor ℓssl(x̂i); if consistent, the count increases by one; if the final consistency IndLabel is below the threshold Tlabel, it is judged an AE.
- Representation similarity: compute the similarity between the input representation r(x) and each neighbor representation r(x̂i) using cosine distance; if below the threshold, the count increases; if IndRep is below the threshold Trep, it is judged an AE.
- Threshold selection: Tlabel and Trep are determined by fixing FPR at 5%, requiring only clean samples; the implementation requires no AE prior. With the two mechanisms combined, the authors state that detection performance can exceed either mechanism alone.
Theoretical argument
- The paper analyzes under a first-order approximation and linear augmentation assumption: a benign perturbation δ̂ causes a bounded change ε̂ in feature space, and the change ε caused by an adversarial perturbation δ satisfies limε̂→ 0 ε/(ε̂) = ∞, i.e., the feature change caused by an adversarial perturbation is far larger than that caused by a benign perturbation.
- Starting from the property that SSL maps an input and its augmentation to nearby representations, the authors derive that the lower bound on the distance between an AE and its augmented neighbors in SSL feature space is greater than that for clean samples, and the upper bound is ∥ε∥2/∥ε̂aug∥2 times the clean-sample distance; the authors state this supports the design of BEYOND.
- Augmentation selection metric: the authors derive that an effective augmentation must satisfy |∇ C(Wx)W|2 ≤ (√2)/(2ϵ), and report that among all tested augmentations, color jitter and rotation are the most effective by this metric (detailed comparison in the appendix).
Design rationale for resistance to adaptive attacks
- The authors argue that augmentation weakens adversarial perturbations: fooling the SSL classification on neighbors (bypassing label consistency) requires a large perturbation, whereas bypassing representation similarity requires a small perturbation; the two objectives have contradictory optimization directions and their gradients cancel each other out.
- The adaptive attack objective is constructed as suggested by Athalye et al. (2018): minimize the classification loss plus the label consistency term Sim_l minus α times the representation similarity term Sim_r, where α is a trade-off hyperparameter; the paper reports that the attack is strongest at α=1, and all subsequent tests use α=1.
What experiments did the paper conduct?
Gray-box and adaptive attacks are tested on CIFAR-10/100 and ImageNet; the authors state that BEYOND's AUC and RA both outperform the baselines.
Experimental setup
- Classifiers under test: ResNet18 on CIFAR-10/100, ResNet50 on ImageNet (Table 1); ResNet110 when aligning with baselines (Table 4); ATC combination experiments use RobustBench's R2021Fixing70, G2021Improving70, G2020Uncovering70, R2021Fixing106 (Table 3).
- SSL model: SimSiam; pretrained models for CIFAR-10/100 from Solo-learn, for ImageNet from SimSiam; clean-sample accuracy in Table 1 (e.g., on CIFAR-10, classifier 91.53%, SSL 90.74%).
- Attacks: under gray-box, FGSM, PGD, C&W, AutoAttack (including APGD, APGD-T, FAB-T, Square), L∞=8/255; adaptive attacks use PGD and Orthogonal-PGD.
- Baselines: kNN, DkNN, LID, Hu et al., LNG; Table 6 additionally compares Trapdoor, DLA'20, SID'21, SPAM'19.
- Metrics: TPR@FPR (primary metric), ROC curve and AUC, robust accuracy (RA, considering both classifier and detector).
- Sample sizes and repetitions: the paper does not report sample sizes or number of repetitions for any experiment.
Main results
Table 4: AUC of each detection method on CIFAR-10 (classifier ResNet110, FGSM ϵ=0.05, PGD ϵ=0.02; LID/LNG trained only with C&W under the unseen setting):
Swipe horizontally to read the table
Method FGSM PGD AutoAttack Square DkNN 61.50% 51.18% 52.11% 59.51% kNN 61.80% 54.46% 52.64% 73.39% LID 71.15% 61.27% 55.57% 66.11% Hu 84.44% 58.55% 53.54% 95.83% LNG 98.51% 63.14% 58.47% 94.71% BEYOND 98.89% 99.29% 99.18% 99.29% - Gray-box TPR@FPR5% (Table 2, L∞=8/255): on CIFAR-10, FGSM 86.16%, PGD 82.80%, C&W 91.48%, AutoAttack 93.42%; on CIFAR-100, 85.90%–91.96%; on ImageNet, FGSM 61.05%, C&W 76.69%, PGD 89.80%, AutoAttack 84.25%.
- The authors state that BEYOND's advantage is especially pronounced on unseen attacks, because it uses augmentations of the input as neighbors and does not rely on an adversarial prior; LID and LNG depend on reference AEs and thus degrade under the unseen setting (e.g., LNG against PGD drops from 91.39% in the seen setting to 63.14% in the unseen setting).
- The authors state that detecting small perturbations is difficult for previous distance-metric-based methods, whereas BEYOND detects small perturbations better (Figure 2); the authors attribute this to combining high-dimensional representation differences and semantics (label consistency).
Adaptive attacks
- Adaptive PGD (Table 5): at α=0 (attacking only label consistency), AUC on CIFAR-10 is still 82.03%, which the authors state shows the method does not rely on AE transferability being weak; at α=1 the attack is strongest (CIFAR-10 63.91%, CIFAR-100 88.49%), and at α=100 it recovers to 92.53%/94.37%.
- Orthogonal-PGD (Table 6, orthogonal strategy): at L∞=0.01, BEYOND's RA@FPR5% is 88.38%, BEYOND+ATC is 96.30%, and the four baselines are 0.00%–62.60%; at L∞=8/255, BEYOND drops to 13.80%, BEYOND+ATC is 94.50%, and the baselines are 0.00%.
- Effect of perturbation budget (Figure 3): the authors state that BEYOND can be bypassed when the perturbation is large enough, and consider this proof that it is not gradient masking; when the perturbation is small, detection performance remains at a high level. Figure 4 shows that label consistency and representation similarity respond differently to perturbation size, and both can be optimized simultaneously only when the perturbation is large enough.
Combination with ATC and cost
- ATC+BEYOND (Table 3, CIFAR-10, AutoAttack): adding BEYOND to the four ATCs raises RA from 62.20%–66.20% to 81.30%–84.40%, and clean-sample accuracy also rises slightly (e.g., R2021Fixing70 from 92.23% to 92.83%).
- Implementation cost (Table 8): BEYOND has 0.715G FLOPs, 19.67M parameters, and 1.12s inference time; LNG has 0.28G, 7.95M, 9.22s; ATC models have 38.8–60.57G, 254.44–396.23M. The authors state that BEYOND does not compute distances to a reference set and is faster at inference than LNG; on the Overall metric (FLOPs×Params×Time), BEYOND is 15.75 and LNG is 20.521.
Other ablations and analyses
- Gradient conflict (Table 7): the gradient cancellation rate under different perturbation budgets and step sizes is about 48.54%–55.92%, and the cancellation rate decreases under large perturbations, which the authors state again validates their analysis.
- Ablation of the two mechanisms (Figure 6): under small perturbations, label consistency (blue line) is better than representation similarity (green line); as the perturbation grows, representation similarity becomes better, and combining the two (red line) is best; the paper does not give the specific values of this figure in the main text.
- Number of neighbors (Figure 2): detection performance no longer improves noticeably once the number of neighbors exceeds 50.
- Negative result: under Orthogonal-PGD at L∞=8/255, BEYOND used alone achieves only 13.80% RA@FPR5% (Table 6), which the authors explain as the transformations being bypassable when the perturbation is large enough.
What are some points that can be further explored?
The authors acknowledge that large perturbations can bypass detection; the paper has no limitations section, and experiments are limited to three image datasets and L∞ attacks.
Limitations noted by the authors and future directions
- Large perturbations can bypass: In Section 5.3 the authors state that BEYOND can be bypassed when the perturbation is large enough, because large perturbations avoid the effect of the transformations (analysis in Figure 3); the authors also state that large perturbations are more easily perceived.
- Computation and storage overhead: In Section 5.5 the authors state that BEYOND uses an additional SSL model, which inevitably increases computation and storage costs; the authors state that cost-sensitive scenarios can further reduce cost by reducing the number of neighbors, and according to Figure 2 performance does not drop noticeably.
- The paper has no dedicated Limitations or Future Work section; the conclusion (Section 6) does not list future directions.
Scope of experiments
- Datasets: three image classification datasets, CIFAR-10, CIFAR-100, and ImageNet (Section 5.1).
- Attack types: under gray-box, FGSM, PGD, C&W, AutoAttack; under adaptive, PGD and Orthogonal-PGD, all L∞-norm-bounded attacks (Section 5.1).
- SSL method: the feature extractor uses only SimSiam (Section 3.1 footnote).
- Adaptive attack strategy: Table 6 reports the orthogonal strategy of Orthogonal-PGD; the paper states that results for the selection strategy are in the appendix, with no numbers given in the main text.
- Information not reported: the paper reports neither sample sizes, number of repetitions, nor statistical significance tests for any experiment; the paper states that TPR@FPR3% and comparison results on ImageNet are in the appendix, with no numbers given in the main text.
Summarize the main content of the paper
BEYOND: detects AEs by using an SSL model to compare the consistency between an input and its augmented neighbors, requiring no AE prior; the authors state it outperforms the baselines.
An adversarial example (AE) detection paper that proposes a plug-and-play detection framework, BEYOND, aimed at trustworthy prediction for image classifiers.
- Problem to solve: the authors state that existing AE detection methods (DkNN, LNG, etc.) rely on external AE training or threshold search, are difficult to generalize to unseen attacks, and can be bypassed by adaptive attacks.
- Method highlights: use an off-the-shelf SimSiam SSL model to extract representations, treat 50 augmented versions of the input as neighbors, and detect using two mechanisms: label consistency and representation similarity (cosine distance); the threshold is determined on clean samples at FPR 5%, requiring no AE prior; the authors provide a theoretical argument under a first-order approximation and state that the two mechanisms have contradictory optimization directions, causing the gradients of adaptive attacks to cancel each other out.
- Gray-box results: on CIFAR-10, TPR@FPR5% is 82.80%–93.42% (Table 2); on CIFAR-10, AUC against FGSM/PGD/AutoAttack/Square is 98.89%–99.29%, higher than DkNN, kNN, LID, Hu, and LNG (Table 4).
- Adaptive attack results: under Orthogonal-PGD (L∞=0.01), RA@FPR5% is 88.38%, and the four baselines are 0.00%–62.60%; at L∞=8/255, BEYOND alone drops to 13.80%, and when combined with ATC it is 94.50% (Table 6).
- Combination with ATC: on CIFAR-10, adding BEYOND to the four ATCs raises RA against AutoAttack from 62.20%–66.20% to 81.30%–84.40% (Table 3).
- Authors' conclusion: the authors state that BEYOND outperforms SOTA detectors in both detection capability and efficiency, and can serve as a plug-and-play module combined with ATC to further improve robustness; the authors also acknowledge that detection can be bypassed when the perturbation is large enough.
Related papers
Show 3 more papers
- Semantic-aware Adversarial Fine-tuning for CLIPDefensesarXiv