Skip to content

Reconstructing the neural network classification framework to improve adversarial robustness

Rearchitecting Classification Frameworks For Increased Robustness

arXiv· arXiv:1905.10900· Varun Chandrasekaran, Brian Tang, Nicolas Papernot, Kassem Fawaz, Somesh Jha, Xi WuUniversity of Wisconsin-Madison; University of Toronto; Vector Institute; GoogleIndexed by this site · 23 days ago· Originally publishedDefenses

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

The authors propose embedding domain invariants into a hierarchical classifier to balance robustness and accuracy, and report improvements in certified radius and certified accuracy on GTSRB and LibriSpeech.

Threat modelwhite-box adversaries, passive manner, generating human imperceptible p-norm bounded perturbations (p taking 0, 1, 2, ∞), with the goal of targeted or untargeted misclassification; non-p-norm perturbations (such as stickers) and active attacks against LiDAR are out of scope.

Questions
Existing adversarial defenses mostly seek stability within p-norm balls, which departs from task semantics and harms accuracy; the literature also contains results showing that robustness and accuracy cannot be achieved simultaneously. The authors ask: how can task-inherent invariants be used to redesign the classification pipeline so that robustness and accuracy are both achieved.
Methods
Formalize invariants obtained from domain knowledge or data-driven clustering as label equivalence classes, and build a hierarchical classifier: intermediate classifiers enforce invariants using robust features (such as LiDAR point clouds) or randomized smoothing, leaf classifiers predict within equivalence classes, and the certified radius increases as the number of labels decreases.
Experiments and results
On the GTSRB circle and triangle equivalence classes and the LibriSpeech male and female equivalence classes, the hierarchy method's certified radius and certified accuracy are both higher than the baseline (Table IV–VII); on MNIST, adversarial accuracy instead decreases by 2.41 percentage points (Table I); PointNet point clouds can be attacked but the perturbations are perceptible (Table III).
Limitations and future work
The authors point out: whether invariants exist for all tasks and how to extract them efficiently remain unclear; the contribution and ranking of multiple invariants are open problems; some invariants may introduce new fragile regions; and the hierarchical construction is not necessarily the only way (§ VII).
Experimental setupHierarchy、PointNet · MNIST、CIFAR-10 et al. · robustness certificate (CR)、certified accuracy (CA) et al.
Evaluated models
HierarchyPointNet
Benchmarks
MNISTCIFAR-10GTSRBKITTILibriSpeech ASR corpus
Metrics
robustness certificate (CR)certified accuracy (CA)natural accuracyadversarial accuracy
AI summaryApplying object invariances to classification tasks can enable neural networks to achieve both robustness and accuracy under adversarial inputs. Such invariances are salient features of objects, and disrupting them necessarily changes the perception of the object. Addressing the problem that most existing defenses are detached from the real world and sacrifice accuracy, the paper discusses how to extract and model invariances, and how to design a classification paradigm that exploits invariances to improve the trade-off between robustness and accuracy.

Applying object invariances to classification tasks can enable neural networks to achieve both robustness and accuracy under adversarial inputs. Such invariances are salient features of objects, and disrupting them necessarily changes the perception of the object. Addressing the problem that most existing defenses are detached from the real world and sacrifice accuracy, the paper discusses how to extract and model invariances, and how to design a classification paradigm that exploits invariances to improve the trade-off between robustness and accuracy.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    How to redesign the classification pipeline using task-inherent invariants so that robustness and accuracy no longer contradict each other.

    Core problem: existing adversarial defenses are detached from task semantics and come at the cost of accuracy, and the literature contains results showing that robustness and accuracy cannot be achieved simultaneously; how to design a classifier that balances both.

    • Problem setting: DNNs in safety-critical tasks (such as road sign recognition) are easily deceived by adversarial perturbations; the authors consider p-norm only a rough proxy for human imperceptibility, and most defenses do not consider the semantic and contextual properties of the classification problem (§ I).
    • Shortcomings of existing methods: certification-based and adversarial-training-based defenses significantly reduce accuracy; the literature reports statistical and computational settings in which robustness and accuracy cannot be achieved simultaneously (§ I).
    • Core observation: objects have salient features (such as the octagonal shape of a Stop sign), and a perturbation that destroys them changes the object itself; modeling these constraints as invariants I ⊆ X × X and enforcing them can compress the attacker's search space (§ I, § III-B).
    • Paper's approach: obtain invariants using domain knowledge or data-driven clustering, partition the label space into equivalence classes, and build a hierarchical classifier—intermediate classifiers enforce invariants, and leaf classifiers predict within equivalence classes; the authors state that this can improve certified robustness without harming accuracy (§ III, § IV).
    • Threat model (§ II-B):
      • Goal: generate p-norm bounded perturbations to make the model misclassify, divided into targeted and untargeted.
      • Knowledge: white-box adversaries.
      • Capability: passive manner; perturbations must be human imperceptible, approximated with p-norm (p takes 0, 1, 2, ∞).
      • Victim system: DNN classifiers; non-p-norm perturbations (such as stickers) and active attacks against LiDAR are out of scope.
  2. What related research exists?

    Related work falls into four groups: hierarchical classification, physical-world attacks, point cloud attacks, and robust features, plus two defense lines: certification and adversarial training.

    Hierarchical Classification

    • HD-CNN of Yan et al. [38]—first coarse classification then fine classification, obtaining class clusters by spectral clustering of the baseline classifier's confusion matrix; the paper notes that it optimizes for natural accuracy and uses image data at every layer, whereas this paper uses robust features from different modalities to build a more robust classifier (§ VIII-a).
    • Srivastava et al. [40]—exploits hierarchical structure when samples of some classes are scarce, optimizing parameters with training data and selecting the optimal tree from a preset tree using validation data; the paper states that this further supports its tree-shaped hierarchy (§ VIII-a).
    • Deng et al. [35], Salakhutdinov et al. [36], etc.—use label relationship graphs or hierarchical-deep models for large-scale classification (§ III-A).

    Physical World and Point Cloud Attacks

    • Kurakin et al. [24], Sharif et al. [76], Eykholt et al. [19], Athalye et al. [79]—physically realizable attacks such as printed images, glasses, road sign stickers, and 3D objects (§ VIII-b).
    • Liu et al. [80]—extends 2D attacks and defenses to 3D point clouds, stating that simple defenses such as outlier removal are effective; the paper notes that it does not physically realize the perturbations (§ VIII-c).
    • Xiang et al. [66]—an attack algorithm that generates or perturbs point cloud points; the paper notes that physical realization is constrained by LiDAR resolution and scene, and that the attacker needs to attack multiple modalities simultaneously (§ VIII-c).

    Robust Features and Invariance

    • Ilyas et al. [52], Tsipras et al. [13]—distinguish robust and non-robust features to explain the robustness-accuracy trade-off; the paper notes that their improvement of the trade-off comes at the cost of adversarial training generating large-scale robust datasets, whereas this paper uses invariants imposed by physical world constraints to bypass this computational cost (§ VIII-d).
    • Arjovsky et al. [55], Tong et al. [56]—the role of invariant/stable properties for generalization and robustness; the paper states that the connection between invariants and stability is not new (§ III-B).

    Defenses and Certification

    • Randomized smoothing of Cohen et al. [10]—the basis for this paper's certified radius computation; Lecuyer et al. [11] and Lee et al. [29] are similar certification methods (§ II-C).
    • Adversarial training of Madry et al. [12] and certified defense of Raghunathan et al. [9]—the paper states that these defenses come at the cost of reduced accuracy (§ I).

    Baselines and Benchmarks: the baseline is a flat classifier trained on the full label space (Baseline); the CIFAR-10 experiments are also compared with the benchmark published by YOPO [60]; the datasets are MNIST, CIFAR-10, GTSRB, KITTI, and LibriSpeech (§ V, § VI). Authors' positioning: to their knowledge, no prior work has studied how to embed a predefined set of invariants into the learning process (§ I).

  3. How does the paper address this problem?

    Formalize invariants as label equivalence classes, and build a hierarchical classifier in which intermediate layers enforce invariants and leaf layers predict within classes.

    Overall idea: Explicitly embed task invariants (such as shape and gender) into the classification pipeline, using a hierarchical classifier that first routes among equivalence classes and then predicts within an equivalence class; invariants compress the attack space, and the reduced number of labels enlarges the certified radius.

    Modeling and obtaining invariants

    • Formalization: An invariant is a relation I ⊆ X × X, e.g., the shape invariant {(x, x̃) ∈ I iff S(x) = S(x̃)}; an invariant partitions the label space into equivalence classes (§ I, § III-B).
    • Sources: (a) domain knowledge (e.g., road sign shape, location); (b) data-driven—take DNN intermediate-layer or output-layer embeddings and run k-means clustering, check the number of clusters and cluster separation, then use each cluster's unique label as an equivalence class (Algorithm 1).
    • Encoding: Prefer robust features (as defined by Madry et al. [12]) or robust input encodings from another modality; when lacking, use an adversarially trained robust classifier to verify the invariant on fragile features (§ III-B).

    Components of the hierarchical classifier

    • Intermediate classifier: F_intermediate : X → K, which routes inputs to equivalence classes; its implementation corresponds to the specific invariant and can be an adversarially trained/smoothed robust classifier or accept robust feature inputs; the authors state it is difficult to attack (§ IV-A).
    • Leaf classifiers: One Fi : X → Yi per equivalence class, predicting only over labels within the class; at inference only one leaf classifier is invoked, resembling a decision tree overall (§ IV-A).
    • Two ways to obtain leaves: renormalization—take the full-label baseline classifier, discard out-of-class label probabilities, and renormalize; retraining—train from scratch using only in-class samples. The authors state the former is more computationally economical and only enlarges the margin, while the latter may reduce the certified radius for some inputs (§ VI-A3).

    Robustness analysis

    • Margin and number of labels: The certified radius is proportional to R ≥ σ/2(Φ−1(pA) − Φ−1(pB)), i.e., the larger the difference between the lower bounds of the top-class and runner-up-class probabilities, the larger the radius (Equation (1), Cohen et al. [10]).
    • Anti-monotonicity: Define H_{c,α}(x, L) = α(x)_c − max_{l∈L∧l≠c} α(x)_l; the authors point out that shrinking L increases H, so reducing the label set can increase the robust radius (§ IV-B).
    • Overall guarantee: The robust guarantee of the hierarchical classifier equals the minimum of the guarantees of its constituent classifiers—an untargeted attack need only break one of them (§ IV-C).

    Success determination and evaluation methods

    • Robustness metrics: The randomized smoothing certified radius and certified accuracy of Cohen et al. [10] are used; § V additionally uses PGD adversarial accuracy (§ II-C, § V).
    • Theoretical support: Under the statistical setting of Tsipras et al. [13] and the PRF-based computational setting of Degwekar et al. [16], the authors state that imposing invariants makes robust classifiers learnable or improves the trade-off (§ III-C, Appendix C).
  4. What experiments did the paper conduct?

    Validated on the MNIST/CIFAR-10 toy settings and the two case studies GTSRB/LibriSpeech; certified radius and certified accuracy improve in most cases, with MNIST as an exception.

    Experimental Setup

    • Models evaluated: GTSRB uses ResNet-20 (root and leaf; the leaf is a smoothed classifier); point cloud attacks use PointNet (about 98% test accuracy); CIFAR-10 uses Wide Resnet 34 layers (YOPO-5-3) and ResNet-110 (smoothing); MNIST uses Small CNN (YOPO-5-10); LibriSpeech uses a CNN with 3 convolutional layers + FC + softmax (§ V, § VI, Appendix B).
    • Datasets: MNIST; CIFAR-10; GTSRB (51,840 cropped road sign images, 43 classes); KITTI (12,919 images; after about 40 hours of manual annotation, 3,138 road sign crops and corresponding LiDAR point clouds were obtained; after data augmentation, 3,642 cropped point clouds); LibriSpeech (105,894 audio clips, 1,172 speakers: 606 male, 566 female, downsampled to 4 KHz and truncated to 3 seconds) (§ VI-A1, § VI-B1).
    • Baselines: a flat classifier trained on the full label space; for CIFAR-10, the benchmark published by YOPO [60] is also listed (Table II).
    • Metrics: certified radius CR (mean ± standard deviation), certified accuracy (CA), natural accuracy, PGD adversarial accuracy (Adv), Budget (a constrained attacker attacks only one classifier).
    • Attacks and parameters: § V uses PGD-20/PGD-40 (l∞, CIFAR-10 ε=8/255, MNIST ε=0.47); smoothing noise σ is 0.25/0.5/1.0 for vision and 0.025/0.05/0.1 for audio (Appendix B).
    • Sample size: GTSRB and LibriSpeech evaluate each equivalence class with 3,000 inputs; the pointwise analysis in Appendices F–H uses about 200 inputs; the paper does not report the number of repetitions (§ VI-A3, § VI-B, Appendix F).

    Main Results

    Certified radius CR and certified accuracy CA, hierarchy method (renormalization) versus baseline (according to Table IV–VII):

    Swipe horizontally to read the table

    Dataset/equivalence classσBaseline CRHierarchy CRBaseline CAHierarchy CA
    GTSRB circle (Table IV)0.251.083±0.5791.090±0.57577.74%79.25%
    GTSRB circle (Table IV)1.02.327±1.4542.400±1.56031.69%35.47%
    GTSRB triangle (Table V)0.251.107±0.5511.127±0.54771.60%76.45%
    GTSRB triangle (Table V)1.02.223±1.2592.233±1.17220.41%32.78%
    LibriSpeech female (Table VI)0.0250.1233±0.04240.2554±0.042988.80%89.62%
    LibriSpeech female (Table VI)0.10.2612±0.15790.5688±0.157624.79%24.96%
    LibriSpeech male (Table VII)0.0250.1351±0.03810.2590±0.037992.67%92.65%
    LibriSpeech male (Table VII)0.10.2883±0.15630.5562±0.159236.92%36.40%
    • The authors state that the hierarchy method outperforms the baseline in both CR and CA under all reported settings in both domains, and that on audio the certified radius is 2.07× (female) and 1.92× (male) that of the baseline (§ VI-B).
    • The authors state that for the male equivalence class at σ=0.025, CA changes from 92.67% to 92.65%, essentially unchanged (Table VII).
    • The authors state that after adding a second invariant (position), the certified radius of SpeedLimit signs is further improved without affecting accuracy (Figure 8; the figure shows only three bars for σ=0.25/0.5/1, the vertical axis is % Improvement, and specific values are not labeled on the figure).

    Attack Evaluation on Robust Features (Point Clouds)

    • What was tested: Xiang et al. [66]'s method was used to attack a PointNet shape classifier (the target model is the attacked PointNet itself), evaluating the perceptibility of l2 perturbations (§ VI-A2).
    • Results: attack success rates are 91.33% for circle, 32.00% for diamond, 98.00% for triangle, and 89.33% for inverted triangle; average l2 perturbation is 0.386–1.059, corresponding to a depth change upper bound of 1.158–3.177 cm (Table III).
    • Authors' interpretation: although the perturbations are small, they are perceptible after physical realization; the clustering attack fails for most target classes; active attacks against LiDAR are outside the threat model (§ VI-A2).

    Toy Settings (MNIST/CIFAR-10, Adversarial Training)

    • CIFAR-10: hierarchy method natural accuracy 85.37%, Adv 45.46%, Budget 59.11%; baseline 83.68%/38.77%/38.77%; YOPO benchmark 83.99%/44.72%/44.72% (Table II); the authors state that Adv is 6.69 percentage points higher than the baseline (§ V-C).
    • MNIST (negative result): hierarchy method Adv 84.72% is lower than the baseline 87.13%, a decrease of 2.41 percentage points; natural accuracy 97.51% is also lower than the baseline 98.79%; but Budget 90.42% is higher than the baseline 87.13% (Table I).
    • Authors' interpretation: data-driven invariants are not necessarily the most suitable for improving robustness, and some invariants may do more harm than good (§ V-C).

    Other Ablations and Analyses

    • Root classifier accuracy: CIFAR-10 root 96.57% natural / 84.25% adversarial; MNIST root 98.51% / 93.68% (§ V-B).
    • CIFAR-10 component classifiers: Non-living leaf 93.48%/69.04%, Living leaf 80.72%/40.78% (Table X).
    • CIFAR-10 smoothed leaves (Appendix E): for Non-living at σ=1.0, CR 1.2270→1.5013, CA 65.23%→74.74% (Table XI); for Living at σ=1.0, CR 0.9319→1.0750, CA 42.68%→53.80% (Table XII).
    • Retraining comparison (Appendix H, n≈200): for individual points, renormalization improves by nearly 7× and retraining by about 20×, but retraining has more points with decreased robustness, so § VI uses renormalization (Figure 15, 16).
    • Theoretical toy example (Appendix C): when d=200, the invariant restricts the first k features from being perturbed; the authors state that a sufficiently small k (relative to d) can also make adversarial accuracy exceed 90% (Figure 9).
  5. What are some points that can be further explored?

    The authors list four open problems: invariant extraction, interactions among multiple invariants, architecture search, and the pros and cons of invariants.

    Limitations and Future Directions Noted by the Authors

    • Whether invariants always exist: the authors state that it is unclear whether their extraction method applies to all classification tasks, and that an algorithm for efficiently extracting invariants for arbitrary tasks is an open problem (§ VII-A).
    • Value of interpretable invariants unclear: whether it is necessary for humans to be able to interpret them is unclear, and imposing subjectivity may harm learning and raise fairness issues (§ VII-A).
    • Interaction of multiple invariants: whether each invariant contributes equally to robustness, how to combine them into a hierarchy, and the ordering of invariants are all open problems (§ VII-B); ordering intermediate classifiers to maximize accuracy is left as future work (§ IV-A).
    • Architecture construction not necessarily unique: whether a tree-shaped hierarchy is the only approach is unclear, and combining with neural architecture search is an interesting future direction (§ VII-C).
    • Invariants may be harmful: DNNs are by default already invariant to various task-related variations, and imposing invariants believed to be beneficial may introduce new vulnerable regions; distinguishing beneficial/harmful invariants is future work (§ VII-D).

    Scope of Experiments

    • There are 4 tasks evaluated: MNIST, CIFAR-10 (toy settings, § V), GTSRB+KITTI road sign classification, and LibriSpeech speaker recognition (case studies, § VI).
    • The certification method uses only randomized smoothing of Cohen et al. [10]; the adversarial training experiments use YOPO-5-3/YOPO-5-10 to approximate PGD-20/PGD-40 (Appendix B).
    • The main evaluation uses n=3,000 per equivalence class; the pointwise analysis in Appendices F–H uses n≈200; the paper does not report the number of repetitions or confidence intervals.
    • The threat model is limited to white-box, passive, p-norm bounded perturbations; non-p-norm physical attacks and active attacks against LiDAR are explicitly excluded (§ II-B, § VI-A2).
    • The paper does not report evaluation of adaptive attacks (attacks specifically targeting the hierarchical structure); § VI-A2 only reuses the existing point cloud attack of Xiang et al. [66].
  6. Summarize the main content of the paper

    Use task invariants to split classification into a hierarchy, improving both certified radius and certified accuracy on GTSRB and LibriSpeech.

    A defense paper on adversarial robustness whose core claim is to explicitly embed invariants inherent to the classification task (such as road sign shape and speaker gender) into the classification pipeline, so that robustness and accuracy are no longer in conflict.

    • Problem: p-norm defenses depart from task semantics and harm accuracy, and the literature contains results showing that the two cannot be achieved simultaneously (§ I).
    • Method: Invariants partition the label space into equivalence classes; the intermediate layer of the hierarchical classifier enforces the invariant using robust features (such as LiDAR point clouds) or randomized smoothing, and the leaf layer predicts within the equivalence class; the reduced number of labels enlarges the margin between top and runner-up, thereby increasing the certified radius, and the overall guarantee is the minimum of the guarantees of the components (§ III, § IV).
    • Road sign case: The root classifier predicts shape from LiDAR point clouds (98.01% test accuracy); for the circular equivalence class at σ=1.0, CR rises from 2.327 to 2.400 and CA from 31.69% to 35.47% (Table IV); adding the location invariant further increases the certified radius (Figure 8).
    • Speaker case: Under the gender invariant, the certified radius is 2.07× (female) and 1.92× (male) the baseline, with CA roughly unchanged or slightly higher (Table VI, VII).
    • Exceptions: On MNIST, the data-driven invariant lowers adversarial accuracy by 2.41 percentage points relative to the baseline (Table I); the l2 attack success rate on PointNet reaches up to 98.00% (triangle), but the authors state that the perturbation would be perceptible after physical realization and is outside their threat model (Table III).
    • Authors' conclusion: Invariants can be provided by domain knowledge or real-world constraints and can improve adversarial robustness with minimal computational overhead; automatic extraction of invariants, ranking of multiple invariants, and their potential harms are left for future work (§ VII, § IX).

Related papers

Show 3 more papers
Read originalarxiv.org