Neural Network Robustness Verification for Semantic Perturbations
Towards Verifying Robustness of Neural Networks Against Semantic Perturbations
The authors propose Semantify-NN, which converts semantic perturbation verification into Lp-norm verification by inserting semantic perturbation layers (SP-layers); the authors state that the certificates are 2-3 orders of magnitude larger than naive conversion.
Verifier's perspective: given a trained neural network and a continuously parameterized semantic threat model (hue, saturation, lightness, brightness/contrast, rotation, etc.), find the maximum verifiable perturbation radius δ, guaranteeing that the top-1 prediction remains unchanged for all perturbed images within the attack space; discretely parameterized perturbations (translation, occlusion) are verified directly by enumeration.
- Existing neural network robustness verification mainly targets the Lp-norm threat model and cannot handle semantic perturbations such as hue and rotation (whose Lp-norm is large); the authors aim to verify network robustness under such semantic attacks.
- Proposes Semantify-NN: insert semantic perturbation layers (SP-layers) before the input layer of any network, writing semantic transformations in neural network form, so that any Lp-norm verifier can be connected; and proposes the implicit input splitting refinement technique to tighten the certificate.
- Verifying perturbations such as hue, saturation, lightness, and rotation on MLP/CNN for MNIST, CIFAR-10, GTSRB; the authors report that without refinement certificates are about 2-3 orders of magnitude larger than the naive Lp-norm baseline, adding input splitting improves them by a further 100-300%, and rotation certificates are close to the grid attack upper bound.
- The authors point out that the semantic attack space is non-convex, and a single certification cannot verify a large interval, so input space splitting is needed; explicit splitting has computational cost that grows linearly with the number of splits, and rotation would require about 40000 splits to be feasible, so implicit splitting is proposed; refinement at the feature layer yields little benefit and its overhead grows exponentially with the number of nodes.
- Evaluated models
- CIFAR, MLP 6 × 2048CIFAR, MLP 5 × 2048CIFAR, CNN 5 × 10GTSRB, MLP 4 × 256GTSRB MLP 4 × 256 sem advMNIST, MLP 2× 1024MNIST, MLP 2× 1024 l∞ advMNIST, MLP 3× 1024MNIST, MLP 4× 1024MNIST, CNN LeNetMNIST, CNN 4 × 5
- Benchmarks
- MNISTCIFAR-10GTSRB
- Metrics
- averaged certified bounds(语义扰动证书)attack upper bound(grid search)runtime
Researchers propose Semantify-NN, a model-agnostic neural network robustness verification method, to address semantic adversarial attacks such as color shifts and lighting adjustments that produce large ℓp-norm perturbations, threats beyond the capability of existing verification tools that mainly target ℓp-norm threat models. The method only requires inserting semantic perturbation layers (SP-layers) at the input layer of any model, allowing existing ℓp-norm-based verification tools to be reused to certify the model's robustness to semantic perturbations. The paper gives design examples of semantic perturbations for image classification such as hue, saturation, value, brightness, contrast and rotation, and proposes an efficient refinement technique that further significantly improves the semantic certificate. Experiments on multiple network architectures and datasets show that Semantify-NN's verification performance is better than verification frameworks that naively convert semantic perturbations into ℓp norms, and can support robustness verification for a wide range of semantic perturbations. Code is open-sourced at https://github.com/JeetMo/Semantify-NN。
Paper analysis
What problem does this paper attempt to solve?
Existing verification methods can only handle the Lp-norm threat model; the authors propose Semantify-NN to verify robustness to semantic perturbations.
Core question: how to verify the robustness of neural networks to semantic perturbations (such as hue, rotation, brightness adjustment), whereas existing verification tools can only handle Lp-norm bounded perturbations.
- Problem setting: The authors point out that semantic adversarial attacks such as hue shift, lighting adjustment, and rotation produce large Lp-norm perturbations in RGB space, exceeding the capability of Lp-norm verification methods.
- Shortcomings of existing methods: According to the paper, directly converting semantic perturbations into Lp-norm in RGB space and then using existing verifiers yields very loose certificates; the experimental section states that such baselines "give very poor results in all Tables".
- Core observation: Continuously parameterized semantic perturbations (such as rotation angle taking real values) cannot be enumerated; the authors use the example of grid search over rotation angle to explain: an attack fails at 30° and at 30.00009°, which does not rule out an angle in between that can fool the classifier, so a verification algorithm is needed to give a certificate.
- Paper's approach: Proposes Semantify-NN, a model-agnostic general verification framework: design semantic perturbation layers (SP-layers) inserted at the input layer of any network, converting the semantic threat model into an Lp-norm threat model, then connect to any Lp-norm verifier; and propose an input splitting refinement technique to tighten the certificate.
- Threat model:
- Goal: find the maximum verifiable semantic perturbation radius δ such that the top-1 prediction remains unchanged for all perturbed images within the attack space ΩA(x) (Equation (1)).
- Perturbation types: continuously parameterized (hue, saturation, lightness, brightness/contrast, rotation) and discretely parameterized (translation, occlusion), with distance measured by the Lp-norm of the semantic parameters, p = ∞ in Table 1.
- Victim system: trained K-class image classification neural networks (MLP, CNN).
- Verification is attack-agnostic and does not depend on a specific attack method.
What related research exists?
Related work falls into three groups: Lp-norm verification, semantic adversarial attacks, and semantic/geometric robustness verification; the authors position this paper as more general.
Lp-norm verification methods
- Kolter & Wong [11], Dvijotham et al. [4], Wong et al. [22] — verification methods based on solving convex relaxations.
- Weng et al. [20], Zhang et al. [23], Singh et al. [16, 19, 17, 3, 18] — design tractable linear bounds for activation functions and propagate them layer by layer; the experiments in this paper use two open-source verifiers, [23] (related to CNN-Cert) and [3] (CNN-Cert).
Verification of semantic/geometric perturbations
- Singh et al. [19] — consider linear brightness and contrast transformations; the paper points out that these still belong to the Lp-norm threat model, whereas this paper handles nonlinear color space transformations.
- Singh et al. [18] — handle rotation attacks with Lp-norm methods; the paper states that this paper's refinement technique can improve certificates by about 50-100% on average, and the setting is more general.
- Balunovic et al. [1] — verify geometric transformations (rotation, translation, scaling, shearing), brightness/contrast, and interpolation, based on sampling and requiring solving linear programs; the paper states that this paper's framework is more general and more efficient.
- Hamdi & Ghanem [7] — evaluate semantic robustness using semantic graphs; the paper points out that they do not address how to develop semantic robustness verification.
Semantic adversarial attacks
- Hosseini & Poovendran [8] — fix value in HSV space, minimize saturation change, and fully exploit hue change to generate semantic adversarial examples.
- Liu et al. [13] — a differentiable renderer propagates pixel gradients to lighting and geometric parameters; Bhattad et al. [2] — texture and shading introduce large Lp-norm but visually imperceptible perturbations; Joshi et al. [9] — use an encoder-generator network to generate semantic adversarial examples by attribute.
- Engstrom et al. [5], Fawzi & Frossard [6] — simple rotation or translation can cause misclassification; this paper directly uses the MNIST/CIFAR models released by [6].
Baselines and benchmarks: The baselines are converting the semantic perturbation range into RGB space and then performing binary search with an L∞-norm verifier (Naive), and a Weighted-eps version that allows a different perturbation amount per pixel; the attack upper bound uses grid search. The datasets are MNIST, CIFAR-10, GTSRB.
Authors' positioning: SP-layers allow any Lp-norm verifier to handle nonlinear semantic transformations, refinement has low overhead, and the overall framework is more general and efficient than [1] and has a more general setting than [19].
How does the paper address this problem?
SP-layers write semantic transformations as network layers that can connect to any Lp-norm verifier, and then use implicit input splitting to tighten the certificate.
Overall idea of Semantify-NN: represent the semantic perturbation function g as neural network layers (SP-layers) composed of commonly used activation functions, inserted before the input layer of the original network, making the semantic verification problem equivalent to the Lp-norm verification problem for the new network f sem = f ◦ gx, thereby reusing any Lp-norm verifier.
Problem formalization
- For threat model A and sample x, define the attack space ΩA(x) and distance dA; the goal is to find the maximum δ such that minx′ ∈ ΩA(x), dA(x′,x)≤ δ minj ≠ c fc(x′) − fj(x′) > 0, i.e., the correct-class logit remains the largest after perturbation (Equation (1)).
- The semantic attack space is parameterized by a function g: ΩkA(x) = {g(x, ε1, …, εk)}, and the distance dA is the Lp-norm of the semantic parameter vector; Table 1 lists the parameter dimension k for each perturbation (e.g., rotation k=1, brightness/contrast k=2), with p = ∞.
- After adding SP-layers, the problem becomes Lp-norm verification of the new network f sem over the semantic space I k (Equation (4)), in the same form as standard Lp-norm verification.
SP-layers design
- Hue: HSL space transformation, h' = (h + εh) mod 6; each RGB channel is written as a piecewise function of h', then expressed using σi(x) = ReLU(x − i) as a network with one ReLU hidden layer (Equation (5)).
- Saturation: s' = s + εs, φs(s') = min(max(s', 0), 1) = σ0(s') − σ1(s') (Equation (6)).
- Lightness: l' = l + εl, φl1 and φl2 are expressed as ReLU combinations (Equation (7)).
- Brightness/contrast: g(x, εb, εc) = min(max((1+εc)·x + εb, 0), 1), i.e., one additional ReLU layer (Equation (8)).
- Rotation: parameterized by rotation angle θ, output pixels are bilinear interpolations of the original image pixels (Equations (9)-(10)); the scaling factor m is highly nonlinear in θ.
- The paper points out that g only needs activation functions with custom linear bounds to be plugged into verifiers such as [23, 19, 16, 3, 20], and is not required to be continuous or differentiable.
Discretely parameterized perturbations
- Translation (2-dimensional parameter: relative position of the top-left corner) and occlusion (3-dimensional parameter: position + side length of the square patch) can be enumerated; SP-layers degenerate to an enumeration operation and are verified directly.
- The authors point out that the computational complexity of enumeration grows combinatorially when multiple discrete attacks are combined.
Input space refinement
- Theorem 3.1: If one certification pass (sharing the same set of linear relaxation values) can verify that all of set S is correctly classified, then all images within the convex hull of S in pixel space can also be verified as correctly classified (proof in Appendix A).
- Remark 3.2: The semantic attack space is non-convex, and its convex hull may contain images of other classes (Appendix Figure 3 gives an example for rotation), so a single certification cannot verify a large interval, and the input space must be split.
- Explicit Input Splitting: Split the parameter interval into small segments and certify each separately, taking the minimum over segments as the certificate; the drawback is that computation time grows linearly with the number of splits.
- Implicit Input Splitting: First compute the pre-activation range of each neuron on each sub-interval, then merge the intervals (taking max/min) to construct the linear relaxation matrices A(k), T(k), H(k) needed for one certification, which only needs to be constructed once; the authors state that this reduces the overhead from S times to 1 time.
- The authors consider that feature-layer refinement (such as the LP/MIP of [19, 17]) yields little benefit and its overhead grows exponentially with the number of nodes, and that input-space refinement already brings the certificate lower bound close to the attack upper bound.
What experiments did the paper conduct?
Verifying 5 types of semantic perturbations on MLP/CNN for MNIST/CIFAR-10/GTSRB; the authors state that certificates are 2-3 orders of magnitude larger than the naive baseline.
Experimental setup
- Models tested: MLP and CNN (LeNet style, 3×3 convolution + two max pooling) on MNIST, CIFAR-10, GTSRB; the MNIST/CIFAR models come from [6], with standard test accuracy 98-99%/60-70%; the GTSRB model is trained from scratch to 94-95%.
- Adversarially trained variants: sem adv (data augmentation according to the corresponding threat model) and l∞ adv (L∞ adversarial training [14]).
- Verifiers: [23] and [3] (open-source, supporting general activation functions for MLP/CNN).
- Baselines: Naive (converting the semantic perturbation range into RGB space and using an L∞ verifier with binary search) and Weighted (different perturbation amounts per pixel).
- Attack upper bound: grid search, with granularity on the same order as the sub-intervals after input splitting; the authors state that this is an upper bound rather than the optimal attack value.
- Sample size: 200 random test images per experiment, random target attack; models trained for 50 epochs.
Main results
Table 2 (HSL space perturbations, average certificates, according to Table 2):
Swipe horizontally to read the table
Network Perturbation Naive Weighted SPL SPL+Refine Attack Grid CIFAR, MLP 6×2048 Hue 0.00316 0.028 0.347 0.974 1.456 CIFAR, CNN 5×10 Hue 0.0067 0.046 0.395 1.794 1.964 GTSRB, MLP 4×256 Hue 0.01477 0.091 0.771 2.310 2.388 CIFAR, MLP 6×2048 Saturation 0.00167 0.004 0.101 0.314 0.342 CIFAR, MLP 6×2048 Lightness 0.00043 0.001 0.047 0.244 0.263 GTSRB, MLP 4×256 Lightness 0.0025 0.005 0.134 0.332 0.365 (Omitted: the two Saturation rows for CIFAR CNN and GTSRB, the Lightness row for CIFAR CNN, and the four sem adv variant rows; full data in Table 2.)
- Authors' interpretation: Lp-norm verification gives loose bounds because the dimension of semantic attacks does not match the dimension of the induced Lp attack; SP-layers explicitly introduce the dimension constraint, and the authors state that certificates improve by 4−50× (w/o refine, vs Weighted).
- Authors' interpretation: input-space refinement brings a further 1.5−5× improvement; HSL transformations are close to linear, and the gap between the certificate lower bound and the attack upper bound becomes fairly small.
- Summary statements in the abstract and introduction: without refinement, certificates are about 2-3 orders of magnitude larger than the baseline, and adding input splitting improves them by a further 100-300% (per the authors).
Rotation perturbations (Experiment II)
- What was measured: rotation angle certificates, first using implicit splits to enlarge the certifiable interval, then tightening with 0.5° explicit intervals + 100 implicit splits (Table 3).
- Results: MNIST MLP 2×1024 improves from 0.627° with 1 implicit split to 46.24° with SPL+Refine, with a grid attack upper bound of 51.42°; GTSRB MLP 4×256 is 31.53° (upper bound 33.43°); CIFAR CNN 5×10 is 10.65° (upper bound 30.81°) (Table 3).
- Authors' interpretation: the rotation transformation is highly nonlinear and the SP-layers linear bounds are very loose; implicit splitting reduces the required number of intervals to a manageable scale; the authors state that SPL+Refine certificates are close to the grid attack upper bound (the gap is relatively larger for the two CIFAR rows).
Brightness/contrast and translation/occlusion (Experiments III, IV)
- Brightness/contrast: fixing the contrast parameter ε at 0.01, 0.03, 0.05, Figure 2 gives the average lower bound curves for brightness perturbations (the pdftotext text does not include the values in the figure).
- Translation/occlusion: the runtime of enumeration verification is affordable, e.g., MNIST MLP 3×1024 translation 4.063 pixels, 0.46 seconds, occlusion 11.080, 0.44 seconds; CIFAR MLP 6×2048 translation 2.755 pixels, 1.36 seconds (Table 4).
Other ablations and analyses
- Number of implicit splits (MNIST MLP 3×1024, 10 images, Table 5): with an explicit interval of 0.3, the certificate is 0.27 with 1 split, rises to 50.0 with 5 splits, and is 50.24 with 20 splits.
- The authors point out that beyond a certain number, adding implicit splits no longer improves the bound (Table 5).
- The larger the explicit interval, the smaller the certifiable radius: with an interval of 3.0, even 20 implicit splits give only 30.0 (Table 5).
- Additional results: the three HSL rows for CIFAR MLP 5×2048 (Table 6) and the three rotation rows for MNIST MLP 4×1024, 3×1024, CNN 4×5 (Table 7), with trends consistent with the main tables.
What are some points that can be further explored?
The authors point out that the non-convex attack space requires splitting and that explicit splitting is costly; experiments cover 3 datasets and 5 types of continuous perturbations.
Limitations and future directions noted by the authors
- The semantic attack space is non-convex, and its convex hull may contain images of other classes; a single certification pass cannot verify a large perturbation interval, so input splitting is necessary (Remark 3.2, Appendix A Remark A.3).
- The computation time of explicit input splitting grows linearly with the number of splits; for rotation on MNIST/CIFAR-10, without splitting into about 40000 pieces no good results can be obtained, making verification computationally infeasible (Section 3.3 Implicit Input Splitting).
- The overhead of feature-layer refinement (LP/MIP) grows exponentially with the number of nodes; the authors observe that input-space refinement already brings the certificate lower bound close to the attack upper bound, so feature-layer refinement would only yield minor improvement (Section 3.3 Remark).
- Appendix B: beyond a certain number, adding implicit splits no longer improves the certificate; the authors state that according to Theorem A.1, certifying this relaxation is harder than certifying all rotated images, which may be the reason why large explicit intervals cannot be certified.
- The computational complexity of enumerating discretely parameterized attacks grows combinatorially when multiple discrete attacks are combined (Section 3.2).
Experimental coverage
- The models tested are MLP and CNN (LeNet style) on MNIST/CIFAR-10/GTSRB, including two adversarially trained variants, sem adv and l∞ adv (Section 4).
- Continuously parameterized perturbations cover hue, saturation, lightness, brightness/contrast, and rotation; discrete perturbations (translation, occlusion) are only verified by enumeration and timed (Table 4).
- The verifiers are [23] and [3]; each experiment evaluates 200 random test images (the ablation in Table 5 uses 10).
- The attack upper bound uses grid search; the authors state that it is not the optimal attack value, and increasing granularity would only yield a tighter upper bound (Section 4).
- The paper does not report the runtime of verifying a single image (only translation/occlusion enumeration is timed), nor does it report a statistical significance test of certificates relative to the attack upper bound.
Summarize the main content of the paper
Semantify-NN uses SP-layers to convert semantic perturbation verification into Lp-norm verification; the authors state that the certificates are 2-3 orders of magnitude larger than the naive baseline.
A paper on neural network robustness verification, aiming to extend verification capability from the Lp-norm threat model to semantic perturbations.
- Problem: Semantic attacks such as hue, saturation, lightness, and rotation produce large Lp-norm perturbations in RGB space, which existing Lp-norm verifiers cannot handle; continuously parameterized perturbations cannot be enumerated, so a verification algorithm is needed to give certificates.
- Method: Semantify-NN designs semantic perturbation layers (SP-layers), writing semantic transformations such as HSL color transformations, brightness/contrast, and rotation as network layers using activation functions such as ReLU, inserted before any classification network, then connected to any Lp-norm verifier; and proposes implicit input splitting, merging the intermediate bounds of sub-intervals into a single certification, which the authors state reduces overhead from S times to 1 time.
- Results (HSL, Table 2): For CIFAR MLP 6×2048 hue perturbation, the certificate improves from 0.028 for the Weighted baseline to 0.347 for SPL and 0.974 for SPL+Refine (attack upper bound 1.456); the authors state that SP-layers bring a 4−50× improvement and refinement a further 1.5−5×.
- Results (rotation, Table 3): For MNIST MLP 2×1024, the certificate rises from 0.627° with 1 implicit split to 46.24° with SPL+Refine, with a grid attack upper bound of 51.42°; the authors state that the certificate is close to the attack upper bound.
- Results (discrete perturbations, Table 4): Translation and occlusion are verified directly by enumeration, with runtimes of 0.14-1.36 seconds, which the authors state are affordable.
- Authors' conclusion: Semantify-NN is a model-agnostic general framework that can support verification of multiple semantic perturbations; the authors also point out that the non-convex attack space prevents a single certification from covering a large interval, and input splitting is a necessary cost.