Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks

1. 01What problem does this paper attempt to solve?

   How to detect adversarial example inputs to DNNs at low cost without modifying the model itself.

   Core problem: how to detect adversarial example inputs at low cost without modifying the DNN model.

   • Problem setting and importance: DNNs are widely used in security-sensitive tasks such as malware classification and face recognition, but attackers can force misclassification through imperceptible pixel modifications. The authors argue that detecting attack attempts is as important as predicting correct outputs: a local classifier can alert or take fail-safe measures upon detecting adversarial inputs; an online classifier operator can use this to identify malicious clients.
   • Shortcomings of existing methods: According to the paper, prior defenses mostly focused on modifying the DNN model itself (e.g., adversarial training, gradient masking), which either had limited effectiveness or was computationally expensive; sample-statistics-based detection methods require many samples and cannot detect a single adversarial example.
   • Core observation: The authors argue that the input feature space is often overly large, leaving attackers a great deal of freedom to construct adversarial examples; "squeezing" multiple feature vectors to the same sample can reduce the attacker's search space.
   • Proposed approach: Propose the feature squeezing detection framework—compare the model's predictions on the original input and the squeezed input, and classify as an adversarial example if the difference exceeds a threshold; explore two types of squeezing methods, reducing color bit depth and spatial smoothing (local and non-local), and combine them into a joint detection framework.
   • Threat model:
     • Attacker goal: generate inputs misclassified by the model, in targeted (specified target class) and untargeted (any incorrect class) forms, with perturbations constrained by L∞, L2, L0 norms.
     • Attacker knowledge: full access to the trained target model (white-box attack techniques).
     • Attacker capability: cannot affect the target model itself; static attackers do not know the defender is doing feature squeezing, while adaptive attackers (Section 5-D) fully understand the defense.
     • Victim systems: image classification DNNs (7-layer CNN on MNIST, DenseNet on CIFAR-10, MobileNet on ImageNet).

   Collapse this answer

2. 02What related research exists?

   Related work falls into four categories: adversarial training, gradient masking, input transformation, and adversarial example detection; this paper belongs to prediction-inconsistency-based detection.

   Adversarial Training

   • Goodfellow et al., Szegedy et al., etc.—add adversarial examples together with their true labels to training. The paper notes that this is costly (at least doubling training cost) and that effectiveness depends on the attack method used during training.
   • Madry et al.—use BIM with random starts (i.e., PGD attack) for adversarial training and increase model capacity; the authors claim it can provide robustness guarantees on MNIST, but the paper notes that it performs poorly on CIFAR-10. This paper compares combinations with this method and FGSM adversarial training in Section 4-B.

   Gradient Masking

   • Gu et al.—add a Frobenius norm penalty on the layer-wise Jacobian to the training objective; the paper notes that this penalty sacrifices model accuracy on multiple tasks.
   • Papernot et al.—defensive distillation, training with smoothed labels and replacing the softmax layer; the paper notes that subsequent research found it cannot resist JSMA variants with the division trick and black-box attacks.

   Input Transformation

   • Bhagoji et al.—use PCA dimensionality reduction as a defense; the paper notes that PCA destroys the image spatial structure, making mainstream CNN models no longer applicable.
   • Meng and Chen (MagNet)—train an autoencoder as an image filter; Osadchy et al.—independently propose using binary filtering and median smoothing to remove adversarial perturbations, and advocate attacking such defenses by increasing perturbation strength. The paper notes that this paper focuses on the detection task and compares performance with MagNet in Table 6.

   Adversarial Example Detection

   • Sample statistics: Grosse et al. (maximum mean discrepancy statistical test), Feinman et al. (kernel density estimation). The paper notes that the former requires many samples and cannot detect a single sample, and the latter is computationally expensive; experiments in both show that statistical methods perform worse than other methods.
   • Trained detectors: Metzen et al. (attach a CNN detection branch to an intermediate layer of the original model), Grosse et al. (add an "adversarial" class). The paper notes that such methods require many adversarial samples for training and easily overfit to specific attacks.
   • Prediction inconsistency: Feinman et al. (Bayesian neural network uncertainty, using dropout to produce multiple predictions), Meng and Chen (compare prediction vectors on the original image and the autoencoder-filtered image). The paper notes that this paper belongs to this category, but uses "hard-coded" transformations rather than a trained autoencoder, making the training phase cheaper.

   Baselines and benchmarks: Comparison baselines are the MagNet detector (Table 6) and FGSM/PGD adversarial training (Figure 4); experimental benchmarks are the three image classification datasets MNIST, CIFAR-10, and ImageNet.

   The authors position this paper as: a detection framework that does not modify the model, is low-cost, and can be combined with other defenses, belonging to prediction-inconsistency-based detection methods, but using simple hard-coded squeezing transformations instead of trained filters.

   Collapse this answer

3. 03How does the paper address this problem?

   Compare the model's prediction differences between the original input and the compressed input, use an L1 distance exceeding a threshold to determine adversarial examples, and jointly detect using the maximum value across multiple compressors.

   Overall idea: use feature squeezing to reduce the input space available to an attacker, then compare the difference between the same model's prediction vectors for the original input and the squeezed input; a large difference indicates an adversarial example (Figure 1).

   Problem setting

   • An adversarial example is defined as: adding a constrained perturbation to a correctly classified seed sample x to obtain x', causing the model to output an error. A targeted attack requires g(x′) = t ∧ Δ(x, x′) ≤ ϵ; an untargeted attack requires the output to differ from g(x); Δ is the L∞, L2, or L0 norm, and ε limits the perturbation strength.
   • Detection must satisfy two properties: squeezing reverses the perturbation effect on adversarial examples; squeezing does not noticeably affect model predictions on legitimate samples.

   Squeezing method one: reducing color bit depth

   • Grayscale images (MNIST) have 8 bit per pixel (256 values), and color images have 24 bit per pixel; the authors hypothesize that reducing bit depth can reduce adversarial opportunities without harming recognition.
   • Implementation: when reducing to i bit, multiply pixel values by 2^i − 1, round, then divide back; input and output keep the [0, 1] value range, and the model needs no modification; MNIST uses 1-bit binary filtering (0.5 as the cutoff), and color datasets test 4-bit, 5-bit, etc.

   Squeezing method two: spatial smoothing

   • Local smoothing (median smoothing): replace the center pixel with the median within a sliding window; the window size is configurable (experiments use 2×2, 3×3), and edges use reflect padding; the authors consider it particularly effective against the salt-and-pepper-like noise produced by L0 attacks. Implemented with SciPy.
   • Non-local smoothing (non-local means): find similar image patches within a large search window, and replace the center patch with a Gaussian-kernel-weighted average; denoted non-local means (a-b-c), where a is the search window, b is the patch size, and c is the filtering strength (experiments use 11-3-2, 11-3-4, 13-3-2, 13-3-4). Uses OpenCV's fast non-local means, denoising the L and AB components separately in CIELAB space.

   Detection method and joint framework

   • The detection score of a single squeezer is the L1 distance between the original prediction vector and the squeezed prediction vector: score(x, x_squeezed) = |g(x) − g(x_squeezed)|₁, with a value range from 0 to 2.
   • Joint detection takes the maximum of the scores of multiple squeezers (scorejoint = max), because different squeezers are effective against different types of perturbation; the authors note that max also selects the squeezer most damaging to legitimate inputs, which may raise the false positive rate.
   • Threshold selection: train only on legitimate samples, and choose the threshold that keeps the false positive rate on legitimate samples at no more than 5%, without relying on adversarial examples, so the training-stage cost is low.

   Evaluation method

   • Detection rates are reported separately on three groups of samples: successful adversarial examples (SAE), failed adversarial examples (FAE), and legitimate samples (false positive rate); unless otherwise specified, "detection rate" means the detection rate on SAE.
   • ROC-AUC is also reported (FAE excluded in the calculation); when comparing with MagNet, the false positive rate is aligned to the same level (MNIST 0.40%, CIFAR-10 1.28%).

   Collapse this answer

4. 04What experiments did the paper conduct?

   The robustness and detection rate of the compressor were measured on three datasets and 11 attacks, and its combination with adversarial training, comparison with MagNet, and adaptive attacks were evaluated.

   Experimental setup

   • Models tested: 7-Layer CNN on MNIST (top-1 accuracy 99.43%), DenseNet on CIFAR-10 (94.84%), MobileNet on ImageNet (top-1 68.36%, top-5 88.25%) (Table 1).
   • Attacks: 11 in total—untargeted FGSM, BIM, DeepFool, and targeted JSMA, CW∞, CW2, CW0 (each with Next and LL targets); FGSM/BIM/JSMA implemented with Cleverhans, DeepFool and CW attacks with the original authors' implementations. DeepFool was excluded on MNIST (generated images are unrecognizable to the human eye), and JSMA had no results on ImageNet due to insufficient memory.
   • Seed samples: the first 100 correctly classified samples from the test (or validation) set of each dataset; attack parameters were tuned to generate high-confidence adversarial examples.
   • Detection datasets: equal numbers of legitimate and adversarial samples—MNIST 2,000 in total (1,000 legitimate + 1,000 adversarial), CIFAR-10 2,200 in total, ImageNet 1,800 in total; randomly split in half into training (threshold selection) and validation.
   • Metrics and threshold: detection rate (by default on successful adversarial examples), false positive rate (threshold selected so that false positives on legitimate samples in the training set do not exceed 5%), ROC-AUC (excluding failed adversarial examples).
   • Hardware: i7-6850K CPU, 64GiB memory, GeForce GTX 1080.

   Main results

   Best joint detector results summarized in Table 5:

   Swipe horizontally to read the table

   Dataset	Detector configuration	Threshold	Detection rate (SAEs)	Detection rate (FAEs)	FPR	ROC-AUC
   MNIST	Bit Depth (1-bit) + Median (2×2)	0.0029	98.15%	20.00%	3.98%	99.44%
   CIFAR-10	Bit Depth (5-bit) + Median (2×2) + Non-local Means (13-3-2)	1.1402	84.53%	22.22%	4.93%	95.74%
   ImageNet	Bit Depth (5-bit) + Median (2×2) + Non-local Means (11-3-4)	1.2128	85.94%	25.00%	8.33%	94.24%

   According to Table 5. The FPR for ImageNet is 8.33%, higher than the 5% target; the paper does not explain in the table notes how this value relates to the threshold selection rule.

   • The authors state that joint detection is usually comparable to or better than the best single squeezer: on MNIST the largest drop is CW0 (LL) from 100% to 93%, on CIFAR-10 joint detection is always better than the best single squeezer, and on ImageNet CW2 (Next) drops from 95% to 91% (Table 4).
   • The authors note that detection rates on failed adversarial examples (20%–25%) are much lower than on successful examples, but higher than the false positive rate on legitimate samples.
   • Figure 5 shows that the L1 score peak for legitimate samples is always near 0 and the peak for adversarial samples is always near 2; the authors state that the two distributions are separable.

   Robustness experiments (Section 4, Table 3)

   • What was measured: classification accuracy of the model on adversarial and legitimate samples after each squeezer is used directly as input preprocessing.
   • Results: On MNIST, 1-bit filtering raises accuracy on CW∞ from 0% to 100%, with legitimate sample accuracy dropping only from 99.43% to 99.33%; on CIFAR-10, 2×2 median smoothing raises accuracy on four L0 attacks from 0% to above 75%, but legitimate sample accuracy drops from 94.84% to 89.29%; on ImageNet, median smoothing raises accuracy on CW0 attack from 0% to 85%.
   • Authors' interpretation: Bit depth squeezing is effective against L∞ and L2 attacks but ineffective against L0 attacks (L0 attacks make large changes to a few pixels); median smoothing is most effective against L0 attacks. The authors explicitly state that a single squeezer is not recommended as a standalone defense.

   Combination with adversarial training (Section 4-B, Figure 4)

   • What was measured: on all 10,000 test images of MNIST, compare accuracy of 1-bit filtering, adversarial training (FGSM and PGD, training ε=0.3), and their combination under different attack strengths ε.
   • Results: Against FGSM attack, at ε=0.3, 1-bit filtering accuracy is 93.03%, adversarial training 92.38%; against PGD attack, at ε=0.4, 1-bit filtering 87.60%, adversarial training 14.84%, combined method 94.21%.
   • Authors' interpretation: The combined method usually achieves the highest accuracy or accuracy comparable to the best single method; feature squeezing is nearly cost-free, whereas adversarial training requires generating adversarial examples and retraining for multiple epochs.

   Comparison with MagNet (Table 6)

   • What was measured: at the same false positive rate (MNIST 0.40%, CIFAR-10 1.28%), compare the two detectors using this paper's target models and detection datasets.
   • Results: On MNIST MagNet is higher (all adversarial examples 91.77% vs. 69.08%, successful examples 95.61% vs. 78.75%); on CIFAR-10 feature squeezing is higher (60.87% vs. 50.36%, successful examples 61.88% vs. 50.46%).
   • Authors' interpretation: MagNet's advantage on MNIST does not carry over to CIFAR-10; moreover, MagNet requires training an autoencoder on the entire dataset, which is more costly, and its pipeline is end-to-end differentiable, making it vulnerable to white-box attacks.

   Adaptive attacks (Section 5-D)

   • What was measured: using He et al.'s adaptive attack (adding an L1 prediction distance penalty term to CW2), attacking the MNIST model and the joint detector of Table 5, threshold 0.0029; also tested a countermeasure adding Gaussian randomness to the binary filtering cutoff (mean 0.5, standard deviation 0.0625), and an ensemble-adaptive attack simultaneously attacking three cutoffs 0.4/0.5/0.6.
   • Results: With no perturbation magnitude limit, untargeted/targeted-next/targeted-ll success rates are 68%, 44%, 24% respectively (Figure 7 shows at ε=0.3 the three are 1%, 0%, 0%); with added randomness they drop to 17.0±2.4%, 16.2±2.6%, 7.6±2.2% (10 runs); the ensemble-adaptive attack rises back to 46.5±2.1%, 34.5±3.5%, 28.5±2.1%, but with larger perturbations.
   • Authors' interpretation: The success rate numbers of adaptive attacks cannot by themselves measure robustness, because with unbounded perturbation misclassification can always be forced, but the resulting images are often unrecognizable to the human eye; the authors acknowledge that resistance to adaptive attackers cannot be guaranteed.

   Other ablations and analyses

   • Single-squeezer detection (Table 4): On MNIST, 1-bit filtering achieves 100% detection rate on 7 attacks, but only 55.6%/56.3% on CW0; 3×3 median achieves above 88% detection rate on L0 attacks.
   • Negative results: On CIFAR-10 the best single squeezer detects only 18.75% of FGSM and 55% of BIM; on ImageNet 35.85% and 55.56% (Table 4). The authors speculate this is because these two attacks have larger perturbations, while squeezers are better suited to small perturbations.
   • Threshold and destructiveness (Table 4): On MNIST, a 1-bit filtering threshold of only 0.0005 suffices for 5% false positives, while 3×3 median requires 0.039; the authors state that the lower the threshold, the more sensitive the detector.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors acknowledge that resistance to adaptive attacks cannot be guaranteed, and list directions such as improving joint detection and exploring new squeezing methods.

   Limitations and future directions noted by the authors

   • It cannot be guaranteed that an adaptive attacker cannot break a specific feature squeezing configuration; robustness to adaptive attackers requires further study in future work (Section 1, Section 5-D).
   • Detection rates against FGSM and BIM on color datasets are low; the authors state that understanding the cause and developing squeezing methods effective against such attacks is an important future research direction (Section 5-C).
   • Joint detection uses a single threshold and max combination; the authors plan to study better ways of combining multiple squeezers (e.g., a different threshold for each squeezer) (Section 5-A, 5-C).
   • The detection score uses the L1 norm; the authors consider it worth exploring whether other metrics are better (Section 5-A footnote).
   • This paper experiments with only two simple squeezing methods; the authors consider it worth continuing experiments to find more effective squeezing methods, and mention lossy compression (e.g., JPEG) and dimensionality reduction (e.g., eigenfaces) as two possible directions (Section 3-C).
   • Adaptive attacks were evaluated only on MNIST because the authors did not have a Tensorflow implementation of non-local means filtering at the time (Section 5-D).

   Experimental coverage

   • 3 models tested: 7-Layer CNN on MNIST, DenseNet on CIFAR-10, MobileNet on ImageNet (Table 1), all image classification models.
   • Attacks cover 11 configurations (FGSM, BIM, DeepFool, JSMA, CW∞/CW2/CW0 each with two targets), each attack using 100 seed samples (Section 4).
   • Comparison baselines for detection evaluation are MagNet (on both MNIST and CIFAR-10 datasets) and FGSM/PGD adversarial training (MNIST only).
   • Adaptive attack experiments were conducted only on MNIST, with the same 100 images as seed samples as in Section 4; results for randomized adversarial measures are the mean and standard deviation of 10 runs.
   • The paper does not report the number of repetitions beyond detection threshold selection, nor confidence intervals for ROC-AUC; apart from the 10 runs of adaptive attacks, the number of repetitions is not stated for the remaining experiments.

   Collapse this answer

6. 06Summarize the main content of the paper

   Feature squeezing detects adversarial examples by comparing prediction differences between the original image and the compressed image, with a joint detection rate of 98.15% on MNIST, but it does not guarantee resistance to adaptive attacks.

   A defense paper from NDSS 2018 that proposes the feature squeezing framework to detect adversarial examples against deep neural networks.

   • Problem to solve: Prior defenses mostly modified the model itself, with limited effectiveness or high computational cost; the authors argue that the overly large input feature space gives attackers too much freedom, and that detecting adversarial inputs itself is valuable.
   • Method highlights: Compress the input by reducing color bit depth and spatial smoothing (median smoothing, non-local means), compare the L1 distance between the model's prediction vectors on the original input and the squeezed input, and classify as an adversarial example if the distance exceeds a threshold; multiple squeezers take the maximum score for joint detection, and the threshold is selected using only legitimate samples so that the false positive rate does not exceed 5%.
   • Main results:
     • Against 11 static attacks, the best joint detector's detection rate on successful adversarial examples: MNIST 98.15% (FPR 3.98%), CIFAR-10 84.53% (FPR 4.93%), ImageNet 85.94% (FPR 8.33%) (Table 5).
     • As input preprocessing, 1-bit filtering restores accuracy on MNIST after CW∞ attack from 0% to 100%, with legitimate sample accuracy dropping by only 0.1 percentage points (Table 3).
     • Combined with adversarial training, accuracy is 94.21% at PGD ε=0.4, higher than either method alone (Figure 4).
     • At the same false positive rate, detection rate on CIFAR-10 is higher than MagNet (60.87% vs. 50.36%), and on MNIST is lower than MagNet (Table 6).
     • Adaptive attacks achieve near 0% success rate at ε=0.3, but untargeted success rate reaches 68% with unbounded perturbation (Figure 7).
   • Authors' conclusion: The authors state that feature squeezing is simple, low-cost, and composable with other defenses, making the adaptive attacker's task more difficult; but they explicitly acknowledge that resistance to adaptive attacks cannot be guaranteed, and that the reason for the low detection rates against FGSM/BIM on color datasets remains to be studied.

   Collapse this answer

Related papers