Solving adversarial examples requires solving the exponential misalignment between machine and human perception
Solving adversarial examples requires solving exponential misalignment
Quick read
Based on the paper's full text (AI-generated); refer to the original
Stanford researchers propose that the perceptual manifold dimension of neural networks for class concepts is orders of magnitude higher than that of natural images for humans, and that this exponential misalignment is the geometric origin of adversarial examples. The perceptual manifold refers to the set of all inputs that the network classifies as a certain class with high confidence; on CIFAR10, the perceptual manifold of models reaches 3060/3072 dimensions, on CLIP about 135,000/150,528 dimensions, whereas natural image concepts are only about 20 dimensions. Because volume grows exponentially with dimension, the machine perceptual manifold almost fills the input space, and any input is extremely close to the manifold of some class, where a bounded perturbation that successfully attacks is very likely to exist nearby. The authors verify two predictions on 18 networks with different robust accuracy on RobustBench: robust accuracy and the distance from random points to the manifold are both negatively correlated with manifold dimension; even the most robust networks still exhibit exponential misalignment, and only a few manifolds whose dimension is close to human concepts can sample samples recognizable to the human eye. The paper thus connects adversarial examples with alignment research, arguing that adversarial robustness cannot be achieved without dimensional alignment between machine and human perception.
- Adversarial attacks (input perturbations imperceptible to the human eye that nonetheless fool neural networks) have remained a persistent failure mode of machine learning for more than a decade, and their geometric cause is unknown. The authors hope to explain, from the perspective of the dimensional mismatch between machine and human perception, why adversarial examples exist and why they are difficult to eliminate.
- The authors define the network's perceptual manifold (PM) for a class of concepts as all inputs that the network judges with high confidence to be of that class, sample the PM from random noise using projected gradient ascent, and estimate its dimension using Participation Ratio and 2NN, then compare it with the dimension of natural image concepts; they also use an ellipsoid toy model to derive the relationship between the PM dimension and the distance from a random point to the PM.
- On CIFAR10, CLIP and ImageNet, the machine PM dimension is several orders of magnitude higher than that of natural concepts (e.g., on CIFAR10, about 3060/3072 versus about 20), and CLIP's PM dimension for meaningless prompts is no different from that for meaningful concepts. On 18 networks with different robust accuracies, robust accuracy is negatively correlated with PM dimension, and the distance from a random point to the PM increases as the PM dimension decreases; even the most robust models still have exponential mismatch, and only a few PMs whose dimension is close to that of natural concepts show samples aligned with human perception.
- The authors propose that adversarial robustness may not be achievable without aligning the machine and human PM dimensions, and suggest extending the analysis from vision to language; the material does not provide a specific discussion of limitations.
The paper attributes adversarial examples to the exponential dimensional misalignment between machine and human perceptual manifolds, and verifies on 18 networks the negative correlation between manifold dimensionality and robust accuracy.