S2O: Enhancing Adversarial Training with Second-Order Weight Statistics

S2O treats model weights as random variables and enhances adversarial training by optimizing the second-order statistics of the weights. The method relaxes the assumption of independent weight statistics in existing PAC-Bayesian frameworks, and on this basis derives a tighter PAC-Bayesian robust generalization bound. Experiments show that S2O used alone can improve the robustness and generalization of neural networks, and can also be combined with other SOTA adversarial training methods.