Fast adversarial training (FAT) for textual adversarial attacks

Fast adversarial training (FAT) methods improve the adversarial robustness of NLP models in scenarios where the attacker's synonym candidates are unknown. Based on the observation that single-step and multi-step gradient ascent perturbations are similar, and that perturbations for the same sample in adjacent rounds are also similar, FAT generates adversarial examples in the embedding space using single-step gradient ascent to accelerate training, and initializes perturbations with historical information. Experiments show that FAT significantly improves the robustness of BERT models, outperforming defense baselines under various character-level and word-level modification attacks.