Gradient-Based Adversarial Attacks against Text Transformers
Gradient-based Adversarial Attacks against Text Transformers
The authors propose GBDA, using a Gumbel-softmax-parameterized adversarial distribution to perform gradient-optimization attacks on text transformers, reducing BERT's accuracy on AG News from 94.2 to 2.5 in the white-box setting.
White-box attack: the attacker has access to the target model's gradients, optimizes the adversarial distribution, and then samples to make the model misclassify. Black-box transfer attack: optimizes the distribution on GPT-2, samples to query a different target model, and requires only hard-label outputs (discrete labels), not continuous scores.
- The discrete nature of text makes gradient optimization difficult to apply directly to adversarial attacks, and existing black-box heuristic methods often introduce unnatural modifications with limited effectiveness. The authors attempt to use gradient methods to generate more fluent and semantically faithful adversarial text.
- Proposes GBDA: using Gumbel-softmax to transform adversarial example search into gradient optimization over a continuous parameter matrix Θ, with an objective combining margin loss, language model perplexity (fluency), and BERTScore (semantic similarity) soft constraints, then sampling attacks from the distribution after optimization.
- White-box attacks on GPT-2, XLM, and BERT reduce nearly all task accuracies to below 10%; transfer attacks (GPT-2→BERT) match or exceed BERT-Attack and BAE in most settings, and require only hard-label outputs.
- The authors note that the method is limited to token replacement and does not support insertion or deletion; the parameter matrix Θ has size n×V, which over-parameterizes long sentences. Experiments cover only English classification tasks, not other languages or generation tasks.
- Evaluated models
- BERTGPT-2XLM (en-de)ALBERTRoBERTaXLNet
- Benchmarks
- DBPediaAG NewsYelp ReviewsIMDBMNLI
- Metrics
- Clean Acc.Adv. Acc.Cosine Sim. (USE)# Queries
Researchers at Facebook AI Research propose the GBDA framework, which the authors describe as the first universal gradient-based attack against Transformer models. Rather than searching for individual adversarial examples, the method optimizes a distribution of adversarial examples parameterized by a continuous matrix, enabling differentiable optimization via Gumbel-softmax, and uses language model perplexity and BERTScore as soft constraints on fluency and semantic similarity. White-box attacks on GPT-2, XLM and BERT across AG News, DBPedia, Yelp, IMDB and MNLI reduce the target model's accuracy to below 10% on almost all models and tasks, and in most cases the USE cosine similarity with the original input is above 0.8; on AG News it reduces BERT accuracy from 94.2 to 2.5, exceeding BERT-Attack's 10.6. The authors note that the method currently supports only token replacement and that the adversarial distribution parameters suffer from over-parameterization.
Why it was selectedThe paper reformulates adversarial example search as distribution optimization, making gradient attacks generally applicable to text Transformers, with black-box transfer attacks requiring only hard-label outputs.
Paper analysis
What problem does this paper attempt to solve?
The discrete nature of text hinders gradient optimization; the authors propose GBDA, which uses an adversarial distribution to achieve gradient-based attacks.
Core problem: how to perform gradient-based adversarial attacks on text transformer models while maintaining the fluency and semantic faithfulness of the perturbation.
- Problem setting: Deep neural networks are sensitive to small perturbations, but the discrete nature of text makes gradient descent difficult to apply directly, and distance measures in continuous domains such as L2/L∞ are not applicable to text.
- Shortcomings of existing work: Existing black-box attacks rely on heuristic word substitution and greedy/beam search, often introducing grammatical errors or semantically unnatural changes; the authors point out that BERT-Attack only reduces BERT's accuracy on AG News from 94.2 to 10.6, whereas image attacks can reduce it to near 0.
- Core observation: Rather than searching for a single adversarial example, it is better to search for a distribution of adversarial examples parameterized by a continuous matrix, so that gradient optimization can be used.
- Proposed approach: Propose GBDA (Gradient-based Distributional Attack), which parameterizes adversarial examples with a Gumbel-softmax distribution and combines BERTScore and language model perplexity as differentiable soft constraints.
- Threat model:
- White-box attack: The attacker has full access to the target model and optimizes the distribution parameters Θ via gradients.
- Black-box transfer attack: Optimize the distribution on a source model (GPT-2), sample and query different target models, requiring only hard-label outputs (discrete labels), not continuous scores.
What related research exists?
Related work includes character/word-level black-box attacks, universal triggers, and adversarial examples in other modalities.
Character-level and word-level black-box attacks
- Character-level attacks: Ebrahimi et al. (2017), Li et al. (2018), Gao et al. (2018) approximate perceptibility through character substitution, swapping, insertion, and deletion.
- Word-level heuristic attacks: Samanta and Mehta (2017), Zang et al. (2020) use synonym substitution; Alzantot et al. (2018), Ren et al. (2019), Jin et al. (2020) use word embedding similarity for substitution.
- Masked language model-based attacks: Garg and Ramakrishnan (2020), Li et al. (2020a,b) use MLMs such as BERT to generate word replacement candidates. The authors point out that such methods require sequential token replacement and may exponentially expand the search space for rare words.
Universal adversarial triggers
- Wallace et al. (2019), Song et al. (2020) propose universal adversarial triggers, but the authors point out that these triggers contain unnatural word combinations and are perceptible to human observers.
Adversarial examples in other modalities
- Research on adversarial examples for images (Carlini and Wagner, 2017; Madry et al., 2017), speech (Carlini and Wagner, 2018), and graph data (Dai et al., 2018; Zügner et al., 2018).
Baselines and comparison
- Baseline methods: TextFooler (Jin et al., 2020), BAE (Garg and Ramakrishnan, 2020), BERT-Attack (Li et al., 2020b).
- Comparison setup: All baselines are evaluated on the finetuned BERT in the TextAttack library; GBDA attacks BERT in the white-box setting, and in the transfer setting transfers from GPT-2 to BERT for a fair comparison.
The authors position the difference between this paper and existing work: existing methods are mostly black-box search and produce unnatural text, whereas GBDA is the first general gradient-based attack framework, achieving more natural and better attack results through distribution parameterization and differentiable constraints.
How does the paper address this problem?
GBDA uses Gumbel-softmax to parameterize an adversarial distribution, combined with differentiable fluency and similarity constraints for gradient optimization.
The core idea of GBDA is to transform discrete adversarial example search into gradient optimization over a continuous parameter matrix, with soft constraints ensuring text quality.
Adversarial distribution and Gumbel-softmax relaxation
- Distribution definition: Define a parameter matrix Θ ∈ ℝn × V; each token is independently sampled from Categorical(Softmax(Θi)), forming an adversarial distribution PΘ.
- Probability vector input: The model input is extended to probability vectors, and the embedding is computed as e(πi) = ∑j (πi)j e(j), making the input differentiable.
- Gumbel-softmax: Sample π̃ from a Gumbel-softmax distribution with temperature parameter T; as T → 0 it converges to the categorical distribution, so Θ can be optimized with gradient descent.
Differentiable soft constraints
- Fluency constraint: Use a causal language model (CLM) g to compute the negative log-likelihood (NLL), extended to be defined for probability vector sequences, and add it to the objective as a soft constraint.
- Semantic similarity constraint: Use BERTScore (recall) to compute the similarity between the original text and the perturbed text on contextualized embeddings, defining ρg(x, π) = 1 − RBERT(x, π) as a differentiable constraint.
Optimization objective and algorithm
- Objective function: L(Θ) = 𝔼π̃ ∼ P̃Θ ℓ(e(π̃), y; h) + λlm NLLg(π̃) + λsim ρg(x, π̃), where ℓ is the margin loss.
- Optimization details: Optimize with Adam, learning rate 0.3, batch size 10, 100 iterations; Θ is initialized to 0, and the positions corresponding to the original tokens are set to C ∈ {12, 15}; λperp = 1, λsim ∈ [20, 200], κ ∈ {3, 5, 10}.
Sampling and transfer attack
- Sampling strategy: After optimization, sample from PΘ until the model misclassifies or the maximum number of samples is reached (100 for white-box, 1000 for transfer).
- Transfer attack: A distribution optimized on a source model can be directly sampled to attack different target models, requiring only hard-label outputs to decide when to stop, with no need for continuous scores.
What experiments did the paper conduct?
The experiments cover white-box attacks, black-box transfer, constraint ablation, and runtime efficiency analysis.
Experimental Setup
- Models tested: GPT-2, XLM (en-de), BERT (white-box); ALBERT, RoBERTa, XLNet (transfer).
- Datasets: DBPedia, AG News (news classification), Yelp, IMDB (sentiment analysis), MNLI (natural language inference, including matched/mismatched).
- Baselines: TextFooler, BAE, BERT-Attack, all evaluated on TextAttack's finetuned BERT.
- Metrics: Clean Acc., Adv. Acc. (accuracy after attack), Cosine Sim. (USE embedding cosine similarity), # Queries (number of queries).
- Sample size: 1000 test inputs randomly selected per task; white-box attacks sample at most 100 times, transfer attacks at most 1000 times.
- Constraint models: GPT-2 uses the pretrained model as CLM; XLM uses a finetuned checkpoint; BERT trains a CLM on WikiText-103.
Main Results
Swipe horizontally to read the table
Target model Task Clean Acc. Adv. Acc. Cosine Sim. Source BERT AG News 95.1 2.5 0.82 Table 1 BERT Yelp 97.3 4.7 0.92 Table 1 BERT IMDB 93.0 3.0 0.92 Table 1 GPT-2 AG News 94.8 6.6 0.90 Table 1 XLM IMDB 87.6 0.1 0.97 Table 1 BERT (transfer) AG News 95.1 8.8 0.69 Table 3 BERT (transfer) Yelp 97.3 2.6 0.83 Table 3 BERT (transfer) IMDB 93.0 8.5 0.92 Table 3 - White-box attacks reduce nearly all model task accuracies to below 10%; BERT on AG News drops from 94.2 to 2.5 (cited in the main text).
- Transfer attacks (GPT-2→BERT) match or exceed BERT-Attack and BAE in most settings, with fewer queries (Table 3).
- Transfer attacks require only hard-label outputs, whereas baselines rely on continuous scores.
Transfer Attack Generalization
- Setup: Attack ALBERT, RoBERTa, and XLNet with the same distribution optimized on GPT-2 (Table 4).
- Results: ALBERT achieves Adv. Acc. 7.5 on AG News, RoBERTa 4.1/15.1 on MNLI (m.), and XLNet 1.7/14.4 on MNLI (mm.).
- Authors' interpretation: The adversarial distribution captures common failure modes across multiple transformer models, showing good generalization.
Constraint Ablation Analysis
- Effect of λ_sim (Figure 2): Lower λ_sim produces more aggressive perturbations and better generalization; higher λ_sim increases the number of queries needed to reach the target accuracy, revealing a trade-off among similarity, success rate, and query budget.
- Fluency constraint (Table 5): Without the fluency constraint, generated text contains meaningless word combinations (e.g., "Thecom", "accessrage"); with the constraint, grammatical correctness is maintained.
- Effect of re-tokenization: Token error rate is about 2%; samples remain adversarial after re-tokenization, and all reported accuracies are computed after re-tokenization.
Other Analysis
- Runtime: White-box optimization requires forward and backward passes, but total runtime is 20s/sample, comparable to the black-box attack BERT-Attack (Section 4.3).
What are some points that can be further explored?
The authors point out that the method is limited to token replacement and is over-parameterized; future work could extend the operation types and reduce parameters.
Limitations and future directions pointed out by the authors
- Limited operation types: The method only supports token replacement, not insertion or deletion (Conclusion and Future Work). The authors hope future work can extend the adversarial distribution framework to cover a broader range of token-level modifications.
- Over-parameterization: The distribution parameter Θ has size n×V, which over-parameterizes long sentences (Conclusion and Future Work). Future work may reduce the number of parameters without affecting the optimality of the generated adversarial examples.
Scope of experiments
- Models tested: White-box experiments cover 3 models in total: GPT-2, XLM (en-de), and BERT; transfer experiments cover 3 models in total: ALBERT, RoBERTa, and XLNet.
- Task types: Only English text classification tasks (news classification, sentiment analysis, natural language inference) are covered; other languages or generation tasks are not involved.
- Attack settings: White-box attacks use 1000 examples per task and sample at most 100 times; transfer attacks sample at most 1000 times.
- Constraint models: The fluency constraint uses a CLM with the same tokenizer as the target model; BERT's CLM is trained on WikiText-103.
- Unreported information: The paper does not report experiments with human evaluation of the naturalness of adversarial examples; it does not report statistical significance tests.
Summarize the main content of the paper
GBDA achieves gradient-based text attacks through distribution parameterization, reaching SOTA in both white-box and transfer settings.
GBDA is the first general gradient attack framework for text transformers, transforming discrete adversarial search into a continuous distribution optimization problem.
- Problem: The discrete nature of text makes gradient methods difficult to apply, and existing black-box attacks produce unnatural text with limited effectiveness.
- Method: Uses Gumbel-softmax to parameterize an adversarial distribution, combined with margin loss, language model perplexity, and BERTScore soft constraints for gradient optimization, supporting white-box attacks and black-box transfer attacks that require only hard labels.
- Key results:
- White-box attacks reduce BERT's accuracy on AG News from 94.2 to 2.5, and XLM's on IMDB to 0.1 (Table 1).
- Transfer attacks (GPT-2→BERT) achieve Adv. Acc. 8.8 on AG News, outperforming BERT-Attack's 10.6, with fewer queries (107 vs 213, Table 3).
- The same distribution generalizes to attack ALBERT, RoBERTa, and XLNet (Table 4).
- Generated samples maintain high semantic similarity (most Cosine Sim. > 0.8) and grammatical correctness.
- Authors' conclusion: GBDA achieves SOTA attack performance across multiple tasks and architectures, providing a practical attack pathway for real-world systems, while noting that future work needs to support more token operation types and reduce parameterization.
Related papers
Show 3 more papers
- No Free Efficiency: Revisiting the Trade-off Between Training Efficiency and Model VulnerabilityAnalysis / interpretabilityarXiv