Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning

1. 01What problem does this paper attempt to solve?

   Studies the robustness of image captioning (CNN+RNN) systems to targeted adversarial perturbations and proposes the Show-and-Fool attack algorithm.

   Core question: whether neural image captioning systems can be misled by visually imperceptible adversarial perturbations to output an attacker-specified caption or keywords.

   • Setting and importance: image captioning is a representative task of vision-language grounding; applications listed by the authors include image search, AI agents that can see and speak, and helping visually impaired people access web pages; if the system can be misled, the authors believe this exposes inconsistency between human and machine vision-language grounding.
   • Gaps in existing work: the authors state that adversarial example research focuses on CNN image classifiers, with much less research on attacks against RNNs and CNN+RNN architectures; and directly applying classification attacks to captioning does not work — the caption output space is nearly infinite, and semantically similar captions should not be counted as different classes.
   • Core observation: the authors believe caption attacks are harder than classification attacks (the target amounts to a search space of nearly zero volume), requiring specially designed loss functions; and cite Carlini and Wagner (2017)'s conclusion that constructing losses with logits is better than with probabilities.
   • Paper proposes: the Show-and-Fool algorithm, including two attack settings — targeted caption (generating a caption exactly matching the target) and targeted keyword (the caption contains specified keywords, with free word order) — and evaluates the transferability of adversarial examples.
   • Threat model:
     • Goal: make the victim model output a randomly specified target caption for the perturbed image, or output a caption containing all specified keywords.
     • Knowledge: white-box — the attack uses a pretrained Show-and-Tell model and performs gradient-based optimization with ADAM.
     • Capability: add L2-norm perturbations to the input image, with pixel values constrained to [−1,1]; 1000 iterations per image.
     • Victim system: CNN+RNN image captioning systems, mainly testing Show-and-Tell (Inception-v3 encoder); transfer experiments target Show-Attend-and-Tell and NeuralTalk2.

   Collapse this answer

2. 02What related research exists?

   Related work falls into three categories: vision-language grounding, CNN+RNN image captioning, and CNN adversarial attacks; the authors claim this paper is the first work on adversarial examples for image captioning.

   Vision-language grounding

   • Multi-task family — The paper lists image/video captioning (Karpathy and Li, 2015; Vinyals et al., 2015, etc.), visual dialog (Das et al., 2017), visual question answering (Antol et al., 2015, etc.), visual storytelling, image generation from captions, etc.; this paper focuses on the robustness of image captioning models among these, and the authors believe the method can also provide robustness evaluation ideas for other tasks using similar multimodal RNN architectures.

   CNN+RNN image captioning models

   • Show-and-Tell (Vinyals et al., 2015) — The vanilla version with a CNN encoder + RNN decoder, the main attack target of this paper.
   • Show-Attend-and-Tell (Xu et al., 2015) — Adds attention to the LSTM and uses the last convolutional layer feature map; the target model of this paper's transfer experiments.
   • NeuralTalk (Karpathy and Li, 2015) — A representative model of the same framework; this paper's transfer experiments also cover NeuralTalk2.
   • The authors state that these models share the CNN+RNN framework with newer systems, so the attention mechanism and the choice of CNN/RNN architecture do not significantly affect the robustness conclusions.

   Robustness research on image captioning

   • Shekhar et al. (2017) — Through foil word (replacing one word) experiments, shows that image captioning models are insensitive to single-word changes; the paper notes this corresponds to the category of untargeted attacks, whereas this paper studies the harder targeted attacks, requiring the model to generate a pre-specified caption or keywords.

   Adversarial attacks on CNN classifiers (see Supplementary Material Section 6 for details)

   • Szegedy et al. (2014), Goodfellow et al. (2015), Carlini and Wagner (2017) — CNN classifiers are vulnerable to adversarial examples; this paper draws on C&W's optimization-based attack framework and logits loss design.
   • I-FGSM (Kurakin et al., 2017) and C&W (Carlini and Wagner, 2017) — Section 4.5 of this paper uses them to attack only the CNN module as comparison baselines, with ImageNet as the benchmark (800 images randomly selected).
   • Transferability and universal perturbations — Papernot et al. (2016a) and Liu et al. (2017c) show that adversarial examples transfer between classifiers; Moosavi-Dezfooli et al. (2017) propose universal adversarial perturbations.

   The authors state that, to their knowledge, this paper is the first work to construct adversarial examples against neural image captioning systems, so no other methods can be directly compared.

   Collapse this answer

3. 03How does the paper address this problem?

   Formulates the attack as an optimization problem with L2 regularization, designs logits and log-prob losses for targeted caption and targeted keyword respectively, and solves with ADAM.

   Overall approach: Show-and-Fool formulates adversarial example construction as an optimization problem — minimizing the weighted sum of the attack loss and the L2 distance of the perturbation, using the arctanh transformation to convert the box constraint into an unconstrained problem, then designing specialized loss functions for the two attack settings.

   Problem formulation

   • For image I, find perturbation δ: min_δ c · loss(I+δ) + |δ|₂², subject to I+δ ∈ [−1,1]ⁿ; c is a regularization constant; the larger c is, the easier the attack succeeds but the larger the perturbation.
   • Introduce y = arctanh(I), w = arctanh(I+δ) − y, converting the constrained problem into unconstrained optimization over w; tanh automatically satisfies the box constraint.
   • c is determined by binary search: initial c=1, multiply by 10 each time until success, then take the average between the successful and failed c, for a total of 5 searches.

   Targeted caption loss

   • Goal: make the conditional log-probability of the target caption S the largest among all possible captions; directly use the negative log-probability as the loss (log-prob version, Equation 5).
   • Logits version (Equation 7): only require that the logit of the target word St at position t exceed the maximum logit of other words by a margin ε; the loss is the sum of hinge terms across positions; when the target word already leads by ε, the gradient of that term is zero, letting the optimizer focus on positions not yet meeting the criterion.
   • The authors cite Carlini and Wagner (2017): the logits loss outperforms the probability loss when the target network has gradient masking (e.g., defensive distillation), so both are provided.

   Targeted keyword loss

   • Keyword Kj is not required to appear at a specific position: for each keyword, take the minimum of the hinge loss over all positions, then sum over the M keywords (Equation 8).
   • RNN input handling: there is no complete target sentence, so the inferred caption of the original image is used as the initial input, and after every T optimization steps the current top-1 prediction is used as the new input to continue iterating.
   • Keyword conflicts: multiple keywords may compete for top-1 at the same position; a gate function gt,j is introduced to mask the term when the top-1 at a position is already another keyword (logits version Equation 9; log-prob version Equations 12–13, see Supplementary Material Section 9).

   Optimization and success criteria

   • ADAM optimization, learning rate 0.005, 1000 iterations; ε=1 fixed except in transfer experiments; about 38 seconds for 1000 iterations on a single image (Nvidia GTX 1080 Ti).
   • Targeted caption success criterion: the inferred caption exactly matches the target caption; targeted keyword success criterion: the inferred caption contains all specified keywords.
   • Target captions/keywords are taken from captions generated by the model for another randomly selected validation-set image, ensuring the target lies within the model's output space; keywords are restricted to nouns, verbs, adjectives or adverbs.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Attacking Show-and-Tell on 1000 MSCOCO images: targeted caption success rate 95.8%, keyword attacks 96.0%–97.5%, and the adversarial examples transfer.

   Experimental Setup

   • Victim models: pretrained TensorFlow version of Show-and-Tell (Inception-v3 as the CNN encoder); transfer experiments use Show-Attend-and-Tell (also Inception-v3, trained on MSCOCO 2014, with CNN parameters differing due to fine-tuning) and NeuralTalk2.
   • Dataset: MSCOCO validation set, with 1000 images randomly selected for each group of experiments; Section 4.5 additionally uses 800 random images from ImageNet (synsets sharing words with the Show-and-Tell vocabulary and correctly classified by Inception-v3).
   • Baselines: Section 4.5 uses I-FGSM and C&W to attack only the CNN module (ImageNet classification task), then applies the adversarial examples to attack Show-and-Tell; the authors state this paper is the first captioning attack work, so no other methods are comparable.
   • Metrics: attack success rate, average L2 perturbation (averaged over successful samples only); caption relevance using BLEU-1/2/3/4, ROUGE, METEOR.
   • Parameters: ADAM learning rate 0.005, 1000 iterations, c binary search 5 times, ε=1 (except in transfer experiments); in keyword experiments the RNN input is updated every T=5 iterations.
   • Loss: the main text reports results for the logits loss (Eqs. 7, 9); log-prob loss results are in the supplementary material (Table 5–8).

   Main Results

   According to Table 1 (logits loss, Show-and-Tell, 1000 images from the MSCOCO validation set):

   Experiment	Success rate	Average L2 perturbation
   targeted caption	95.8%	2.213
   1-keyword	97.1%	1.589
   2-keyword	97.5%	2.363
   3-keyword	96.0%	2.626
   C&W on CNN (transferred to Show-and-Tell)	22.4%	2.870
   I-FGSM on CNN (transferred to Show-and-Tell)	34.5%	15.596

   • The authors state that the high success rates and low perturbations show that Show-and-Tell is not robust to targeted adversarial perturbations; the adversarial examples shown in Figure 1 are visually indistinguishable from the original images.
   • The authors state that keyword attack success rates are higher than caption attack success rates (at least 97% for 1-keyword and 2-keyword, at least 96% for 3-keyword) because caption generation has more degrees of freedom.
   • I-FGSM/C&W attacking only the CNN achieve 100% targeted success on ImageNet classification, but transfer to Show-and-Tell with success rates of only 34.5%/22.4% and larger perturbations; the authors use this to argue that attacking captioning systems is not a simple extension of classification attacks.

   Failure Sample Analysis

   • The 4.2% of samples where targeted caption failed (Table 2): the generated captions remain highly related to the target captions, e.g., at c=10, BLEU-1 is .725 and ROUGE is .664; as c increases, the perturbation rises from 1.726 (c=1) to 23.31 (c=10⁴).
   • The 4.0% of samples where 3-keyword failed (Table 3): most still contain some of the keywords; at c=10², 93.1% of samples contain at least 1 target keyword and 58.6% contain 2, with an average of 1.52; the authors state that the failure samples are still partial successes.

   Transferability Experiments

   • Setup: first generate successful adversarial examples for 1000 images on Show-and-Tell using the targeted caption method, then feed them into Show-Attend-and-Tell; ori measures the difference between the transferred sample's caption and the original image's caption (lower is better), tgt measures similarity to the target caption (higher is better), and mis is the inherent difference between the two models' captions for the same original image (model mismatch).
   • Results (Table 4): increasing c or ε improves transferability but increases perturbation; at C=1000, ε=10, tgt is closest to mis (e.g., BLEU-1 tgt=.534 vs. mis=.649); the authors state that at this point the difference between the two models' adversarial captions is mainly determined by the models' inherent difference, and the adversarial perturbations are highly transferable.
   • NeuralTalk2: at c=10⁴, ε=10, the average L2 perturbation between the original image and the transferred adversarial caption is 38.01, BLEU-4 is 0.440, and METEOR is 0.473; the authors state that the adversarial examples can also fool NeuralTalk2, and consider this to reflect shared robustness leakage across different captioning models.

   Other Ablations and Analyses

   • log-prob loss (supplementary Table 5): targeted caption success rate 95.4%, average perturbation 1.858; 1/2/3-keyword success rates 99.2%/96.9%/95.7%, similar to the logits loss; the authors explain that when the target model has no defenses, the softmax gradient vanishing problem is not significant, but recommend generally using the logits loss.
   • log-prob transferability (Table 8): at C=1000, BLEU-1 tgt=.500 (mis=.657); the authors state that compared with Table 4 (C=1000, ε=10), ori and tgt are worse, and the additional ε parameter in the logits loss helps transfer.
   • attention visualization (Figure 11–13): on original images, Show-Attend-and-Tell's attention aligns with human perception, while on transferred adversarial examples the attention is misaligned (per the authors).
   • Failure sample statistics (log-prob): targeted caption failure rate 4.6% (Table 6); 3-keyword failure rate 4.3%, of which, with a suitable c, over 90% contain at least 1 keyword and over 60% contain 2 (Table 7).

   Collapse this answer

5. 05What are some points that can be further explored?

   The paper does not specifically discuss limitations; the conclusion mentions that adversarial training or data augmentation could be used to improve robustness, and the experiments cover only three models with no defense evaluation.

   Limitations and future directions noted by the authors

   • The paper has no Limitations or Future Work section and does not explicitly state limitations.
   • Extended applications (Conclusion): the authors state that Show-and-Fool can be easily extended to other applications using RNN or CNN+RNN architectures.
   • Defense directions (Conclusion): the authors believe the method in this paper can be used to evaluate and potentially improve the robustness of vision-language grounding and other NLP models, for example through adversarial training or data augmentation — this is a possibility raised by the authors; the paper does not conduct defense experiments.
   • Recommendation to use the logits loss (Supplementary Material Section 8): when evaluating the robustness of general image captioning models, the authors recommend the logits loss because it is not affected by potential vanishing gradients and can reveal the model's inherent robustness.

   Scope of experiments

   • The attacked captioning models are Show-and-Tell (main test), Show-Attend-and-Tell and NeuralTalk2 (transfer experiments only), 3 in total; all based on the Inception-v3 encoder.
   • Datasets are the MSCOCO validation set (1000 randomly selected images per experiment group) and ImageNet (Section 4.5, 800 randomly selected images).
   • Perturbations are measured only with the L2 norm; the I-FGSM baseline in Section 4.5 uses an L∞ constraint (ε∞=0.3).
   • Success is determined by exact match (targeted caption) or containing all keywords (targeted keyword), without LLM or human review; caption relevance uses automatic metrics BLEU/ROUGE/METEOR.
   • The paper does not report the number of experimental repetitions, success rate variance or confidence intervals; no defenses or adaptive attacks are evaluated.

   Collapse this answer

6. 06Summarize the main content of the paper

   Show-and-Fool uses optimization methods to attack CNN+RNN image captioning models, achieving a 95.8% success rate on targeted caption attacks, and the adversarial examples transfer across models.

   An adversarial attack paper targeting neural image captioning systems (the authors state it is the first such work), studying the robustness of CNN+RNN architectures on vision-language grounding tasks.

   • Problem: whether image captioning models can be misled by visually imperceptible perturbations to output an attacker-specified caption or keywords; the authors believe caption attacks are harder than classification attacks because the output space is nearly infinite.
   • Method: Show-and-Fool formulates the attack as an optimization problem minimizing the attack loss plus L2 perturbation, uses the arctanh transformation to handle the box constraint, and solves with ADAM; two settings — targeted caption (exact match to the target caption) and targeted keyword (contains specified keywords), each with logits and log-prob losses, and the keyword version uses a gate function to handle keyword conflicts.
   • Main results: attacking Show-and-Tell on 1000 MSCOCO validation images, under the logits loss the targeted caption success rate is 95.8% with an average L2 perturbation of 2.213; 1/2/3-keyword success rates are 97.1%/97.5%/96.0% (Table 1).
   • Transferability: adversarial examples generated on Show-and-Tell transfer to Show-Attend-and-Tell (at C=1000, ε=10, tgt approaches the model's inherent difference mis) and NeuralTalk2 (BLEU-4 0.440, METEOR 0.473); the authors state this exposes shared robustness leakage across models.
   • Comparison experiments: I-FGSM/C&W attacking only the CNN achieve 100% success on ImageNet classification, but transfer to Show-and-Tell at only 34.5%/22.4%; the authors use this to argue that attacking captioning is not a simple extension of classification attacks.
   • Authors' conclusion: the experimental results point to weaknesses of the tested image captioning systems in vision-language grounding, and there is inconsistency between human and machine grounding; the authors believe the method can be extended to other RNN/CNN+RNN applications and can be used to evaluate and even improve robustness through adversarial training.

   Collapse this answer

Related papers