Skip to content

Simple transparent adversarial examples

Simple Transparent Adversarial Examples

arXiv· arXiv:2105.09685· Jaydeep Borkar, Pin-Yu ChenMIT-IBM Watson AI Lab, IBM Research; University of PuneIndexed by this site · 23 days ago· Originally publishedAttacks

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

The authors propose a transparent adversarial example attack that requires no machine learning knowledge, with object detection evasion ASR of 52% on Azure Cloud Vision and 36% on Google Cloud Vision (Table 1).

Threat modelBlack-box attack: the attacker needs no machine learning or programming knowledge, only uses public online tools (transparency tools, image editors) to modify images, and queries MLaaS vision APIs through the web demo interface; the victim systems are the object detection and OCR services of Google Cloud Vision, Microsoft Azure Computer Vision, Sightengine, and PicPurify.

Questions
MLaaS vision APIs are used in high-risk applications, but existing robustness evaluations mostly focus on small-noise adversarial attacks requiring specialized knowledge; the authors argue this does not cover the threat posed by simple manipulations that anyone can carry out.
Methods
Propose Simple Transparent Adversarial Examples: use public online tools to add white transparent patches to images to evade object detection, or embed invisible text close in RGB to the background to fool time-limited humans but be recognized by OCR.
Experiments and results
On 50 modified images, object detection evasion ASR: Azure 52%, Google 36%, Sightengine 9%, PicPurify 2% (Table 1); among 40 OCR test images, 90% fooled time-limited humans and were recognized by Google OCR.
Limitations and future work
The authors note in A.2.4: when the target object is small relative to the image, it is difficult to apply a transparent patch, and the patch may also make the object blurred to humans. The experiments cover only 4 APIs, with 50 detection samples each and 40 OCR samples.
Experimental setupGoogle Cloud Vision API、Microsoft Azure's computer vision API et al. · Kaggle's Weapons Dataset、Caltech 101 et al. · ASR、平均修改约束(透明度百分比) et al.
Evaluated models
Google Cloud Vision APIMicrosoft Azure's computer vision APIsightengine.comPicPurify
Benchmarks
Kaggle's Weapons DatasetCaltech 101Caltech-256
Metrics
ASR平均修改约束(透明度百分比)平均查询次数
AI summaryThe paper proposes two simple adversarial image generation methods, secret embedding and transparent adversarial examples, which go beyond traditional small-noise adversarial attacks and are used to evaluate the robustness of commercial vision APIs. The authors test on real deployed services including sightengine.com, picpurify.com, Google Cloud Vision API and Microsoft Azure Computer Vision API; transparent adversarial examples achieve an attack success rate of 52% against Azure Cloud Vision and 36% against Google Cloud Vision. 90% of the images were embedded with secret text that can fool human vision under limited time but is detected by Google Cloud Vision API optical character recognition. The authors state that these methods are simple enough for non-experts to construct and pose a threat to APIs used in high-risk applications.

The paper proposes two simple adversarial image generation methods, secret embedding and transparent adversarial examples, which go beyond traditional small-noise adversarial attacks and are used to evaluate the robustness of commercial vision APIs. The authors test on real deployed services including sightengine.com, picpurify.com, Google Cloud Vision API and Microsoft Azure Computer Vision API; transparent adversarial examples achieve an attack success rate of 52% against Azure Cloud Vision and 36% against Google Cloud Vision. 90% of the images were embedded with secret text that can fool human vision under limited time but is detected by Google Cloud Vision API optical character recognition. The authors state that these methods are simple enough for non-experts to construct and pose a threat to APIs used in high-risk applications.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    The authors argue that existing robustness evaluations of vision APIs only focus on small-noise attacks that require specialized knowledge, ignoring simple manipulations that anyone can carry out.

    Core problem: robustness evaluations of MLaaS vision APIs lack coverage of simple adversarial manipulations that "anyone can do without specialized knowledge."

    • Scenario and importance: MLaaS vision APIs (prebuilt models and algorithms) are deployed in high-risk applications; the authors consider their robustness to various manipulations crucial, while recent work has evaluated only typical adversarial attacks.
    • Shortcomings of existing methods: the authors state that current adversarial example generation methods require attackers to have knowledge of the victim model (e.g., input gradients for white-box attacks) or a large number of prediction queries (e.g., gradient estimation and substitute model training for black-box attacks), which does not closely match real scenarios where attackers may lack machine learning skills and computational budgets.
    • Core observation: the authors argue that although established threat models such as ℓp-norm-constrained perturbations are important, they cannot provide a complete evaluation spectrum for semantically similar adversarial examples; simple methods require no computational power and may be more dangerous in the real world.
    • What the paper proposes: two simple adversarial image generation methods—transparent patch to evade object detection, and secret embedding of text to fool time-limited humans but trigger OCR—evaluated on 4 real deployed vision APIs.
    • Threat model:
      • Attacker goal: make the modified image evade object detection or be misclassified; or embed text so that it is recognized by Google Cloud Vision OCR but fools time-limited human vision.
      • Attacker knowledge: black-box—requires no machine learning or programming knowledge and no understanding of model internals.
      • Attacker capability: modify images using only public online tools (transparency tools, image editors), query APIs through the web demo interface, requiring only a small number of queries.
      • Victim systems: the object detection and OCR services of Sightengine, PicPurify, Google Cloud Vision API, and Microsoft Azure Computer Vision API.
  2. What related research exists?

    Related work falls into three categories: adversarial attacks requiring expertise, attacks against object detection and cloud vision APIs, and simple transformation-based attacks; the authors position this paper as a simpler complement.

    Classic adversarial attacks (requiring expertise)

    • Szegedy et al. (2014), Biggio et al. (2013), Goodfellow et al. (2015)—found that deep neural networks are susceptible to adversarial perturbations, and that perturbed images remain semantically similar to humans.
    • Madry et al. (2019), Papernot et al. (2016), Carlini & Wagner (2017), Brown et al. (2017), Hosseini & Poovendran (2018)—various adversarial attack methods; the paper states they require machine learning expertise such as first-order or zeroth-order optimization.

    Black-box attacks and attacks against object detection/cloud APIs

    • Papernot et al. (2017)—practical black-box attacks; the paper notes that such methods require thousands of queries and are costly.
    • Chen et al. (2017)—ZOO, a black-box attack based on zeroth-order optimization that does not require training a substitute model.
    • Liu et al. (2019) (DPatch), Li et al. (2019), Chen et al. (2019) (ShapeShifter), Song et al. (2018), Hosseini et al. (2017), Goodman (2020)—attacks against object detectors and the Google Cloud Vision API; the paper states they are all small-noise adversarial attacks requiring machine learning and programming knowledge.

    OCR adversarial attacks

    • Song & Shmatikov (2018), Chen & Xu (2020)—attack OCR systems with adversarial text images/adversarial watermarks; the paper states that current OCR robustness evaluation mostly centers on traditional adversarial examples.
    • Elsayed et al. (2018)—generate adversarial examples that fool both computer vision and time-limited humans; the paper calls it the work closest to this paper's OCR attack.

    Simple transformation attacks

    • Engstrom et al. (2019)—rotation and translation alone can fool CNNs.
    • Goh et al. (2021)—non-programmatic Typographic attacks. In A.1, the paper states that these simple transformations may also require machine learning knowledge, whereas this paper focuses on transparent regions and invisible text embedding.

    Baselines and benchmarks: the paper sets no numerical baseline method comparison; evaluation uses Kaggle's Weapons Dataset (object detection) and Caltech 101, Caltech-256 (OCR). Authors' positioning: this paper's method complements existing adversarial example generation pipelines by evaluating robustness with simpler, non-traditional manipulations.

  3. How does the paper address this problem?

    Proposes Simple Transparent Adversarial Examples: transparent patches evade object detection, secretly embedded text fools time-limited humans, using only public online tools throughout.

    Overall approach: use public online tools that anyone can operate to make two types of simple modifications to images—adding white transparent patches and embedding invisible text—to evaluate the robustness of visual APIs in deployment. The method is named Simple Transparent Adversarial Examples, with a double meaning: simple and transparent to produce (no machine learning knowledge required), and the modification method introduces transparent white regions and secret invisible text.

    Problem setting

    • Object detection: introduce into image x0 a white transparent patch p whose transparency is controlled by the modification constraint ε, so that the resulting image x is misclassified or completely evades detection; the value of ε requires that the modified image remains unambiguous to humans and the class is preserved.
    • OCR secret embedding: find an image x with embedded text t such that t fools time-limited human vision but is detected by Google Cloud Vision OCR. RGB difference is defined as the sum of the differences between the text font color and the embedding region color across the R, G, and B channels (A.3.1 footnote).

    Object detection attack (A.2.1)

    • Tools: the transparency maker on onlinejpgtools.com; data from Kaggle's Weapons Dataset.
    • Two patch methods: (1) do not choose the region yourself; the tool decides the patch location; (2) randomly select a region yourself, use a public tool to obtain the RGB values of that region, and then apply the patch.
    • Search procedure: method (1) starts at 10% transparency; on success, decrease, on failure, increase, in steps of 5–10%, until finding the minimum transparency that both evades detection and is unambiguous to humans; method (2) starts at 5% with steps of 1–2%.

    OCR secret embedding attack (A.3.1)

    • Tools: imagecolorpicker.com to obtain the RGB values of the embedding region, Photopea to embed text; data from Caltech 101 and Caltech-256.
    • Search procedure: start from font size 15 px and minimum RGB difference 30; if OCR can recognize it, continue decreasing the RGB difference or font size; if it cannot recognize it, fix 15 px and increase the RGB difference, until the text can be recognized by OCR but fools time-limited humans.
    • Parameter trade-off: the authors observe a trade-off between font size and RGB difference—a larger RGB difference with a smaller font size (≤15 px) or a smaller RGB difference with a larger font size (>15 px) is more suitable; a smaller RGB difference with a smaller font size is the ideal condition.

    Success criteria

    • Object detection: a modified image that completely evades detection or is misclassified counts as success (A.2.1 footnote); report ASR, average modification constraint (Avg ε1/ε2), and average number of queries.
    • OCR: the embedded text is recognized by Google Cloud Vision OCR but fools time-limited human vision; the paper does not state the specific number of participants or timing settings for the time-limited human experiment.
  4. What experiments did the paper conduct?

    Test 50 transparent patch images on each of 4 visual APIs (ASR 2%–52%), and test OCR secret embedding on 40 images (90% fool time-limited humans and are recognized).

    Experimental setup

    • Systems tested: Azure Cloud Vision, Google Cloud Vision (general object detection), Sightengine, PicPurify (weapon detection), and Google Cloud Vision's OCR service; all queried through the web demo interface.
    • Datasets: Kaggle's Weapons Dataset for object detection; images from Caltech 101 and Caltech-256 for OCR.
    • Metrics: ASR (attack success rate, including evading detection and misclassification), average modification constraints Avg ε1 (not selecting regions themselves)/Avg ε2 (selecting regions themselves), and the corresponding average number of queries.
    • Sample size: 50 modified images tested per API for object detection; 40 images tested in the OCR experiment.
    • Baselines and repetitions: the paper sets no numerical baseline and does not report the number of repetitions; the number of participants and the time limit for the time-limited human evaluation are not stated.

    Main results

    According to Table 1 (A.2.3), the performance of each API against the transparent patch object detection attack:

    Swipe horizontally to read the table

    APIASRSamples TestedAvg ε1Queries for ε1Avg ε2Queries for ε2
    Azure Cloud Vision52%5039.37%46.87%3
    Google Cloud Vision36%5047.08%56.71%4
    Sightengine9%5043.25%46%2
    PicPurify2%5030%3NANA
    • Authors' interpretation: the authors acknowledge that this attack's ASR is lower than existing norm-constrained black-box attacks, but consider its scenario more realistic and its queries fewer, making it more likely to occur in the real world.
    • API differences: the authors believe PicPurify and Sightengine are more robust because they are designed specifically for weapon detection, whereas Azure and Google are general object detection APIs.
    • Impact of failure cases: the authors observe that in unsuccessful attack cases, the modifications still significantly reduce detection accuracy (the paper gives no specific numbers).

    OCR secret embedding experiment

    • What was tested: invisible text was embedded in 40 images from Caltech 101/256, testing Google Cloud Vision OCR and time-limited human vision.
    • Results: 90% of the images fooled time-limited humans and were recognized by Google Cloud Vision OCR; the authors additionally report that on samples with very small font size and RGB difference set to 0, the evasion rate reaches close to 100% for both time-limited humans and OCR (an example is shown in Figure 6 of A.4, font size 5 px, RGB difference 0).
    • Feasible parameters: Table 2 (A.3.4) lists 10 successful configurations, with font sizes 9–15 px and Min RGB difference 20–79; in the case shown in Figure 4, 9 px can be recognized by OCR while 8 px cannot.

    Other ablations and analyses

    • Patch region selection: the average transparency when selecting regions themselves (ε2) (6%–6.87%) is lower than when the tool selects (ε1's 30%–47.08%), and the number of queries is also fewer (Table 1).
    • Illustrated cases: Figure 1 shows that 55% transparency causes the image to be misclassified as hat by Google Cloud Vision; Figure 9 shows that 28% transparency evades Sightengine weapon detection.
    • Venn diagram positioning: in Figure 7, the authors classify this attack as—in the object detection scenario, fooling only the deep neural network; in the OCR scenario, fooling time-limited humans, and when both the RGB difference and font size are very small, fooling both OCR and humans.
  5. What are some points that can be further explored?

    The authors note in A.2.4 and A.6 limitations such as the difficulty of patching small objects and three future directions; the experiments cover only 4 APIs, with sample sizes of 50/40.

    Limitations and future directions noted by the authors

    • Small objects are hard to patch (A.2.4): if the object to be patched (e.g., a gun) is small relative to the whole image, it may be difficult to apply a transparent patch; even if applied, the object may become completely blurred to humans.
    • Extending to other objects (A.6): one future direction is whether transparent patches can be applied to objects other than guns so that they remain unambiguous to humans but evade object detection.
    • Robustness of adversarially trained models (A.6): the authors consider it worth observing the robustness of adversarially trained models to such simple attacks; this paper mainly focuses on publicly deployed APIs, because attackers can easily query them through web interfaces.
    • Feasible defenses (A.6): the authors suggest it is worth exploring which existing defenses might be used to defend against this attack.

    Experimental coverage

    • Systems tested: object detection covers 4 APIs in total: Azure Cloud Vision, Google Cloud Vision, Sightengine, PicPurify; OCR tests only Google Cloud Vision's OCR service.
    • Sample size: 50 modified images per API for object detection (Table 1); 40 images in the OCR experiment; Table 2 lists 10 successful OCR parameter configurations.
    • Data sources: object detection images come from Kaggle's Weapons Dataset; OCR images come from Caltech 101 and Caltech-256.
    • Information not reported: the paper does not report the number of participants, time limit, or consistency for the time-limited human evaluation; does not report the number of experimental repetitions; does not compare numerically against existing black-box attacks; does not test any defenses.
  6. Summarize the main content of the paper

    The authors propose transparent adversarial examples that require no expertise, and validate two types of attacks—object detection evasion and OCR secret embedding—on visual APIs in 4 deployments.

    An attack paper published at the ICLR 2021 Workshop, focusing on the robustness of MLaaS vision APIs under simple manipulations.

    • Problem: the authors argue that existing robustness evaluations focus on small-noise adversarial attacks that require machine learning expertise or a large number of queries, ignoring simple manipulations that attackers without specialized skills can carry out with public online tools, which are more dangerous in real deployments.
    • Method: proposes two variants of Simple Transparent Adversarial Examples—using transparency tools to add white transparent patches to images to evade object detection; using color-picking tools and online editors to embed invisible text with RGB close to the background, so that it fools time-limited humans but is recognized by OCR. The attack is black-box and requires only a small number of queries.
    • Object detection results: on 50 modified images generated from the Kaggle weapons dataset, ASR is Azure Cloud Vision 52%, Google Cloud Vision 36%, Sightengine 9%, PicPurify 2% (Table 1); when selecting regions themselves, the average transparency is only about 6%–6.87%, with an average of 2–4 queries.
    • OCR results: among 40 test images, 90% of the embedded text fooled time-limited humans and was recognized by Google Cloud Vision OCR; the authors additionally report that when the font size is very small and the RGB difference is 0, evasion is close to 100% for both parties.
    • Authors' conclusion: such simple attacks are cheap to produce and anyone can carry them out, posing a threat to APIs deployed in high-risk scenarios; the authors hope to motivate the field to focus on such simple attacks in order to build broader defenses, and note that the robustness of adversarially trained models and feasible defenses are future directions.

Related papers

Show 2 more papers
Read originalarxiv.org