Audio adversarial examples: targeted attacks on speech-to-text

1. 01What problem does this paper attempt to solve?

   Can targeted adversarial examples be constructed against speech recognition: arbitrary audio, after adding a small perturbation, is transcribed as an arbitrary target phrase.

   Core question: can targeted adversarial examples be constructed against automatic speech recognition systems—given an arbitrary audio waveform, add a perturbation that is nearly imperceptible to human ears so that it is transcribed as a phrase arbitrarily chosen by the attacker.

   • Problem setting and importance: The authors note that adversarial example research had previously concentrated in the image domain, with little research in the audio domain; speech recognition is used in products such as Apple Siri, Google Now, and Amazon Echo, and its adversarial robustness is worth examining.
   • Shortcomings of existing methods: Hidden and inaudible voice commands are targeted attacks, but they require synthesizing new audio and cannot modify existing audio; the authors state that the then-SOTA targeted attack Houdini could only construct target phrases with similar phonemes, and its authors considered targeted attacks on speech recognition much harder than on vision systems.
   • Core idea: The authors use a strong iterative optimization attack based on [10], end-to-end directly optimizing over the raw sample points of the classifier input, penetrating the MFC preprocessing transform that was previously considered difficult to optimize.
   • Paper contributions: The authors report achieving targeted attacks on DeepSpeech with a 100% success rate, starting from arbitrary waveforms (including music), reaching a transcription density of 50 characters per second, and also hiding speech as silence; the authors plan to release code and dataset.
   • Threat model:
     • Goal: Given a waveform x and a target transcription y, construct x′=x+δ such that C(x′)=y and x and x′ sound similar; success is counted only when the network output exactly matches the target phrase (no spelling errors, no extra characters).
     • Knowledge: white-box, the attacker fully knows the model and its parameters.
     • Capability: directly modify the raw sample points of the input audio; assume the adversarial example does not pass through noise-introducing stages such as over-the-air playback and microphone recording.
     • Victim system: Mozilla's DeepSpeech end-to-end speech-to-text network.

   Collapse this answer

2. 02What related research exists?

   Related work is divided into four groups: adversarial examples in images and other domains, attacks on audio, DeepSpeech and CTC technology, and the optimization attack method on which this paper is based.

   Adversarial examples in images and other domains

   • Szegedy et al. [40] proposed that neural networks have adversarial examples; Goodfellow et al. [14] proposed FGSM and explained adversarial examples with a local linear explanation; Carlini & Wagner [10] proposed the iterative optimization attack on which this paper is based, and pointed out that the choice of loss function can make the final distortion differ by more than 3 times.
   • Adversarial example work beyond image classification: image generation models [26], image segmentation [1], face detection [37], manipulating images seen by RL agents [6, 21]; discrete domains include text classification [23] and malware classification [16, 20].
   • Physical world and black-box directions: Athalye et al. [2] and Kurakin et al. [27] realized image adversarial examples in the physical world; [22, 35] demonstrated black-box attacks; [31] proposed universal adversarial perturbations; [34, 28] studied transferability.

   Attacks in the audio domain

   • Hiding voice commands (Carlini et al. [11]), inaudible voice commands [39], DolphinAttack [41]: targeted attacks, but they require synthesizing new audio and cannot modify existing audio; among them, the commands of [11, 41] are effective when played over the air.
   • Gong & Poellabauer [13], Kereliuk et al. [24]: constructed standard non-targeted adversarial examples on different audio systems.
   • Houdini (Cisse et al. [12]): the paper calls it the then state-of-the-art targeted attack on speech recognition, but it can only construct target phrases with similar phonemes; the paper cites its authors as saying that targeted attacks on speech recognition are much more difficult than on human vision systems.

   Foundations of speech recognition technology

   • DeepSpeech (Hannun et al. [18]): an end-to-end speech recognition system; this paper attacks its Mozilla implementation [32]; internally it consists of MFC preprocessing and an LSTM [19] recurrent network.
   • CTC (Graves et al. [15]): a method for training sequence-to-sequence networks when the input-output alignment is unknown; this paper uses it both as the attack loss and refers to the survey in [17].

   Comparison baselines and benchmarks: the paper does not compare numerical baselines of other attack methods; §IV-A compares the FGSM single-step attack with this paper's iterative optimization attack on the CTC loss descent curve (Figure 3). The evaluation benchmark is the first 100 test instances of the Mozilla Common Voice dataset.

   Positioning: the authors state that this paper is an end-to-end white-box iterative optimization attack that acts directly on raw sample points, differs from the hidden voice commands [11] generated sequentially frame by frame, and can achieve targeted attacks for arbitrary target phrases rather than phonetically similar phrases.

   Collapse this answer

3. 03How does the paper address this problem?

   White-box end-to-end iterative optimization: differentiate through MFC, start with the CTC loss, then use a fixed-alignment per-character logit loss to reduce distortion.

   The overall idea is to formulate the construction of adversarial examples as an optimization problem, solve it end-to-end with gradient-based iterative optimization, and directly modify the raw audio sample points. The method is based on the optimization attack of Carlini & Wagner [10], and a new loss function is designed for the CTC decoding structure of audio.

   Problem formalization and distortion metric

   • Optimization objective: given a natural sample x and a target phrase t, solve minimize dBx(δ) such that C(x+δ)=t, and x+δ ∈ [−M, M] (M=2^15, i.e., the maximum representable value of 16-bit sampling, handled by clipping).
   • Distortion metric: a relative dB scale is used, dBx(δ) = dB(δ) − dB(x), i.e., the loudness difference of the perturbation relative to the original waveform; the perturbation is quieter than the original signal, so it is negative, and the smaller it is, the quieter. The authors state that as long as the perturbation is small enough, it is imperceptible to the human ear.
   • Solvable reformulation: because the constraint C(x+δ)=t is nonlinear, following [40], it is changed to minimize dBx(δ) + c·ℓ(x+δ, t), where the loss ℓ satisfies ℓ(x′,t) ≤ 0 if and only if C(x′)=t.

   Initial attack (§III-B): CTC loss + l2 relaxation

   • Loss choice: directly use the CTC loss as ℓ(x′, t). The authors point out that this loss only satisfies a one-way implication (ℓ ≤ 0 implies C(x′)=t), so the solution is still an adversarial example, only the perturbation may not be minimal.
   • l∞ oscillation problem: when optimizing directly with the l∞ metric, it oscillates near the solution and does not converge, so it is changed to solve minimize |δ|₂² + c·ℓ(x+δ, t) with the constraint dBx(δ) ≤ τ, and after obtaining a partial solution, τ is gradually reduced and optimization continues until no solution can be found.
   • Optimization settings: optimize over the entire audio sample simultaneously (unlike the frame-by-frame sequential generation of [11]), differentiating through MFC and the neural network; use the Adam optimizer, learning rate 10, at most 5,000 iterations.

   Improved loss (§III-C): fixed-alignment per-character logit loss

   • Motivation: minimizing the CTC loss causes already-correct characters to continue being classified "more confidently", wasting the perturbation budget; instead use ℓ(y, t) = max(y_t − max_{t′ ≠ t} y_t′, 0), i.e., once the target term's probability exceeds the others, the loss no longer decreases. The meaning of this expression: once the target character probability is already the largest, the optimizer no longer gains.
   • Per-character weights: different characters in the audio have different difficulty, and a single constant c must be large enough to cover the hardest character, so instead use an independent ci for each term, minimizing |δ|₂² + Σᵢ ci·Li(x+δ, πᵢ).
   • Two-step attack: the first step uses the CTC loss attack to obtain x′ and extract the alignment π it induces; the second step fixes π, starts from δ = x′ − x, and re-optimizes with the improved loss to obtain a sample with smaller distortion. The authors point out that this version is only guaranteed to be effective for greedy decoding.

   Extended attack variants

   • Maximum information density (§III-D): DeepSpeech outputs 50 frames per second, one character distribution per frame, so the theoretical maximum density is 50 characters per second; in this case the alignment is unique (π=p), and the logit loss can be used directly without first finding the alignment.
   • Non-speech source (§III-E): simply set the initial waveform to classical music with no speech, with no additional technical changes; the authors report that this setting is more computationally expensive, using 20,000 gradient descent iterations.
   • Targeted silence (§III-F): set the target to a pure space sequence of arbitrary length, and take the loss as the hinge form of the difference between the maximum probability over ε and space at each frame and the maximum probability of the remaining characters, so that the network transcribes empty content.

   Success criterion

   • Criterion: counted as success only when the network transcription exactly matches the target phrase (no spelling errors, no extra characters); evaluation involves both greedy decoding and beam-search decoding (the improved loss version is only guaranteed to be effective under greedy decoding).

   Collapse this answer

4. 04What experiments did the paper conduct?

   100% success on the first 100 test samples of Common Voice, average distortion −31dB, reduced to −38dB with the improved loss.

   Experimental setup

   • Model under test: DeepSpeech (Mozilla implementation [32]), MFC preprocessing + LSTM, trained with CTC; audio is 16KHz, 16-bit sampling.
   • Dataset: the first 100 instances of the Mozilla Common Voice test set; each sample is paired with 10 randomly selected target transcriptions, requiring (a) the transcription is incorrect, (b) theoretically reachable, for a total of 1000 source-target pairs.
   • Metrics: success rate (transcription exactly matches the target); perturbation distortion dBx(δ), in dB, the more negative the quieter.
   • Optimization settings: Adam optimizer, learning rate 10, at most 5,000 iterations (20,000 for non-speech sources); a single NVIDIA 1080Ti.
   • Comparison: no numerical baselines from other attack methods; §IV-A uses a single-step FGSM attack as a control (Figure 3).
   • Number of repetitions: the paper does not report the number of repetitions for each configuration.

   Main results

   The paper has no numerical tables; the main results are given as numbers in the text:

   Swipe horizontally to read the table

   Attack setting	Success rate	Average distortion	Source
   CTC loss attack (§III-B), 100 samples × 10 targets	100%	−31dB	§III-B Evaluation
   Improved loss attack (§III-C, greedy decoding), first 100 samples	not reported in the paper	−38dB	§III-C Evaluation
   Maximum density attack (§III-D, 50 characters per second)	not reported in the paper	−18dB	§III-D
   Non-speech source attack (§III-E, 5 seconds of classical music)	not reported in the paper	−20dB	§III-E
   Targeted silence (§III-F)	not reported in the paper	below −45dB	§III-F

   • The authors report that the §III-B attack succeeds for every source-target pair, with a 95% interval of distortion from −15dB to −45dB; the authors state that −31dB is roughly equivalent to the loudness difference between the ambient noise of a quiet room and a person speaking (citing [38]).
   • The authors state that each additional character in the target phrase increases distortion by about 0.1dB; the longer the source phrase, the easier the targeting, and the two effects roughly cancel out, but the authors were unable to confirm this with statistical significance.
   • Generating a single adversarial example takes about 1 hour (single 1080Ti); using GPU parallelism, 10 can be constructed simultaneously and then fine-tuned one by one, taking only a few minutes each.

   Effect and cost of the improved loss (§III-C)

   • What was tested: repeat the §III-B evaluation, comparing the CTC loss with a fixed-alignment per-character logit loss.
   • Results: average distortion drops from −31dB to −38dB; Figure 2 shows a randomly selected sample with distortion −30dB, where the original waveform and the adversarial waveform are nearly indistinguishable when overlaid.
   • Cost and the authors' interpretation: the improved version is only guaranteed effective for greedy decoding; for beam-search decoding, the transcription is often closer to the target than the original phrase, but does not exactly match.

   Single-step method and the linearity assumption (§IV-A)

   • What was tested: interpolate between the original sample and the adversarial sample, comparing the CTC loss when advancing along the known adversarial direction versus along the FGSM direction (Figure 3).
   • Results: Figure 3 shows that the FGSM direction initially decreases the loss faster, but soon reaches a plateau and stops decreasing; the iterative optimization direction eventually reaches the adversarial sample (the authors note that the transcription is only correct when the CTC loss is below 10).
   • The authors' interpretation: the authors believe that the nonlinearity introduced by MFC and multiple LSTM layers makes single-step attacks ineffective for targeted audio adversarial examples, and that the "local linearity" explanation from images does not apply to audio; but FGSM can construct non-targeted audio adversarial examples (the optimization method has smaller distortion).

   Robustness modifications (§IV-B)

   • Pointwise noise: adversarial examples with minimal perturbation fail once pointwise random noise is added; using Expectation over Transforms [2] to generate high-confidence adversarial examples maintains adversariality under −30dB synthetic noise, with perturbation increasing by about 10dB.
   • MP3 compression: using a straight-through estimator [7] (treating the gradient of MP3 compression as identity) to construct samples robust to MP3 compression, distortion increases by about 15dB.

   Other ablations and analyses

   • Distortion interval: the 95% interval of distortion for the §III-B attack is −15dB to −45dB (main text).
   • Length effect: each additional character in the target is about +0.1dB; the longer the source, the easier (main text, not statistically significant).
   • Silence is easier: the authors state that targeted silence is easier than targeting a specific phrase, and any phrase can be turned into silence with distortion below −45dB (§III-F).
   • Negative results: minimal-perturbation samples are not robust to pointwise random noise (§IV-B); the improved loss version does not exactly match under beam-search decoding (§III-C); single-step FGSM is ineffective for targeted attacks (§IV-A).

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors list four open problems: playing over the air, universal perturbations, transferability, and defenses; the experiments cover only a single model and 100 samples.

   Limitations and future directions noted by the authors

   • Failure when played over the air: the authors point out that the adversarial examples constructed in this paper no longer remain adversarial after being played over the air, so the real-world threat is limited; the authors believe future work may produce audio adversarial examples that are effective over the air (§V).
   • Universal adversarial perturbations: whether there exists a single perturbation that can be added to any waveform to make it transcribe as a target phrase is listed by the authors as an open problem (§V).
   • Transferability: whether perturbations effective simultaneously against multiple classifiers can be generated has not been evaluated, and the authors consider evaluating transferability in the audio domain an important future direction (§V).
   • Applicability of defenses: the authors state that, to their knowledge, existing adversarial example defenses have only been evaluated in the image domain, and audio adversarial examples can provide another comparison point for defenses (§V).
   • Black-box extension: the authors expect the white-box attack in this paper can be extended to a black-box attack, analogous to subsequent developments in the image domain (§III-A).
   • Further study of property differences: the authors hope future work will continue to study audio adversarial examples, distinguishing the fundamental properties of adversarial examples from properties that hold only for image recognition (§VI).

   Scope of experimental coverage

   • The model under test is one model, the Mozilla implementation of DeepSpeech (§II).
   • The evaluation data are the first 100 instances of the Common Voice test set, each paired with 10 random target phrases (§III-A).
   • The improved loss version (§III-C) is only guaranteed to be effective for greedy decoding, and does not achieve an exact match under beam-search decoding.
   • Robustness experiments cover two transformations: pointwise random noise (−30dB) and MP3 compression (§IV-B).
   • The paper does not report the number of repetitions for each configuration, nor the success rate numbers for the improved loss version and the §III-D/E/F variants.

   Collapse this answer

6. 06Summarize the main content of the paper

   White-box iterative optimization targeted attack on DeepSpeech, 100% success rate, and demonstrates differences in the properties of audio and image adversarial examples.

   A paper on a targeted adversarial example attack against automatic speech recognition, attacking the Mozilla implementation of DeepSpeech.

   • Problem to solve: previously, the audio domain had only targeted attacks that require synthesizing new audio (hidden/inaudible voice commands) and Houdini, which can only target phonetically similar phrases; the authors want to verify whether arbitrary existing audio can be modified so that it is transcribed as an arbitrary target phrase.
   • Method highlights: white-box end-to-end iterative optimization, differentiating through MFC preprocessing to the raw sample points; first solve with CTC loss plus l2 relaxation and a gradually tightened dB constraint, then reduce distortion with a fixed-alignment per-character logit loss; Adam optimization, learning rate 10, at most 5,000 iterations.
   • Main results: on the first 100 test samples of Common Voice, each paired with 10 random target phrases, 100% success rate, average distortion −31dB (95% interval −15dB to −45dB); with the improved loss the average distortion drops to −38dB, but it is only guaranteed to be effective for greedy decoding.
   • Extended capabilities: can reach the theoretical maximum transcription density of 50 characters per second (average −18dB); can transcribe 5 seconds of classical music as an arbitrary phrase (average −20dB, 20,000 iterations); can hide arbitrary speech as silence (distortion below −45dB).
   • Property analysis: Figure 3 shows that the FGSM single-step direction quickly reaches a plateau, from which the authors conclude that the local linear explanation on images does not apply to audio; the minimum-perturbation samples are vulnerable to pointwise noise, but robustness to −30dB synthetic noise (perturbation about +10dB) and MP3 compression (about +15dB) can be obtained through Expectation over Transforms and the straight-through estimator, respectively.
   • Authors' conclusion: the authors state that targeted audio adversarial examples exist with a 100% success rate, opening a new field for adversarial example research; playing over the air, universal perturbations, transferability, and applicability of defenses are left as open problems.

   Collapse this answer

Related papers