Characterizing Audio Adversarial Examples Using Temporal Dependency

1. 01What problem does this paper attempt to solve?

   Explores whether image adversarial defense experience can transfer to audio, and proposes using audio temporal dependencies to detect speech adversarial examples.

   Core question: can the experience of image adversarial example defenses transfer to the audio domain, and can the temporal dependency inherent to audio be used to distinguish adversarial examples.

   • Problem setting: DNN speech recognition systems (such as Google Home and Amazon Alexa) have been shown to be vulnerable to audio adversarial example attacks; an attacker can mislead the model by adding perturbations imperceptible to human ears. The authors argue that different types of data have their own unique properties that may be used to gain discriminative power against adversarial examples.
   • Shortcomings of existing methods: input transformation is a common defense in the image domain, but the authors point out that many methods have been bypassed by subsequent or adaptive attacks; Athalye et al. (2018) pointed out that input transformations cause obfuscated gradients, giving a false sense of robustness. The authors' experiments find that audio input transformations have the same weakness.
   • Core hypothesis: the transcription of normal audio is consistent across different segments (temporal dependency), whereas adversarial perturbations aim to twist the output toward the target text and may break this consistency; therefore, comparing the consistency between segment transcriptions and the whole-segment transcription can identify adversarial inputs.
   • Paper contributions: systematically evaluate four audio input transformation defenses, propose the TD detection method based on temporal dependency, and design strong adaptive attacks against TD to test its robustness.
   • Threat model:
     • Attacker goal: generate adversarial audio that causes a classification model to misclassify (targeted/untargeted) or causes ASR to output a specified target text (targeted speech-to-text).
     • Knowledge: under the adaptive attack setting, the attacker knows all details of the defense implementation, including the k value of the TD method.
     • Capability: add small perturbations to the original waveform; can carry out gradient-obfuscation-aware adaptive attacks against input transformations.
     • Victim systems: the DeepSpeech speech transcription network, a convolutional speech command classification model, and the Kaldi speech recognition platform.

   Collapse this answer

2. 02What related research exists?

   Related work falls into three categories: audio adversarial attacks, image input transformation defenses and their circumvention, and adversarial training; this paper positions itself as a detection method that exploits intrinsic properties of the data.

   Audio adversarial example attacks

   • Alzantot et al. (2018) propose a genetic algorithm for a Speech-to-Label attack; Cisse et al. (2017) propose a probabilistic loss function (Houdini).
   • Carlini & Wagner (2018) use an optimization-based method to achieve a targeted Speech-to-Text attack on the raw waveform; Iter et al. (2017) use MFCC features; Yuan et al. (2018) propose the Commander Song "wav-to-API" attack that can be played over the air.

   Input transformation defenses in the image domain and their limitations

   • Input transformations (bit quantization, image filtering, autoencoder reconstruction, etc.) are widely used because of low cost and easy integration (Luo et al., 2015; Wang et al., 2016; Dziugaite et al., 2016; Xu et al., 2017; Guo et al., 2017; Meng & Chen, 2017).
   • The paper points out that these defenses are mostly circumvented by subsequent or adaptive attacks (Carlini & Wagner, 2017b; He et al., 2017; Carlini & Wagner, 2017c; Lu et al., 2018); Athalye et al. (2018) point out that input transformations cause gradient obfuscation, creating a false sense of robustness, and that this problem can be circumvented.

   Adversarial training and data augmentation

   • Serdyuk et al. (2016), Michelsanti & Tan (2017), Sriram et al. (2017), Sun et al. (2018) improve speech model robustness through adversarial training or data augmentation; the paper explicitly states that this is beyond the scope of this paper.

   Baselines and benchmarks: the baseline for the detection method is a one-layer LSTM (64-dimensional hidden layer) classifier trained on collected adversarial/normal audio; the defense comparison targets are four input transformations. The datasets are LibriSpeech, Mozilla Common Voice, and Speech Commands.

   The authors position this paper as: obtaining discriminative power against adversarial examples through temporal dependencies embedded in audio, compatible with any ASR model, without requiring adversarial training or data augmentation.

   Collapse this answer

3. 03How does the paper address this problem?

   First evaluate four audio input transformation defenses, then propose the TD detection method that compares the consistency between segment and whole-segment transcriptions, and test both with adaptive attacks.

   The overall approach has two parts: port input transformation defenses from the image domain to audio and evaluate their effectiveness; propose a TD (temporal dependency) detection method that exploits audio temporal dependencies, and design adaptive attacks to test both routes.

   Audio input transformations (defenses)

   • Quantization: round the sample amplitude to the nearest integer multiple of q; the perturbation amplitude is usually small and can be eliminated; parameter q = 128, 256, 512, 1024.
   • Local smoothing: use a fixed-length sliding window, take the K−1 samples before and after each sample as the reference sequence, and replace with the mean or median.
   • Down sampling: according to the sampling theorem, downsample 16kHz audio to 8kHz and then restore it, eliminating the perturbation during reconstruction.
   • Autoencoder: modeled on the image defense MagNet, use a sequence-to-sequence autoencoder to split audio into frame-level segments, reconstruct each separately, and then concatenate them; the paper states that passing the whole segment directly through the autoencoder performs poorly and makes it difficult to exploit low-level information.

   Temporal dependency detection method (TD)

   • Process: take the first k segments (prefix) of the audio and input them into ASR to obtain the transcription Sk; pass the whole audio through ASR and take the first k prefix S{whole,k} of the transcription result; compare the consistency of the two (Figure 1).
   • Hypothesis: normal audio, due to temporal dependency, should have Sk and S{whole,k} similar; adversarial perturbations aim to twist the output toward the target text and may fail to preserve the temporal information of the original sequence, causing the two to be inconsistent.
   • Distance metrics: word error rate WER, character error rate CER, and longest common prefix LCP (defined as max_{b1[:k]=b2[:k]} k, i.e., the maximum length of the identical prefix of the two strings).

   Evaluation methods for defense and detection

   • Defense metrics: WER/CER measure the recovery effect; effectiveness ratios R_benign = D(g(T(x)),y)/D(g(x),y) and Radv measure the impact of the transformation on normal samples and its ability to recover adversarial samples, respectively; Rbenign close to 1 indicates little impact on normal samples, and a small Radv indicates effective recovery.
   • Detection metrics: AUC, measuring the ability to capture adversarial samples (true positive) without falsely flagging normal samples (false positive).

   Adaptive attack design

   • Against input transformations: the attacker knows the transformation details and uses gradient-mask-aware methods: the quantization attack changes the optimization objective to min_δ |qδ|₂² + c · l(x+qδ, t); the downsampling attack computes gradients directly on the sampled elements (the process is differentiable); the median smoothing attack backpropagates gradients to the median point for updating (similar to max-pooling backpropagation).
   • Against TD: the authors state that this process is non-differentiable, and even if the attacker knows k it is difficult to attack directly, so they design three strong adaptive attacks: segment attack (perturb only the first k segments), concatenation attack (attack the two segments separately and then concatenate, in two ways), combination attack (the optimization objective includes the CTC-loss of both the whole sentence and the first k segments, i.e., min_δ |δ|₂² + c · (l(x+δ, t) + l((x+δ)_k, t_k))), and consider settings where the attacker's kA and the defender's kD are the same/different, and where kD is randomly sampled.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Evaluated on three datasets and three attacks: input transformation defenses are circumvented by adaptive attacks; TD detection achieves AUC of 0.930–0.936, and the authors state it resists the adaptive attacks considered.

   Experimental setup

   • Models tested: DeepSpeech (biRNN + beam search) for the speech-to-text task; a convolutional speech command classification model for the audio classification task; the Commander Song attack is evaluated on the Kaldi platform.
   • Datasets: LibriSpeech (test-clean, average duration 4.294s), Mozilla Common Voice (16kHz, average 3.998s, the first 100 entries of the test set are used for attacks, same setup as Carlini & Wagner 2018), Speech Commands (65000 single-command audio clips, 1 second each).
   • Attack methods: GA (Alzantot et al. 2018, classification attack, 500 iterations, perturbation magnitude limit 5, 50 adversarial examples generated per attack task), Commander Song (adversarial audio provided by the authors, 25 samples obtained via private communication), Opt (Carlini & Wagner 2018, optimization attack based on CTC-loss, 50 selected per dataset).
   • Metrics: for defense, WER/CER and effectiveness ratio Rbenign/Radv; for classification tasks, attack success rate; for detection, AUC; for Commander, detection rate (threshold set to 0, following the setup of Yuan et al. 2018).
   • Detection baseline: a one-layer LSTM (64-dimensional hidden layer) trained on collected adversarial and normal audio for binary classification.
   • Number of repetitions: not reported in the paper.

   Main results

   According to Table 1 and Table 3:

   Swipe horizontally to read the table

   Method	GA (SpeechCommand, attack success rate)	Commander (detection rate)	Opt/LibriSpeech	Opt/CommonVoice
   No defense	84%	100% (target command recognition rate)	1.0/1.0 (Rbenign/Radv)	1.0/1.0
   Down Samp.	3.2%	8%	3.87/0.40	1.13/0.73
   Quan-256	2.1%	4%	1.13/0.28	1.56/0.74
   Median-4	2.7%	4%	1.18/0.34	0.98/0.77
   Autoencoder	8.2%	not reported	9.84/0.97	2.09/0.80
   TD detection (AUC)	not evaluated	1.00 (detection rate)	0.930	0.936

   • The authors believe input transformations are more effective for classification tasks; a possible reason is that classification does not rely on temporal dependencies but focuses on local features, and speech-to-text is harder to defend with input transformations.
   • The authors state that the autoencoder also has negative effects on normal samples (WER rises from 27.5 to 57.6 on Common Voice, and from 12.4 to 30.0 on LIBRIS), and cannot restore adversarial audio (WER 76.5 and 99.4 respectively).
   • TD was not evaluated on the classification task because each audio clip in Speech Commands is only 1 second long and temporal dependencies are not obvious.

   Detailed results of TD detection

   • What was measured: AUC comparison of three metrics (WER/CER/LCP) with the LSTM baseline at k = 1/2 (Table 3).
   • Results: on Common Voice, TD(WER) 0.936, TD(CER) 0.916, TD(LCP) 0.859, LSTM 0.712; on LIBRIS, TD(WER) 0.930, TD(CER) 0.933, TD(LCP) 0.806, LSTM 0.645.
   • Authors' interpretation: the TD method is easy to implement and can effectively distinguish adversarial audio; at k = 4/5, the AUC of CER can reach 0.969 (Table A6; the paper does not state which dataset this value belongs to).

   Adaptive attack evaluation

   • Against input transformations (Table 4, 50 samples per dataset): adaptive attack success rate against Down Samp. is 92% (LibriSpeech)/90% (CommonVoice), against Quan-256 is 98%/100%, against Median-4 is 98%/96%; most perturbations fall in the −15dB to −45dB range (Table 6, e.g., Quan-256 on CommonVoice is −9.48dB), which the authors state is tolerable to the human ear.
   • Against TD: the segment attack achieves a success rate of only 2% on 50 samples per dataset across the two datasets; the concatenation attack fails (transcription after concatenation is inconsistent with segmented transcription); the authors speculate the reason is that Sk alone is insufficient to be attacked to the target and concatenation disrupts the perturbation.
   • combination attack (Table 5, A7): when the attacker's kA and the defender's kD are the same, the attack is relatively effective (e.g., when kA={1/2}, kD=1/2, AUC drops to 0.607/0.518/0.643); when kA and kD differ or kD is randomly sampled, AUC is mostly above 0.8; the authors state that TD with randomly sampled kD is robust in all cases.

   Other ablations and analyses

   • Effect of k value (Table A6): as k goes from 1/2 to 5/6, the AUC of WER is between 0.930–0.955 and CER between 0.933–0.969; the authors state the results do not change much.
   • Recovery after GA transformation: Quan-256 restores the original label for 63.8% of adversarial examples; normal audio classification accuracy drops slightly from 89.2% to 89.0%.
   • Results without language model (Table A3, A4): the WER/CER trends under each transformation are consistent with those with the language model.
   • Commander detection: Quan-256 achieves a 100% detection rate on 25 samples (threshold 0); TD with k=1/2 detects all samples as inconsistent.

   Collapse this answer

5. 05What are some points that can be further explored?

   The paper does not specifically discuss limitations; the experiments cover three datasets and three attacks, with a sample size mostly of 50 per condition.

   Limitations and future directions noted by the authors

   • The paper does not specifically discuss limitations. In the conclusion, the authors state that methods that exploit unique properties of data to improve robustness can naturally extend to other data types (e.g., video, multimodal); this is a directional outlook rather than an explicitly listed limitation.

   Scope of experimental coverage

   • The models tested are DeepSpeech, a convolutional speech command classification model, and the Kaldi platform, 3 in total; the attacks cover GA, Commander Song, and Opt, three types.
   • The datasets are LibriSpeech, Common Voice, and Speech Commands; the Opt attack uses 50 samples per dataset, GA uses 50 samples per attack task, and Commander has 25 samples (obtained via private communication).
   • TD detection is evaluated only on speech-to-text attacks (Commander, Opt), not on the 1-second single-command classification task; no autoencoder was trained on Commander (training data not public).
   • The adaptive attack evaluation covers three input transformations and three types of attacks targeting TD (segment, concatenation, combination), mainly conducted on the Opt attack.
   • The paper does not report the specific selection of detection thresholds (except Commander, which uses threshold 0), false positive rate values, number of experimental repetitions, or statistical significance tests.

   Collapse this answer

6. 06Summarize the main content of the paper

   The authors propose using audio temporal dependencies to detect speech adversarial examples, achieving AUC of 0.930–0.936, and state that it resists the adaptive attacks considered, whereas input transformation defenses do not.

   A paper on defenses against audio adversarial examples, whose core claim is to use the temporal dependency inherent in audio to obtain discriminative power against adversarial examples.

   • Questions: Can input transformation defenses from the image domain transfer to audio? Can audio temporal dependencies be used to distinguish adversarial examples?
   • Methods: Evaluate four audio input transformations (quantization, local smoothing, downsampling, autoencoder); propose the TD detection method—comparing the consistency between the transcription of the first k segments of the audio and the first k prefix of the full transcription (WER/CER/LCP metrics); design adaptive attacks targeting both types of methods.
   • Main results:
     • Input transformations have some effect against non-adaptive attacks (Quan-256 reduces the GA attack success rate from 84% to 2.1%), but under adaptive attacks the success rate rises back to 90%–100% (Table 4).
     • The autoencoder defense is ineffective and harms normal samples (normal WER on Common Voice rises from 27.5 to 57.6).
     • TD detection achieves AUC of 0.936 (Common Voice)/0.930 (LIBRIS) on the Opt attack, higher than the LSTM baseline's 0.712/0.645; detection rate on Commander Song is 1.00.
     • The segment attack targeting TD has a success rate of only 2%, and the concatenation attack fails; when the defender randomly samples k, the combination attack is also difficult to make work (per the authors).
   • Authors' conclusion: The weaknesses of image defenses are likely to transfer to the audio domain; exploiting unique properties of data (such as audio temporal dependencies) can build defenses that resist adaptive attacks, and the method requires no model retraining and is easy to operate.

   Collapse this answer

Related papers