Synthetic Speech, Real Threats: Evaluating Harmful Audio Generation by Large Text-to-Speech Models

The researchers propose the HARMGEN attack suite, which uses two categories totaling five attacks—semantic obfuscation and audio modality exploitation—to reduce commercial TTS systems' refusal rate for harmful content and increase the toxicity of generated speech. The first category uses Concat and Shuffle to hide harmful content in the text, and the second category uses Read, Spell, and Phoneme to inject harmful content through an auxiliary audio channel while keeping the text prompt harmless. The study evaluates on five commercial TTS systems based on large audio language models and three datasets covering two languages, and tests post-hoc handling by audio streaming platforms and pre-hoc defenses by TTS providers. Results show that deepfake detectors perform poorly on high-fidelity audio, post-hoc moderation can be bypassed by adversarial perturbations, and pre-hoc moderation can detect 57% to 93% of attacks.