Fairness, Safety, and Security Evaluation of Audio-Language Models

The researchers propose a taxonomy and a unified evaluation framework for audio-language models, and on two representative systems find systematic differences between speech and text inputs in refusal rate, attack success rate, and toxicity. The taxonomy divides systems along two axes: first, the form of audio input representation (discrete or continuous); second, where semantic reasoning occurs (cascaded, multimodal, or audio-native). The evaluation framework covers three aspects: semantic invariance under paralinguistic variation, refusal and toxicity behavior under unsafe prompts, and robustness to adversarial audio perturbations. On this basis, the authors argue that fairness, safety, and security behavior is closely related to how acoustic information is integrated into semantic reasoning, and that structure-aware evaluation of audio-language models is needed.