Skip to content

AudioTrace: File-Level Membership Detection for Audio LLMs

AudioTrace: File-Level Membership Detection for Audio Large Language Models

arXiv: 其他 AI 安全论文· arXiv:2611.21088· Guangzhi Sun, Xiao Zhan, Shuai Wang, Jose Such, Phil WoodlandUniversity of Cambridge; School of Intelligence Science and Technology, Nanjing University; State Key Lab. of Novel Software Technology, Nanjing University; INGENIO (CSIC-Universitat Politècnica de València)Indexed by this site · 2 days ago· Originally publishedAttacks· Its place in the scan record →

Translation in progress — untranslated text is omitted.

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

AI summaryThe paper proposes AudioTrace, for determining whether a given recording was used to train an audio LLM without assuming knowledge of the recording's training task and annotations; the authors call this the first systematic study of this challenge. Training audio LLMs on public recordings raises privacy and copyright concerns, while existing membership inference attacks typically assume knowledge of the full training samples and their textual supervision, whereas in reality recordings often come with private transcripts, descriptions or question-answer pairs that owners cannot access. The authors build the AudioTraceBench benchmark, containing 1,285 recordings with explicit fine-tuning membership labels; evaluations on three audio LLMs show that existing text and video membership inference methods, when transferred to audio, are close to random guessing. AudioTrace uses the target model to generate audio-conditioned proxy questions, extracts membership evidence from its answers and aggregates it per recording, reaching an AUROC of 0.60, which further increases with greater fine-tuning exposure or adaptation capability.

Questions
Public audio being used to train Audio LLMs raises privacy and copyright concerns, but recording owners usually cannot obtain the transcriptions, annotations, or QA pairs used in training, and so cannot determine without knowing the training annotations whether their recordings were used for fine-tuning.
Methods
The authors construct AudioTraceBench, containing 1,285 recordings with controlled fine-tuning membership labels, and propose AudioTrace: the target model self-generates proxy questions and answers them, membership evidence is extracted from token-level statistics of the answers, and then aggregated into file-level scores by top-K averaging.
Experiments and results
On Qwen2.5-Omni, SALMONN-2, and Step-Audio-2-mini, the adapted text/video methods have AUROC close to 0.5, while AudioTrace reaches 0.55–0.60; the privileged setting with known training QA reaches about 0.82, which the authors state shows the difficulty of the unknown-annotation setting.
Limitations and future work
The authors note: AudioTrace's inference compute is about 5 times that of direct audio-level methods; the evaluation is limited to the controlled SFT setting; selecting effective proxy questions under unknown training details remains difficult, and the authors plan to explore adaptive query selection.
Read originalarxiv.org