Averaging collusion attacks undermine the traceability of personalized neural audio watermarking

1. 01What problem does this paper attempt to solve?

   Traceability of personalized neural audio watermarking under multi-copy collusion: simple averaging alone can cause tracking failure.

   Core question: when an attacker obtains multiple personalized watermarked copies of the same recording, can a neural audio watermarking system still trace the leaker.

   • Scenario and importance: personalized audio watermarking embeds a user-specific payload in each distributed copy, so that after a leak the payload can be decoded to assign accountability; tracking failure (the decoded payload matches no registered user or points to an innocent party) may cause economic and reputational losses (Section 1).
   • Shortcomings of existing evaluations: the authors state that current evaluations mainly study distortions or attacks on a single copy (compression, resampling, filtering, noise, watermark removal and forgery, desynchronization and replay, overwriting), generally assuming the attacker operates on only one copy, and do not cover multi-copy scenarios.
   • Core observation: multiple personalized copies share the same audio content and differ mainly in the user-specific watermark signal, so sample-level averaging can be done without knowing the watermarking algorithm or model architecture; the authors find that this simple operation alone can change the decoded identity and break traceability.
   • Contributions: systematically evaluate the tracking failure of five neural audio watermarking systems under multi-copy collusion; analyze the two outcomes of untargeted and targeted misattribution; evaluate confidence screening as a mitigation (Section 1).
   • Threat model (Section 2):
     • Goal: make the decoded payload match no colluding member (untargeted), or equal a preselected non-member payload (targeted).
     • Knowledge: the basic setting uses only the colluding waveforms, with no clean reference, model parameters, or decoder queries; the payload-aware setting additionally knows the colluding payloads and candidate target payloads.
     • Capability: perform sample-level averaging over copies; the main analysis uses aligned, uncompressed copies, with additional tests on time offset and lossy encoding.
     • Victim systems: five personalized neural audio watermarking systems (AudioSeal, WavMark, TimbreWM, VoiceMark, WMCodec).

   Collapse this answer

2. 02What related research exists?

   Related work falls into three groups: neural audio watermarking systems, single-copy robustness evaluations, and classical collusion fingerprinting; this paper brings the collusion problem end-to-end to neural watermarking.

   Neural audio watermarking systems (attack targets)

   • AudioSeal (San Roman et al., 2024), WavMark (Chen et al., 2023), TimbreWM (Liu et al., 2024) — bit-based systems that encode messages in the waveform or spectrogram domain and directly give bit scores.
   • VoiceMark (Li et al., 2025), WMCodec (Zhou et al., 2025) — latent/codec-based systems that first recover latent classes and then map them to payload bits.

   Single-copy robustness evaluations

   • AudioMarkBench (Liu et al., 2024), Özer et al. (2025), IDEAW (Li et al., 2024) — evaluate distortions such as compression, resampling, and neural codecs, as well as removal and forgery attacks.
   • DRAW (Li et al., 2024) — targets desynchronization and replay attacks; Yao et al. (2026) — watermark overwriting attacks. The paper points out that these evaluations generally assume the attacker operates on only a single copy.

   Classical collusion fingerprinting and watermarking

   • Spread-spectrum watermarking by Cox et al. (1997) and multimedia fingerprinting by Trappe et al. (2003), Boneh and Shaw (1998), Tardos (2003), Celik et al. (2004), Tirkel et al. (2011) — generate fingerprints using predefined codes or watermark signals and analyze them under known collusion models.
   • Laarhoven (2014) — the classical interleaving rule, used in this paper to construct the ideal bit-choice reference as a control.

   Baselines and benchmarks: the attack control is the ideal bit-choice reference (each bit is chosen uniformly at random from the colluding members); data use AISHELL-3 and LibriSpeech.

   Positioning: the authors state that classical coding analysis cannot answer the end-to-end question for neural watermarking — after the effective watermark waveforms are averaged, how a learned decoder maps the mixture back to a payload, and whether that payload still points to a colluding member; this paper systematically studies this question.

   Collapse this answer

3. 03How does the paper address this problem?

   Uniform-averaging attack + Target-Bit Margin targeted weight optimization + decoder response analysis + confidence screening defense.

   Overall approach: perform sample-level averaging over multiple personalized copies of the same recording to mount an untargeted collusion attack, then use Target-Bit Margin to optimize the mixing weights under a payload-aware setting to achieve targeted misattribution, and explain the results and explore defenses through bit support analysis and confidence statistics.

   Problem Formulation

   • Let the shared speech waveform be x, the payload of user i be mi, and the watermark residual be w(x, mi) = yi − x; the user receives yi = x + w(x, mi).
   • For a collusion of K members, the uniform average is ȳ_K = 1/K∑_{i∈ C} y_i = x + 1/K∑_{i∈ C} w(x, m_i): the shared waveform is preserved, and the payload-related residuals are averaged.
   • Tracking success = the decoded payload equals the payload of some colluding member; equaling a predetermined non-member is targeted misattribution, and equaling another non-member is untargeted misattribution.

   Target-Bit Margin (targeted attack)

   • Optimize the weights using only the colluding payloads and candidate target payloads, without querying the decoder. The signed support for target bit j is γ_j = (2t_j−1)(∑_i a_i b_ij − 0.5), where bij is the j-th bit of member i, tj is the target bit, and ai is the weight.
   • Maximize the soft minimum of the target-bit margin, plus entropy regularization over the convex weights; require the effective collusion size Keff = (Σ ai²)⁻¹ ≥ ρK to prevent the weights from concentrating on a few copies.
   • Parameters: inverse temperature 8, entropy weight 0.05, ρ = 0.6, fixed across systems and collusion sizes; rank all non-member payloads by the optimized soft-minimum margin, and take the top 10 targets per trial.

   Decoder Response Analysis

   • Bit support: the number c of colluding payloads that have 1 at a given bit; in the ideal bit-choice reference, the probability that the output bit is 1 is c/K (the classical interleaving rule).
   • Majority agreement: bit agreement is the proportion of non-tie bits that agree with the colluding majority; trial agreement is the proportion of trials in which all non-tie bits follow the majority.
   • Cross-bit coupling: using two payloads A and B that differ in only one bit, vary the mixing weight from 0 to 1 (300 pairs) and observe whether shared bits other than the differing bit are also changed.

   Confidence Screening (defense)

   • Position confidence = the decoder's support for the chosen bit value: use native bit probabilities where available, the valid-window vote proportion for WavMark, and the sum of latent class probabilities consistent with each bit for VoiceMark/WMCodec.
   • Screen using three statistics: minimum confidence, mean confidence, and log-variance of confidence; thresholds are calibrated by Gaussian fitting (mean, log-variance) and empirical quantiles (minimum confidence) so that the rule retains at least 95% of valid single-copy outputs on calibration speakers; five-fold cross-validation (80% speakers for calibration, 20% for testing).

   Collapse this answer

4. 04What experiments did the paper conduct?

   On the five systems, the K=2 average tracking failure rate is 87.7–99.7%; TimbreWM's targeted success rate is 92.3%; confidence screening suppresses the target success rate to ≤4.4%.

   Experimental Setup

   • Systems under test: three bit-based systems, AudioSeal, WavMark (16-bit), and TimbreWM (10-bit); two latent-based systems, VoiceMark and WMCodec (16-bit).
   • Data: 300 ten-second recordings from 100 speakers, 3 per speaker, split evenly between AISHELL-3 and LibriSpeech.
   • Sampling rate: each system's native sampling rate—AudioSeal/WavMark/VoiceMark 16 kHz, TimbreWM 22.05 kHz, WMCodec 24 kHz.
   • Collusion size: K ∈ {2, 3, 5, 8}; for each recording, each system, and each K, one copy per member was generated with a different payload, and each copy could be decoded correctly before mixing.
   • Metrics: tracing-failure rate (proportion of trials with tracing failure), target success rate (proportion where the decoded payload matches the chosen non-member; for K ∈ {5, 8}, the proportion of exact matches among 300×10 attempts), PESQ/STOI (similarity to the first valid copy in the collusion).
   • Robustness conditions: at K=5, apply a ±10/±20/±50 ms shift to one copy, or first encode with MP3 128 kbps / Opus 64 kbps and then average.

   Main Results

   Tracing failure rate (TF%) and audio quality under uniform averaging (from Table 1, 300 trials per cell):

   Swipe horizontally to read the table

   System	K=2 TF	K=8 TF	K=2 PESQ/STOI	K=8 PESQ/STOI
   AudioSeal	99.0	100.0	4.60 / 0.999	4.58 / 0.999
   WavMark	96.7	99.3	4.57 / 0.999	4.52 / 0.998
   TimbreWM	87.7	92.0	4.56 / 0.998	4.53 / 0.996
   VoiceMark	93.7	98.7	4.20 / 0.983	4.05 / 0.971
   WMCodec	99.7	99.7	4.37 / 0.989	4.24 / 0.983

   • The authors state that just two valid copies can cause 87.7–99.7% tracing failure, and PESQ/STOI shows limited audio quality loss; the TF of each system is mostly close to the ideal bit-choice reference (16-bit reference K=2 is 97.9%, 10-bit is 89.1%).
   • At K=5 with time shift, average TF remains above 96.5%; under MP3/Opus encoding, average TF is between 96.5% and 96.9%, with PESQ above 4.25 and STOI above 0.989 (detailed data are in the code repository and not given in the paper).

   Decoder Response to Bit Support

   • Figure 2 (K=8): the proportion of decoded 1s for AudioSeal and WavMark rises with bit support, TimbreWM follows the collusion majority, and VoiceMark and WMCodec change little and remain near 0.5; the authors state that none of the five decoders follows the interleaved reference's linear c/K relationship.
   • Table 2 (K=8 majority agreement): TimbreWM's bit agreement and trial agreement are both 100.0%, WavMark is 95.0%/54.3%, AudioSeal 84.9%/20.7%, VoiceMark 61.9%/0.7%, and WMCodec 62.5%/1.0%.
   • The authors note that TimbreWM follows the majority at all non-tie bits, yet its K=8 tracing failure rate still reaches 92.0% (Table 1), i.e., consistent majority decoding can also produce non-member payloads.

   Cross-Bit Coupling and Targeted Misattribution

   • Cross-bit coupling (Figure 3, 300 payload pairs differing at only one bit): in 299 of 300 trials, AudioSeal returned only A or B; VoiceMark often returned payloads other than the two when weights were near 0.5, and these outputs necessarily changed shared bits.
   • Target-Bit Margin target success rate (from Table 3): TimbreWM is 78.4%/92.3% at K=5/K=8, WavMark 51.4%/54.4%, AudioSeal 41.9%/43.6%; VoiceMark is 1.1%/0.6% and WMCodec 1.0%/1.3%, despite the latter two having high tracing failure rates under uniform averaging.
   • The authors state that Target-Bit Margin is most effective when decoded bits vary predictably with collusion support; whether latent-aware attacks can achieve stronger control remains unresolved.

   Confidence Screening (Defense)

   • Figure 4 (K=8 minimum confidence distributions): the numbers of successful targeted trials for the Single, Average, and Targeted groups are AudioSeal n=294, WavMark n=300, TimbreWM n=300, VoiceMark n=19, WMCodec n=34 (Targeted n is the number of trials contributing one successful hit).
   • Table 4 (K=8): screening accepts about 95% of Single outputs (94.3%–95.0%), rejects all Average outputs of the three bit-based systems (100.0%), and the Average rejection rates for VoiceMark/WMCodec are 77.7%/88.0%.
   • Target success rate after screening: AudioSeal 43.6%→2.4%, WavMark 54.4%→4.4%, TimbreWM 92.3%→0.0%, VoiceMark 0.6%→0.3%, WMCodec 1.3%→0.7%; the authors state that after screening the target success rate drops to at most 4.4%.

   Other Ablations and Analyses

   • Time shift (K=5, ±10/±20/±50 ms): average TF is above 96.5% in all cases, and quality scores decrease as the shift increases (Section 5.1).
   • Lossy encoding (MP3 128 kbps / Opus 64 kbps, K=5): average TF 96.5%–96.9% (Section 5.1).
   • The authors note that a decrease in quality scores does not necessarily imply an equivalent loss of perceptual quality.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors state that latent-aware targeted attacks remain unresolved and confidence screening is only a preliminary mitigation; the experiments cover five systems and 300 recordings.

   Limitations and Future Directions Noted by the Authors

   • Target-Bit Margin exploits bit-level support; whether latent-aware attacks can achieve stronger control remains an open question (Section 5.4).
   • Confidence screening is described by the authors as a preliminary mitigation (Section 6, abstract).
   • The authors state that these findings motivate incorporating multi-copy collusion into neural audio watermarking evaluation (Section 7).
   • The final mapping from payload to user identity depends on registry coverage, rejection policy, and handling of unregistered payloads (Section 2).

   Scope of Experimental Coverage

   • The systems under test are five in total: AudioSeal, WavMark, TimbreWM, VoiceMark, and WMCodec (Section 4.1).
   • The data are 300 ten-second recordings from 100 speakers, split evenly between AISHELL-3 and LibriSpeech (Section 4.1).
   • Collusion sizes are K ∈ {2, 3, 5, 8}, with 300 trials per cell; targeted attacks are evaluated at K ∈ {5, 8} (Section 4.2).
   • The main analysis uses aligned, uncompressed copies; time shift and lossy encoding are evaluated only at K=5, with detailed results in the code repository and no specific numbers given in the paper (Sections 2, 5.1).
   • The paper does not report attack runtime or computational cost; optimization details for the targeted attack only point to the code repository (Section 4.2).

   Collapse this answer

6. 06Summarize the main content of the paper

   Averaging two copies across five systems breaks tracing (87.7–99.7%), the targeted attack reaches 92.3% on TimbreWM, and confidence screening can partially mitigate.

   A study of multi-copy collusion attacks against personalized neural audio watermarking. Personalized watermarking embeds a user-specific payload in each copy for leak accountability, but existing evaluations mostly assume the attacker has only a single copy.

   • Attack: the attacker performs sample-level uniform averaging over multiple valid copies of the same recording, requiring no watermarking algorithm, model parameters, or decoder queries; under a stronger payload-aware setting, Target-Bit Margin optimizes the mixing weights to steer the decoding result toward a preselected non-member.
   • Main results: for the five systems (AudioSeal, WavMark, TimbreWM, VoiceMark, WMCodec) under K=2 uniform averaging, the tracking failure rate is 87.7–99.7%, with PESQ ≥ 4.20 and STOI ≥ 0.983 (Table 1); with K=5 plus time offset or MP3/Opus encoding, the average TF remains above 96.5%.
   • Mechanism: the decoded bits of bit-based systems change predictably with the colluding bit support, and TimbreWM fully follows the majority at K=8 (Table 2), but majority decoding still produces non-member payloads; latent-based systems respond close to 0.5 and exhibit cross-bit coupling.
   • Targeted attack: Target-Bit Margin achieves a 92.3% target success rate at K=8 on TimbreWM, 43.6%/54.4% on AudioSeal/WavMark, and only about 1% on VoiceMark/WMCodec (Table 3).
   • Defense: screening based on minimum/mean confidence and log-variance retains about 95% of single-copy outputs, rejects all averaged outputs of bit-based systems, and reduces the target success rate to at most 4.4% (Table 4).
   • Authors' conclusion: multi-copy traceability should be an additional evaluation dimension for personalized neural audio watermarking; whether latent-aware attacks can be stronger remains unresolved, and confidence screening is only a preliminary mitigation.

   Collapse this answer

Related papers