TANGO: Embedding watermarks in token pairs for masked diffusion language models

The researchers propose TANGO, a watermarking method for masked diffusion language models that embeds watermarks into adjacent token pairs, and report that on two masked diffusion models it can detect almost all unedited watermarked text and most edited text. Masked diffusion models fill masked positions in parallel and without a fixed order, so the commonly used left-to-right watermarking is difficult to apply directly, while a fixed green list can be recovered and forged by comparing token frequencies of watermarked and unwatermarked text. TANGO uses a key to partition the vocabulary into color classes, biases new tokens according to the key and the color of a nearby already-decoded token, so that the preferred color varies with position and token frequencies are closer to unwatermarked text. Detection requires only the text and the key, does not assume a decoding order, and frequency attacks against a fixed green list do not work.