Provably Robust Watermarks for Open-Source Language Models

The paper proposes a watermarking scheme for open-source large models that embeds a watermark by modifying model parameters and can be detected from model outputs. The authors prove that, under specific assumptions about the attacker's knowledge, the watermark is non-removable. Experiments on OPT-6.7B and OPT-1.3B show that the scheme is robust to token substitution and model parameter perturbation; among these, the stronger parameter perturbation attack requires reducing the quality score to 0/100 in order to lower the detection rate to 50%.