Asking Back: interaction-layer anti-distillation watermarking

The researchers propose interaction-layer anti-distillation watermarking, moving the anti-distillation marker from the output token layer up to the teacher model's interaction behavior. The method uses a system prompt to make the teacher model intermittently exhibit behaviors such as asking follow-up questions, low-frequency variants, or declarative paraphrasing; distilled students inherit these behaviors, and defenders then audit using black-box queries and LLM judging. In experiments with a Llama-3.3-70B-Instruct teacher on 63 LoRA-distilled students and 35,343 judged samples, the relative fidelity of watermark transfer to Gemma, OLMo, and Qwen students was 88.9%, 80.9%, and 45.2%, respectively; under non-adaptive DIPPER rewriting, the teacher's own upper bound was about 66.4%, student relative retention was 21-112%, and the OLMo student's watermark retention was higher than the teacher's own. A 20-person preregistered experiment showed Likert gaps between each marker variant and the baseline within 0.22 points, and the authors state that the interaction layer is a viable design location complementary to defenses at the token, model, and reasoning-trace layers.