Localize-and-Detect: Auditing task-level poisoning in instruction-tuned models

Researchers propose Localize-and-Detect, a two-stage black-box auditing method for task-level poisoning in instruction-tuned models that requires only the outputs of the base model and the fine-tuned model. Task-level poisoning exploits the task structure of fine-tuning to make the model generate attacker-specified biased content on a specific target task without requiring an explicit input trigger. The first stage of the method locates the target task by finding the candidate task with the largest difference in next-token distributions between the two models, and the second stage searches the candidate tasks for biased content that repeatedly appears in the fine-tuned model but not in the base model. The authors evaluated on 216 poisoned models across two model families, covering different target tasks, poisoning methods, poisoning budgets, and types of biased content. Results show that the method can effectively locate the target task and detect biased content, that detection results correspond to attack success, and that there are few false positives on clean models.