Fundamental limitations of pointwise defenses in LLM fine-tuning APIs

AI summaryThe researchers propose a class of pointwise undetectable fine-tuning attacks, claiming they bypass single-sample detection in fine-tuning APIs using all-benign training and inference samples, and induce harmful multiple-choice answers on the OpenAI fine-tuning API. The attacks rewrite harmful questions into benign requests: one is a classification request asking the model to judge whether a question is harmful, and the other embeds the harmful question in an unrelated conversation about randomly naming a flower, then encodes the answer back to the user using systematic wording or flower differences in the model's responses. On GPT-4o, both attacks achieve 98.4% accuracy on self-built IED multiple-choice questions and 100% on copyright multiple-choice questions, and are largely not intercepted by the authors' self-built model specification and suspicious behavior monitoring; previous ciphertext-like attacks were intercepted about 66% to 77% under suspicious behavior monitoring. The authors also extend the attacks to scenarios where training data contains no harmful text at all and where each option corresponds to multiple outputs, and accordingly argue that pointwise defenses have fundamental limitations, recommending a shift to multi-sample detection.