Pop Quiz Attack: A Black-Box Membership Inference Attack Against Large Language Models

Researchers propose Pop Quiz Attack, a black-box membership inference attack against large language models that rewrites target data into quiz-style multiple-choice questions and infers from the model's answers whether a sample belongs to the training data. Across six models—GPT-3.5, GPT-4o, LLaMA2-7b, LLaMA2-13b, Mistral-7b, and Vicuna-7b—and four datasets, the method achieves an average ROC-AUC of 0.873, 20.6% higher than existing methods. The authors analyze the impact of query complexity, data type, data structure, and training settings on attack effectiveness, and evaluate instruction-based, filtering-based, and differential privacy-based defenses; these defenses reduce attack effectiveness but cannot eliminate the risk. The authors state that the results highlight persistent privacy vulnerabilities in modern large language models.