Extracting Books from Production-Grade Language Models
Extracting books from production language models
The researchers propose a two-stage pipeline to extract copyrighted book text from four production-grade LLMs: Claude 3.7 Sonnet, GPT-4.1, Gemini 2.5 Pro, and Grok 3. The first stage uses a short excerpt from the book's beginning to probe whether continuation is possible; for Claude 3.7 Sonnet and GPT-4.1, a Best-of-N jailbreak is needed, while Gemini 2.5 Pro and Grok 3 comply directly. The second stage repeatedly requests continuations to complete long-form extraction. Measured by approximate verbatim recall, the jailbroken Claude 3.7 Sonnet achieves 95.8% on Harry Potter and the Philosopher's Stone, Gemini 2.5 Pro and Grok 3 reach 76.8% and 70.3% respectively, and GPT-4.1 only 4.0%, refusing to continue at the end of the first chapter. The study tested 13 books, 11 of which are copyrighted in the United States; the authors notified the relevant vendors in September 2025 and disclosed the results after a 90-day disclosure period.
论文给出了从四款生产级 LLM 中提取受版权书籍的具体流程与成功率，可用于评估记忆化与输出防护的实际强度。