Extracting Books from Production-Grade Language Models
Extracting books from production language models
Quick read
Based on 论文全文 （AI 生成）; refer to the original
The researchers propose a two-stage pipeline to extract copyrighted book text from four production-grade LLMs: Claude 3.7 Sonnet, GPT-4.1, Gemini 2.5 Pro, and Grok 3. The first stage uses a short excerpt from the book's beginning to probe whether continuation is possible; for Claude 3.7 Sonnet and GPT-4.1, a Best-of-N jailbreak is needed, while Gemini 2.5 Pro and Grok 3 comply directly. The second stage repeatedly requests continuations to complete long-form extraction. Measured by approximate verbatim recall, the jailbroken Claude 3.7 Sonnet achieves 95.8% on Harry Potter and the Philosopher's Stone, Gemini 2.5 Pro and Grok 3 reach 76.8% and 70.3% respectively, and GPT-4.1 only 4.0%, refusing to continue at the end of the first chapter. The study tested 13 books, 11 of which are copyrighted in the United States; the authors notified the relevant vendors in September 2025 and disclosed the results after a 90-day disclosure period.
- 生产型 LLM 的版权争议核心在于记忆化：训练数据是否被编码进权重、能否在输出中被抽取。开放权重模型已被证明可抽取大量受版权文本，但对带有模型级和系统级安全措施的生产 LLM 是否同样可行尚不清楚。
- 用两阶段流程：先用 Best-of-N 越狱（有时不需要）尝试补全书中一个真实前缀，成功后反复请求模型续写以抽取长文本，再用基于最长公共子串贪心近似的块匹配指标 nv-recall 衡量抽取比例。在 Claude 3.7 Sonnet、GPT-4.1、Gemini 2.5 Pro、Grok 3 上测试。
- 四款生产 LLM 均能抽出大量受版权书籍文本。Gemini 2.5 Pro 和 Grok 3 无需越狱即可完成探测与续写；Claude 3.7 Sonnet 越狱后在某些配置下近乎逐字抽出整本书（如 nv-recall 达 95.8%），GPT-4.1 需更多越狱尝试且常在第一章节末拒绝继续。作者强调各 LLM 生成配置不同，结果只描述具体实验，不比较各模型的整体抽取风险。
- 实验规模小、仅覆盖十四本书和固定时间窗口，生产 API 会变化且难以复现；部分模型单次抽取成本超 100 美元。作者只做描述性陈述，不做跨模型比较评价，并指出未评估非逐字生成文本，抽取测量可能低估实际记忆量。
论文给出了从四款生产级 LLM 中提取受版权书籍的具体流程与成功率，可用于评估记忆化与输出防护的实际强度。