Generative AI red teaming: Copyright-specific testing report of the Dana-Farber Cancer Institute

Dana-Farber Cancer Institute conducted a copyright compliance red-team test of its internal generative AI tool GPT4DFCI, which is based on OpenAI models. In November 2024, 42 participants from academia, industry, and government agencies formed four teams and attempted to extract copyrighted material from four content types: literary works, news articles, scientific papers, and access-restricted clinical notes. The test succeeded in verbatim extraction of book dedications and obtained near-original passages; news articles could not be extracted even under jailbreak attempts, scientific papers yielded only high-level summaries, and the clinical notes test showed that privacy protections were effective. The authors note that the successful extraction of literary works suggests that copyrighted material may be present in the training data and requires inference-time filtering, and that the differences in success rates across content types reflect different protection mechanisms. This test led GPT4DFCI to deploy a copyright-specific meta-prompt, which has been in production since January 2025.