Skip to content
The Verge AI· Lauren Feiner·Indexed by this site · Originally published· Tools →· Its place in the scan record →

OpenAI wants to stop deepfakes from overwhelming child abuse reporting hotlines

OpenAI wants to stop deepfakes from overwhelming child abuse tiplines

AI summary

OpenAI announced a classifier that detects child sexual abuse material (CSAM), saying it can identify whether an image is AI-generated; it will be made available free to organizations that pass review, and OpenAI will provide detection guidance to developers and help train law enforcement personnel. NCMEC's CyberTipline received 400,000 generative AI-related reports in 2025, an exponential increase from 67,000 in 2024. OpenAI says images flagged by the classifier decreased by 48%, and flagged images were twice as likely to contain abuse material; John Buckley, its head of child safety public policy, says this is the first widely available CSAM classifier specifically for flagging AI-generated content. ChatGPT's image generator already refuses CSAM requests; now those who try will see a new warning, and first-time violators will have access suspended for 24 hours and be told their account may be permanently closed. North Carolina's attorney general said at the event that the surge in reports is "explosive," and that this approach of partnering with companies to prevent harm remains experimental.

Read originaltheverge.com