OpenAI releases an AI image classifier and safety tools for detecting child sexual abuse material
OpenAI Releases AI Image Classifier and Safety Tools to Help Detect Child Abuse Material
On November 16, OpenAI released an image classifier for detecting child sexual abuse material (CSAM) that can identify abusive content, including AI-generated images, while providing additional protections for ChatGPT users and launching developer resources and law enforcement investigation tools. The classifier scores suspected abusive images based on visual indicators to help reviewers prioritize checks; OpenAI says the tool was trained and evaluated with the assistance of the U.S. National Center for Missing & Exploited Children (NCMEC) and is available free of charge through the OpenAI API to vetted organizations. The report also reminds companies that using AI to identify harmful content requires accompanying processes for evaluating detection accuracy, managing false positives, and handling sensitive information.