OpenAI releases new image classifier and ChatGPT warning mechanism to combat child sexual abuse material
OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM
OpenAI released a set of safety measures to combat child sexual abuse material (CSAM), including an image classifier, ChatGPT warnings and temporary bans, developer guidance and law enforcement tools. The new classifier was developed with NCMEC and can detect CSAM images including AI-generated ones; it is available free of charge through the API to some companies and shared in partnership with Tech Coalition and ROOST; early testing showed flagged volume decreased by 48% while still detecting the same number of abusive images, and flagged images were twice as likely to be abusive. ChatGPT will show a warning to users attempting to create or access sexualized content involving minors, direct them to professional support and suspend the account for 24 hours, with repeat offenders possibly permanently banned. OpenAI also released developer guidance and provides law enforcement with a dedicated plugin, investigation tools, data analysis and training. OpenAI faces multiple lawsuits alleging its technology harms children, and Common Sense Media has called its technology an unacceptable risk to teens.