AI
OpenAI Launches Child Sexual Abuse Material Image Classifier and ChatGPT Warnings
OpenAI released several tools to prevent CSAM, including a classifier that can identify AI-generated child sexual abuse images, ChatGPT warnings and a 24-hour suspension mechanism, developer guidance, and law enforcement tools. Early partner testing showed the classifier flagged 48% fewer images while finding the same number of abuse images.
OpenAI announced new tools to address child sexual abuse material (CSAM), covering a new image classifier, ChatGPT warnings and temporary bans, guidance for developers, and a set of tools for law enforcement. The company said the new classifier can detect CSAM images, including AI-generated images, and is being made available free through the API to some companies. The tool looks for previously unseen visual signs of abuse in images and provides scores, giving moderators direction for their judgments.
OpenAI said the classifier was developed jointly with the U.S. National Center for Missing & Exploited Children (NCMEC) and that it is working with Tech Coalition and ROOST to share access. In early testing with partners, the classifier reduced the number of images flagged by 48% while finding the same number of abuse images, thereby reducing moderation workload; images it flagged were twice as likely to be abuse images.
ChatGPT will add a warning page. When users attempt prompts that sexualize children, the system will display a warning directing them to specialized support to stop harmful behavior and will suspend the account for 24 hours as a warning. The platform already blocks CSAM content and related requests, and OpenAI said repeated violations may result in permanent account deletion. The relevant information and guidance were developed with participation from organizations and researchers including Protect Children, the Lucy Faithfull Foundation, and Joel Scanlan of the University of Tasmania, who also independently evaluated the deterrent-message trials.
OpenAI also released developer guidance explaining how to use its tools to prevent, detect, and help users report CSAM. The company said early testing has helped several developers adopt child-safety tools and register to report to NCMEC. For law enforcement personnel handling child-safety cases, OpenAI is also providing dedicated plugins, investigative tools, and AI features such as complex data analysis, with access managed by project administrators, along with investigator training and field guides for AI use.