OpenAI Launches AI Tool to Detect Child Sexual Abuse Material
OpenAI released a new image classifier to detect child sexual abuse material and implemented strict account suspensions for ChatGPT users seeking such content.
OpenAI has released an AI-powered image classifier designed to detect child sexual abuse material (CSAM), including content generated by artificial intelligence. Developed in collaboration with the National Center for Missing & Exploited Children, the Tech Coalition, and the ROOST project, the tool analyzes visual signs of abuse rather than relying on digital fingerprints. This approach allows the system to flag previously unidentified material while reducing false positives by 48%.
The classifier is available at no cost via the OpenAI API to law enforcement and vetted organizations to help triage high-risk cases. To support these efforts, the company introduced AI-powered investigative tools and training to help officers analyze chat logs while minimizing their direct exposure to abusive imagery. OpenAI also joined the Tech Coalition's Lantern program to share exploitation data across platforms and published guidance for developers to prevent and report abuse.
Within ChatGPT, the company implemented new safeguards to deter users from seeking CSAM. Users attempting to find such material will receive warnings, have their conversations removed, and face an initial 24-hour account suspension. Repeated violations may lead to permanent account closure or reports to authorities. These initiatives were developed with input from several safety organizations, including Thorn and the Internet Watch Foundation.