Loading...
OpenAI launches new tools to combat child sexual abuse material
The new features include a new image classifier

OpenAI launches new tools to combat child sexual abuse material

Nov 17, 2026
12:13 pm

What's the story

OpenAI has launched a suite of tools aimed at combating child sexual abuse material (CSAM), amid rising concerns over the safety of AI technologies. The new features include a new image classifier for CSAM detection, alerts and temporary blocks to deter users from creating or accessing content that sexualizes children. A new guideline for developers and a set of tools for law enforcement agencies are also part of this initiative.

Tech details

New image classifier detects CSAM content

The new image classifier, which can detect CSAM images including those generated by AI, was developed in collaboration with the National Center for Missing and Exploited Children (NCMEC).

OpenAI is providing select companies free access to its classifier via its API.

The tool looks for visual signs of abuse in previously unseen images and assigns a score that guides reviewers on how to deal with CSAM.

Efficiency boost

Classifier reduces reviewers' workload

In early tests with partners, OpenAI's classifier has shown to flag 48% fewer images while detecting the same number of abusive ones. This means it cuts down on reviewers' workload.

Plus, images flagged by this tool are twice as likely to be abusive.

The company is also working with Tech Coalition and ROOST to share access to the tool further.

ADVERTISEMENT

User alerts

New warning screens in ChatGPT

OpenAI is also adding new warning screens in ChatGPT to prevent users from accessing material that sexualizes children.

The app already blocks CSAM content and related requests.

If a user tries to access such content, they will be shown an alert and directed toward specialized support services to prevent harmful behavior.

Their account could also be suspended for 24 hours as a warning.

ADVERTISEMENT

Developer guidelines

Guidelines for developers and tools for law enforcement agencies

OpenAI has also released guidelines for developers using its tools to prevent, detect, and report CSAM content.

In early tests, these guidelines have helped several developers adopt child safety tooling and register to report CSAM to the National Center for Missing & Exploited Children in the US.

The company is also equipping law enforcement agencies with AI-powered features like dedicated plug-ins, investigative tools, and complex data analysis capabilities.

ADVERTISEMENT