OpenAI launches new tools to combat child sexual abuse material
What's the story
OpenAI has launched a suite of tools aimed at combating child sexual abuse material (CSAM), amid rising concerns over the safety of AI technologies. The new features include a new image classifier for CSAM detection, alerts and temporary blocks to deter users from creating or accessing content that sexualizes children. A new guideline for developers and a set of tools for law enforcement agencies are also part of this initiative.
Tech details
New image classifier detects CSAM content
The new image classifier, which can detect CSAM images including those generated by AI, was developed in collaboration with the National Center for Missing and Exploited Children (NCMEC).
OpenAI is providing select companies free access to its classifier via its API.
The tool looks for visual signs of abuse in previously unseen images and assigns a score that guides reviewers on how to deal with CSAM.
Efficiency boost
Classifier reduces reviewers' workload
In early tests with partners, OpenAI's classifier has shown to flag 48% fewer images while detecting the same number of abusive ones. This means it cuts down on reviewers' workload.
Plus, images flagged by this tool are twice as likely to be abusive.
The company is also working with Tech Coalition and ROOST to share access to the tool further.
User alerts
New warning screens in ChatGPT
OpenAI is also adding new warning screens in ChatGPT to prevent users from accessing material that sexualizes children.
The app already blocks CSAM content and related requests.
If a user tries to access such content, they will be shown an alert and directed toward specialized support services to prevent harmful behavior.
Their account could also be suspended for 24 hours as a warning.
Developer guidelines
Guidelines for developers and tools for law enforcement agencies
OpenAI has also released guidelines for developers using its tools to prevent, detect, and report CSAM content.
In early tests, these guidelines have helped several developers adopt child safety tooling and register to report CSAM to the National Center for Missing & Exploited Children in the US.
The company is also equipping law enforcement agencies with AI-powered features like dedicated plug-ins, investigative tools, and complex data analysis capabilities.