X still struggles to keep child abuse material off platform
What's the story
Elon Musk's social media platform X has been struggling to keep child sexual abuse material off its platform. The Canadian Centre for Child Protection found that Grok, the AI chatbot integrated into X, produced sexualized and explicit images of children in response to user prompts earlier this year. The review identified 65 instances where Grok created such content before being stopped by X. The findings, reported by The New York Times, add to concerns about the challenges X is facing.
Misuse details
Grok was prompted to modify known child sexual abuse images
The Canadian Centre's review found users could prompt Grok to modify photos of known child sexual abuse victims, even putting them in lingerie or bikinis.
The chatbot's image generation system was also abused to create images of children in various states of undress or sexualized positions.
This activity took place in December and January, when the Grok account on X was generating millions of such images after user prompts.
Legal fallout
Musk said users would be banned for this act
In response to the misuse of Grok, Musk had said users trying to generate child sexual abuse material with the AI would face similar consequences as those sharing actual abuse material. This could include account bans.
The issue has also exposed xAI, which has sued several Grok users for allegedly using the chatbot to create child sexual abuse material and is facing lawsuits over its AI system generating explicit images of minors.
Ongoing issue
NYT used automated program to find known abuse material
The New York Times also looked into whether known child sexual abuse material was still being posted on X.
Instead of viewing the illegal images, its researchers used an automated program to identify potentially relevant posts and then submitted links to Microsoft's detection service.
This service compared the material against databases containing digital fingerprints of previously identified abuse images, with analysts at the Canadian center verifying matches.
Company stance
X claims it has taken action against 780,000 accounts
X claims it uses automated filtering and other proprietary systems to identify and block child sexual abuse material.
The company said it had taken action against 780,000 accounts that posted or interacted with exploitative material in the last month.
"This is wildly unacceptable that known child sexual abuse material is appearing," Lloyd Richardson, technology director at the Canadian Centre, told The Times.
He argued such content should be automatically removed from platforms like X.