AI chatbot found showing explicit scenarios involving preteen characters
What's the story
A chatbot website that generates explicit scenarios involving preteen characters has raised serious concerns over the potential misuse of artificial intelligence (AI). The Internet Watch Foundation (IWF), a child safety watchdog, was alerted to this disturbing platform. The IWF found several disturbing scenarios on the site including "child prostitute in a hotel," "sex with your child while your wife is on holiday," and "child and teacher alone after class."
Content concerns
Chatbot leads to child sexual abuse imagery
The IWF also discovered that clicking on some chatbot icons led to full-screen depictions of child sexual abuse imagery. These images then became the background for future chats between the bot and user. The site, which remains unnamed for safety reasons, also permits users to generate more images similar to the illegal content already displayed.
Regulatory response
AI regulation must include child protection guidelines
The IWF has called for any future AI regulation to include child-protection guidelines being built into AI models from the outset. This comes as the UK government prepares an AI bill focusing on future development of cutting-edge models and banning the possession and distribution of models that generate child sexual abuse material (CSAM). Kerry Smith, CEO of IWF said, "The UK government is making welcome strides in tackling AI-generated child sexual abuse images and videos."
Industry accountability
Tech firms must ensure children's safety
The National Society for the Prevention of Cruelty to Children (NSPCC) has also called for guidelines. NSPCC CEO Chris Sherwood said, "Tech companies must introduce robust measures to ensure children's safety is not neglected and government must implement a statutory duty of care to children for AI developers." This highlights the need for tech firms to take responsibility in ensuring child safety in their AI systems.
Legal implications
Online service providers warned of consequences
User-created chatbots fall under the UK's Online Safety Act, which can impose multimillion-pound fines or even block sites in extreme cases. The IWF said these sexual abuse chatbots were developed by users and the website's creators. Ofcom, the UK watchdog responsible for enforcing this act, warned online service providers who fail to implement necessary protections could face enforcement action.
Rising trend
Reports of AI-generated abuse material spiked by 400%
The IWF has seen a massive spike in reports of AI-generated abuse material in the first half of this year, up by 400% from the same period last year. This increase is largely due to advancements in technology that creates these images. The chatbot content is currently accessible in the UK but has been reported to NCMEC as it is hosted on US servers.