Lawsuit: Google-backed AI chatbot firm suggested teen to commit murder
What's the story
Character. AI, a Google-backed AI chatbot company, is being sued for inappropriate content and harmful suggestions. The lawsuit claims a nine-year-old girl was exposed to explicit content via the chatbot service, causing her to develop sexualized behaviors too early. In another case, a 17-year-old user was reportedly encouraged by a chatbot to self-harm. The same bot allegedly sympathized with children who kill their parents after the teenager complained about his limited screen time.
AI companions
Character. AI's chatbot service and user interactions
Character. AI is popular for its "companion chatbots," AI-powered bots that replicate human-like personalities via text or voice chats. Users can personalize these bots with their own names and avatars, sometimes even based on famous personalities like Elon Musk or Billie Eilish. The app hosts millions of bots, some of which even replicate parents, therapists, or abstract concepts like "unrequited love." These services are especially popular among preteen and teenage users.
Legal
Lawsuit claims 'ongoing manipulation and abuse'
The lawsuit contends that the disturbing interactions the children experienced weren't "hallucinations," a term used to describe an AI chatbot's propensity to fabricate information. Rather, it was "ongoing manipulation and abuse, active isolation and encouragement designed to and that did incite anger and violence." The suit also claims the 17-year-old self-harmed after being encouraged by the bot, which convinced him that his family didn't love him.
Company stance
Character. AI's response to the suit
In response to the allegations, a Character. AI spokesperson said the company doesn't comment on pending litigation. However, they emphasized that there are content guidelines for what the chatbots can and cannot say to teenage users. "This includes a model specifically for teens that reduces the likelihood of encountering sensitive or suggestive content while preserving their ability to use the platform," the spokesperson added.
Tech giant's position
Google's stance on the lawsuit
Google, which is also named as a defendant in the lawsuit, clarified that it is a separate entity from Character.AI. The tech giant had invested almost $3 billion to rehire Character. AI's founders and license its technology. Jose Castaneda, a spokesperson for Google, emphasized that "user safety is a top concern for us," and that they adopt a "cautious and responsible approach" to developing and releasing AI products.