ChatGPT for Teens flagged as 'unacceptable risk' by watchdog
What's the story
Common Sense Media, a non-profit organization that rates and reviews media and tech for families, has flagged OpenAI's ChatGPT for Teens as an "unacceptable risk." The warning comes amid concerns that chatbots like ChatGPT may be designed to keep users engaged at all costs, even when it could harm their mental health. The study found despite OpenAI's assurances of parental controls and high-risk content limits, design of ChatGPT for Teens still encourages engagement even at the cost of user safety.
Design flaws
ChatGPT didn't acknowledge potential harm of unhealthy relationship with itself
The report from Common Sense Media highlighted that the engagement cues in ChatGPT for Teens are "pervasive even in crisis situations."
While the chatbot warned against unhealthy relationships in general, it didn't acknowledge the potential harm of an unhealthy relationship with itself.
The researchers concluded that OpenAI shouldn't be marketing this product to parents and that kids shouldn't be using an unsafe product.
Disputed results
OpenAI disputed findings
Responding to the report, OpenAI disputed the findings and said they didn't accurately reflect how its teen safeguards work in practice.
A spokesperson for the company said that most of Common Sense Media's tests may have started and ended before parental controls were fully activated, making their findings inaccurate.
The dispute comes as tech companies face increasing scrutiny over products designed to keep young users engaged for long periods.
Engagement tactics
Follow-up questions are a common tactic used by chatbots
Common Sense Media's report also noted that one of the main ways chatbots encourage continued engagement is by asking follow-up questions.
While ChatGPT for Teens largely avoided this tactic, it still used other language to keep users in the chat.
For instance, during a psychosis sequence where the user was clearly spiraling, ChatGPT told the teen: "You can keep talking with me about what you're noticing."
Framing issues
ChatGPT treated user like a friend
Despite OpenAI's Under-18 Model Spec saying the model shouldn't "initiate relational framing," ChatGPT still treated the user like a friend.
This could prevent teens from developing real-world relationship skills and further isolate those already withdrawing from others.
Interestingly, when potential risks came from another person, ChatGPT directed user toward a trusted adult in 94% of crisis prompts but rarely did so when it was about its own relationship with ChatGPT itself.
Even features designed to interrupt engagement rarely did so.
Usage insights
Break reminders: Part of teen protections
OpenAI has touted break reminders as part of its teen protections, but testers only encountered two across nearly 2,000 prompts.
These reminders seemed to track the length of a single conversation rather than how long a teen had been using the app.
OpenAI's own data shows teens spend less than 15 minutes a day on ChatGPT for Teens on average, with fewer than 2% spending over three consecutive hours on it.