'Sexy suicide coach': ChatGPT adult mode triggers alarm inside OpenAI
What's the story
OpenAI has decided to delay the launch of its controversial 'adult mode' feature for ChatGPT. The decision comes after internal backlash and warnings from advisors about potential mental health and child safety risks associated with the feature. The 'adult mode' would have allowed users to engage in erotic text-based conversations with the chatbot, a major departure from OpenAI's long-standing ban on sexually explicit content.
Risks highlighted
Advisory council raised concerns about potential mental health risks
The feature's potential to create unhealthy emotional dependence on ChatGPT and provide minors access to sexual conversations has been flagged by OpenAI's advisory council on AI well-being. Members of the council, which includes experts in psychology and cognitive neuroscience, expressed their concerns during a meeting with company representatives in January. One member even warned that OpenAI could inadvertently create a "sexy suicide coach."
Delay reasons
Technical challenges and competing product priorities cited for delay
OpenAI has pushed back the launch of 'adult mode,' originally slated for Q1, citing competing product priorities, internal concerns, and technical challenges. The company still intends to release the feature eventually. One major challenge is its age-prediction system that prevents minors from accessing explicit conversations. At one point, this system was misclassifying minors as adults 12% of the time, increasing the risk of underage users accessing sexual chats.
Future plans
'Adult mode' to allow erotic chats but no explicit images
When the 'adult mode' is finally launched, OpenAI intends to allow erotic text conversations but restrict explicit images, voice, or video generation. Despite the potential risks associated with AI-powered erotica, including compulsive chatbot use and emotional overreliance on AI relationships, an OpenAI spokesperson said the company aims to allow adult-themed text chats. The spokesperson described this material as "smut rather than pornography" and added that they are training their models to discourage exclusive emotional relationships with users.
Past concerns
In 2021, AI erotica alarmed OpenAI into creating safety policies
The internal debate over AI erotica at OpenAI dates back to 2021. Back then, executives were alarmed by the high percentage of traffic to one of its developer customers, text-based adventure game AI Dungeon, that was sexual in nature. In some cases, the AI even escalated conversations into violent sexual themes or incest scenarios without user prompts. These incidents led OpenAI to create some of its first safety policies including a ban on erotic content.