OpenAI contractors read ChatGPT chats to improve AI: Report
What's the story
OpenAI has been using hundreds of contract workers to review real ChatGPT conversations, a recent investigation by 404 Media has revealed. The practice, codenamed Project Lily, is aimed at improving the company's AI models. The reviewers sometimes see entire exchanges between users and the AI chatbot, which can include sensitive personal information. This process is separate from publicly disclosed safety checks for harmful content in chats.
Evaluation process
How reviewers are training the AI
The contract workers are tasked with evaluating how well ChatGPT responds to prompts, summarizing user intent, and rating multiple model outputs.
They use a dashboard that first shows a real user prompt. Reviewers then write a brief summary of the user's intent and assess four ChatGPT responses, marking which parts are aligned or misaligned with the instructions for the model being trained.
Training objectives
The model is being trained to sound less human
The reviewers are helping train ChatGPT to sound less human and be less likely to blindly agree with users.
They check whether responses are clear, natural, and appropriately warm while avoiding excessive emojis or "AI-speak."
The model is also expected to match a user's tone without suggesting that it is a human or has emotions.
Privacy concerns
Reviewers can see sensitive information in chats
The report notes that while usernames are not shown to reviewers, the chats can still contain personal or sensitive information.
In some cases, reviewers can also see a "user memories summary" showing what a person has previously used ChatGPT for and where they may live.
OpenAI says chats pass through a Privacy Filter designed to remove personal information before reviewers see them but admits it can sometimes miss sensitive details.
Opt-out option
How to stop your chats from being used for training
OpenAI has a section on its website stating that humans may review some content.
The company also says users can stop their chats from being used to improve its models by turning off the "Improve the model for everyone" setting.
This option is enabled by default for Free, Plus, and Pro users and applies only to new conversations. However, it is disabled by default for Enterprise, Business, and Edu accounts.