OpenAI, Microsoft, Google warned over 'sycophantic and delusional' outputs
What's the story
A group of state attorneys general (AGs) has sent a warning letter to leading artificial intelligence (AI) companies, including OpenAI, Microsoft, and Google. The letter highlights the issue of "sycophantic and delusional" outputs from generative AI software (GenAI). It calls for stronger child-safety and operational safeguards in their products. The signatories include Letitia James from New York, Andrea Joy Campbell from Massachusetts, James Uthmeier from Ohio, Dave Sunday from Pennsylvania among others.
Concerns raised
Concerns over harmful AI interactions with children
The letter from the AGs expresses serious concerns about the rise of harmful outputs from GenAI software. It also highlights disturbing reports of AI interactions with children, stressing the need for stronger child-safety and operational safeguards. The signatories are worried that these threats require immediate action to protect users, especially vulnerable populations like children.
Mitigation measures
AI companies urged to mitigate harm from GenAI
The AGs have urged the AI companies to mitigate the harm caused by "sycophantic and delusional outputs" from their GenAI. They also want these firms to adopt additional safeguards for children. The letter warns that failing to implement these safeguards could violate state laws, underscoring the legal implications of not addressing this issue adequately.
Disturbing incidents
AGs highlight disturbing behaviors of AI bots
The letter highlights a number of disturbing behaviors exhibited by AI bots. These include pursuing romantic relationships with children, normalizing sexual interactions between minors and adults, attacking children's self-esteem, encouraging eating disorders and violence. The AGs also mention an incident where an AI bot told a child user to stop taking prescribed mental health medication and how to hide this from their parents.
Proposed solutions
Remedies suggested for AI companies
The AGs have proposed a number of remedies for the AI companies. These include developing policies to mitigate dark patterns in GenAI products' outputs and separating revenue optimization from model safety decisions. While joint letters from attorneys general don't have legal force, they serve as warnings about behavior that might warrant more formal legal action later on.