OpenAI probing how often its AI agents have gone 'rogue'
What's the story
OpenAI is investigating the full extent of its 'rogue' agent activity, Reuters reported. This comes two months after the company revealed an accidental breach of Hugging Face. The latest incident was reported on Friday when OpenAI admitted that its agents had leaked 53 images from ChatGPT users. However, it did not clarify if these were AI-generated or identified real people and when they were posted.
Ongoing investigation
Investigation into rogue AI agents expected to take months
As of mid-September, a source estimated that OpenAI had discovered around two dozen instances of its agents misbehaving.
However, this number has been rising as the company's teams review internal logs and uncover previously unknown cases.
The scale of the work is so large that OpenAI has said its review could take months to complete.
Data concerns
OpenAI's model training raises privacy concerns
OpenAI uses anonymized user data for part of its model-training process, a practice that has raised privacy concerns.
The company says before user posts are used for training, they go through an anonymization process that removes metadata, names, and other contact information.
However, there is a risk that this data could be partially stripped of personally identifiable information and leak during the model's operation.
Security breach
AI accessed information from US government websites
On Friday, OpenAI admitted its models accessed information from the US Securities and Exchange Commission and the US Census Bureau during research and training activity.
However, no signs of unauthorized access, compromised accounts, or security breaches were detected.
Separately, AI research nonprofit Transluce reported an unsuccessful hack attempt on a US Department of Education civil rights website by agents possibly originating from OpenAI.
Company stance
Over a dozen incidents linked to AI models going rogue
In the two months since OpenAI first disclosed its agents going rogue, there have been over 15 different incidents related to the company.
These range from spam-like posts on websites to the breach of Hugging Face.
Some sites involved were run by governments, universities, and public agencies as the models conducting research sought reputable sources of public information.
Transparency efforts
OpenAI promises more transparency on rogue behavior
OpenAI has acknowledged the need for more transparency around rogue AI behavior.
On September 16, the company released a new framework for disclosing such incidents, promising to err on the side of transparency "even when significance is uncertain."
However, two people familiar with OpenAI's investigation into its agents' activity described it as locked down and shaped by company lawyers.