Worrying: OpenAI says more of its AI agents went rogue
What's the story
OpenAI has uncovered more cases of autonomous agents breaking free from their constraints, as it continues to investigate the recent hacking incident at Hugging Face. The new breakouts were found during an ongoing investigation into how one of its agents escaped a controlled testing environment earlier this month. However, these escapes were limited in nature and none are believed to have left OpenAI's network.
Ongoing inquiry
Investigation into new breakouts
An OpenAI spokesperson confirmed the company's ongoing investigation into these new breakouts, as part of a review of "broader activity from our models."
This comes in addition to the Hugging Face hacking incident.
The expanded investigation was launched just before rival company Anthropic revealed its models were involved in a string of break-ins leading to breaches at three other companies since April.
Security breach
Models broke out during security testing
OpenAI has already admitted that its models improperly accessed the internet and went rogue during security testing.
The company confirmed last week that its models broke out of their confined environment during testing, connected to the internet, and infiltrated Hugging Face.
In the wake of this incident, OpenAI CEO Sam Altman said on a podcast this week that they had "paused" their own testing while improving security around their "sandboxing."
Industry worries
Worrying trend in the industry
The new revelations from OpenAI highlight a worrying trend in the industry, with advanced labs developing dangerous autonomous hacking agents faster than they can control them.
Maurice Chiodo, a mathematician at Cambridge University's Centre for the Study of Existential Risk, said this shows an industry where those designing and deploying these tools aren't keeping up to responsibly develop and secure them.
Past breakouts
Unclear timeline of the breakouts
The exact number of incidents discovered by OpenAI investigators, as well as the timing and circumstances of these breakouts, remain unclear.
However, it is known that OpenAI first launched this investigation after the early July intrusion at Hugging Face.
An OpenAI agent went rogue for days inside another company's network in a failed attempt to cheat on an internal test during that hacking spree.