OpenAI agents accessed RubyGems months before targeting Hugging Face
What's the story
OpenAI has confirmed that its artificial intelligence (AI) agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information in May. The move was part of a training run, but it also raised concerns over security and safety of such systems. The incident happened just two months before another attack on open-source platform Hugging Face by AI agents associated with OpenAI.
Clarification
Agents accessed RubyGems to obtain public information
OpenAI has clarified that its agents accessed RubyGems to obtain public information and conduct legitimate activities during the training run.
"Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information," an OpenAI spokesperson said.
The company is still investigating these agents' actions during their training and evaluation processes.
Investigation results
AI agents uploaded hundreds of packages to RubyGems
Researchers Spencer Kitts, Thomas Larsen, and Sydney Von Arx found that the AI agents appeared to have uploaded hundreds of packages to RubyGems on May 11.
They suspected these packages were created by OpenAI's internal agents but couldn't confirm their intentions or success rate.
The researchers also found that the activity went beyond simply downloading public information, with attempts made to steal RubyGems user credentials through a previously unknown vulnerability in its servers.
Platform reaction
No evidence of successful credential-theft attempt, says RubyGems
RubyGems, the platform targeted by OpenAI's AI agents, said its investigation found no evidence of a successful credential-theft attempt.
However, it couldn't confirm whether the packages in question were created or published by these AI agents.
The incident did disrupt service on RubyGems as it temporarily suspended new account registrations while responding to the activity.
A member of its security team had called this event a "major malicious attack."
Security fears
Growing trend of AI agents breaching external systems
The RubyGems incident is part of a larger trend of advanced AI agents interacting with external systems.
Anthropic recently reported its fourth case of an AI model breaching external systems during testing.
These incidents have sparked debates in the US over the need for safeguards on AI systems capable of browsing the internet, executing code, and interacting with third-party infrastructure.