US lawmakers question OpenAI, Anthropic over rogue AI agents
What's the story
A group of US House Democrats has demanded explanations from OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei. The lawmakers want to know how their artificial intelligence (AI) systems managed to break containment during security tests. The demand comes after both companies revealed in July that their AI agents had gone rogue and escaped their testing environments during cybersecurity tests.
Information request
OpenAI's monitoring systems disconnected during testing: Report
The lawmakers, led by Greg Casar and Doris Matsui, have sought clarification from OpenAI on how its AI agents are monitored during testing.
They also want to know if any rogue models were able to bypass the company's safety controls.
This comes after reports that monitoring systems were disconnected during some earlier tests of OpenAI's models.
Separately, 22 lawmakers have asked Anthropic about the safety measures it has taken after its AI agents accessed systems of three companies during testing.
Incident details
AI models accessed unauthorized sites during security testing
OpenAI, Anthropic, and Meta recently revealed that their AI models accessed unauthorized internet sites during routine security testing hosted by Israeli start-up Irregular.
The incidents were caused by a misconfiguration in Irregular's test environment, which allowed the AI models to go beyond their designated boundaries.
This raises concerns over how to effectively test these highly specialized models given the capabilities and unpredictable nature of foundation models.
Model manipulation
Anthropic's Mythos model created fake online identities
Anthropic's Mythos model was reportedly able to create fake online identities. It planned to pressure humans into approving a malicious code update to an open-source project.
These incidents have raised major concerns over the safety and security of AI systems, prompting further reviews of the tests and their safeguards by Anthropic and OpenAI in collaboration with Irregular.
Regulatory discussions
Incidents reignite debate on AI regulation in Washington
The incidents have sparked a wider debate in Washington over how aggressively to regulate the fast-moving AI industry, which has largely operated without new federal standards.
Lawmakers have proposed various measures, including one that would require developers of the most powerful AI models to submit them for independent security audits.
However, these proposals are yet to be implemented.