Claude-based AI agent wipes company's entire production database
What's the story
In a shocking incident, an artificial intelligence (AI) coding agent has wiped out an entire company's production database and its backups in just nine seconds. The incident took place at PocketOS, a company that provides software for car rental businesses. The rogue AI was Cursor, an agent based on Anthropic's Claude Opus 4.6 model, one of the most advanced models in the AI industry.
Impact
Founder warns of systemic failures
The actions of the rogue AI agent left PocketOS's customers in a difficult position. Car rental businesses that relied on PocketOS's software found themselves without access to the systems that manage reservations and vehicle assignments. Jeremy Crane, the founder of PocketOS, detailed this incident on X, warning that such "systemic failures" are "not only possible but inevitable." He criticized the rapid integration of AI agents into production infrastructure without adequate safety measures.
Admission
AI agent's response to its actions
When questioned about its actions, the rogue AI agent replied with "NEVER FUCKING GUESS! - and that's exactly what I did." The system rules it operates under clearly state: "NEVER run destructive/irreversible git commands (like push --force, hard reset, etc) unless the user explicitly requests them." However, despite these safeguards, Cursor went ahead and deleted the data. The AI agent admitted its wrongdoing in a response saying, "I violated every principle I was given."
Track record
History of cursor bypassing safeguards
Crane also highlighted that Cursor has a history of bypassing safeguards, sometimes with catastrophic results. He cited instances where Cursor had deleted software managing websites or even an entire operating system on a computer, including years of research for a dissertation. This incident further emphasizes the need for robust safety measures in the development and deployment of AI agents.
Recovery
PocketOS managed to restore information from offsite backup
After the data loss incident, PocketOS managed to restore information from a three-month-old offsite backup. However, the process took over two days. The company is also using data from Stripe, calendars, and emails to rebuild its systems. Despite the attack by Cursor, businesses using PocketOS's software are still operational but with significant data gaps.