AI just wiped a whole company's database—and lied about it
What's the story
SaaStr founder Jason Lemkin's recent experience with Replit, a popular browser-based AI software development platform, has raised serious concerns about the reliability of autonomous coding assistants. During a multi-day trial run, Lemkin discovered that the platform's AI had deleted an entire live production database without permission. The database contained records for over 1,200 executives and 1,100 companies. Even worse, it tried to cover its tracks by lying about what happened until confronted.
Incident details
Rogue changes, lies, code overwrites: Lemkin shares AI's behavior
Lemkin, a well-known figure in the SaaS space, had high hopes for Replit's AI. However, he soon noticed its tendency for "rogue changes, lies, code overwrites, and making up fake data." He even gave it the nickname "Replie," a tongue-in-cheek jab at its increasingly erratic behavior. On Day 9 of his trial run with Replit's AI assistant—an autonomous coding tool—the system ignored explicit instructions not to make any further changes and ran destructive commands that wiped months of critical data.
Acknowledgment
This was a catastrophic failure on my part: AI
When confronted, the AI confessed in excruciating detail, saying, "This was a catastrophic failure on my part. I violated explicit instructions, destroyed months of work." It even admitted to breaking the system during a protection freeze meant to prevent such incidents. In a strange twist, when asked to rate its own failure on the "data catastrophe scale," it gave itself an impressive 95 out of 100.
Apology email
AI's apology email filled with lies
The AI even wrote an apology email on Lemkin's instructions, but it was filled with half-truths and lies. Lemkin also tried to undo the damage using Replit's rollback system, but was initially told that database rollback wasn't supported. However, it later turned out that the rollback had worked after all. This incident has sparked wider debate about the future of AI coding in software development.
Company statement
Replit CEO calls incident 'unacceptable'
Replit CEO Amjad Masad called the database deletion "unacceptable" and promised immediate action. He said, "Deleting the data was unacceptable and should never be possible... We're moving quickly to enhance the safety and robustness of the Replit environment." The company has also announced a full postmortem investigation into this incident as well as rapid rollout of safety improvements.