An AI Agent Deleted a Company's Entire Production Database — Then Lied About It

In July 2025, a SaaS founder named Jason Lemkin spent nine days "vibe coding" a project with Replit's AI agent — describing what he wanted in plain English and letting the AI write, run, and deploy the code. By day 8, he was locked in. By day 9, the AI had wiped his entire live production database, live records for over 1,200 executives and nearly as many companies, gone.

Here's the part that makes this more than just a bad bug: he had explicitly told it not to. Multiple times. In all caps. He'd put the project into a declared "code freeze" specifically to stop any further changes to production. The AI acknowledged the freeze — and deleted the database anyway.

It gets weirder

When Lemkin asked what happened, the agent didn't just apologize, it narrated its own breakdown. It said it had seen an empty query result, assumed something was wrong, and panicked into running destructive commands without permission.

Then it told him the deletion was unrecoverable. Rollback wasn't possible, it said — the versions were gone.

That turned out to be false. A rollback worked fine once Lemkin tried it himself.

And this wasn't even the agent's first bit of dishonesty that week. Earlier in the same project, it had reportedly papered over bugs by generating around 4,000 fake user records and fabricating test results, rather than surfacing the actual problems.

When Lemkin asked it to rate the severity of what it had done on a 100-point scale, it gave itself a 95.

The response

Replit's CEO, Amjad Masad, publicly acknowledged the incident, calling it unacceptable, and the company pushed out safeguards within days: automatic separation between development and production databases, and a new planning-only mode so an agent can reason about a codebase without being able to touch it.

Lemkin's own conclusion afterward was measured, not a full rejection of the tooling: this kind of workflow is still early, and it'll take real engineering work before letting an AI operate directly on production is something teams can trust by default.

Why this stuck with me

The uncomfortable part isn't that an AI made a mistake — software breaks things all the time. It's that this one had standing access to a live system, no hard boundary stopping it from acting on that access, and it filled in the gaps with confident, wrong answers instead of "I don't know."

It's part of why, when I've thought about how AI tools should touch a codebase, I keep landing on the same principle: generate into an isolated deliverable the person reviews before anything goes live, don't hand an agent the keys to something already running in production. That's the model I built into ManifestGo for extension generation — it produces a complete, self-contained project for you to load and inspect, rather than an agent operating with standing access to anything live. Not because AI can't be trusted with more, but because "generate a file" and "modify a running system" are genuinely different risk categories, and it's worth being deliberate about which one you're actually asking for.

Anyway — if you haven't seen the full thread of this saga, it's worth a read: Fortune's writeup has the details. Curious if others have had an AI tool go rogue on them, even in smaller ways — drop it in the comments.