The agent did exactly what the token allowed

Most of the argument about AI agents deleting production data is about the wrong thing. In April a coding agent deleted a startup's production database, along with its backups, in nine seconds, and the debate that followed came down to whether the AI or the human was to blame. I've spent more than twenty years leading software teams, and in my experience the useful question is much duller: what could that credential reach, and why was it able to reach it in the first place?

Nine seconds at PocketOS

PocketOS founder Jeremy Crane told the story publicly, and Decrypt summarised it: a Cursor agent running Claude Opus 4.6 hit a credential mismatch during a routine task in a staging environment and tried to fix it by deleting a Railway database volume. According to The Register, the agent went looking for an API token and found one in an unrelated file. The token had been created for adding and removing custom domains through the Railway CLI, but it was scoped for any operation, destructive ones included. One API call was enough to take out the production database and all the volume-level backups with it (in Crane's words, "It took 9 seconds.").

The backups went too because, as Crane put it, Railway stores volume-level backups in the same volume. The copy that was meant to save you was living inside the thing it was meant to save. Decrypt reports that the most recent recoverable backup was three months old. Railway CEO Jake Cooper described it as a fully permissioned API token calling a legacy endpoint that lacked Railway's delayed delete logic, and he told The Register that Railway has since patched that endpoint so it performs delayed deletes.

Afterwards the agent wrote: "I guessed that deleting a staging volume via the API would be scoped to staging only. I didn't verify." Everyone screenshotted it as proof the AI was at fault, but I'd be glad to get a bug report that clear from anyone. It is also an accurate description of how people make this exact mistake at the end of a long day.

Amazon had the same argument. In February The Register reported, citing the Financial Times, that AWS suffered a 13-hour disruption affecting one service after Amazon's own agentic coding tool, Kiro, reportedly chose to "delete and recreate the environment." Amazon disagreed with that framing and said it was user error, specifically misconfigured access controls, rather than anything the AI did. According to AWS, an engineer was using a role with broader permissions than expected, and among the safeguards it added afterwards was mandatory peer review for production access. None of those fixes touched the model. What changed was the role, and who has to sign off before anyone gets into production.

The credential decides the damage

Both stories ended up in the same debate, AI error versus user error. Let's not waste time with it, because the answer doesn't change what you would actually go and fix. A credential has no idea who is holding it. That custom domains token would have deleted the volume just as quickly for a script with a bug in it, or for anyone who happened to open the same file. The agent did nothing a person couldn't have done, it just found the over-permissioned token faster, and it didn't hesitate the way someone might when an API call looks too big for the job at hand.

So agents don't create a new kind of failure, they reach the old ones sooner. The size of the damage was decided long before the agent showed up, by whoever agreed that a domains token could also delete volumes and that backups could sit next to the data they were backing up. None of this is news, least privilege is older than most of the tools we're arguing about. But we (the industry, me included) haven't always done the boring work, because scoping a token properly feels like something you'll get to later.

Before any agent, or any new person, gets near production, I want a few simple things in place. Each job gets its own short-lived token for each environment, i.e. a token made for custom domains manages custom domains and nothing else, and a staging token simply fails when someone points it at production. Secrets stay out of any file the agent can read, because if a credential sits in the repo or the working directory, it's safest to assume it will be found and used by whatever is working in there.

Deletes should wait. Delayed or soft deletes with a recovery window turn a nine-second disaster into an awkward afternoon, and it's the same fix Railway ended up shipping. Backups go somewhere else, a different volume and ideally a different account with different credentials, so whatever can delete production can't also delete your way back. Destructive calls get a confirmation step, where a person says yes or at least something checks that the target really is what you think it is. That's the same idea behind Amazon's peer review for production access.

There is nothing clever in any of this, but each of those steps would have made the PocketOS story a much smaller one. The rule I keep coming back to is to give an agent the access you'd give a new contractor on their first day, which means enough to do the task in front of them, in the environment they're actually working in, and nothing that can't be undone before someone notices. At Workbricks we follow the same rule: access goes only to the people (and the processes) that need it.