Your AI Agents Don't Have an Off Switch. That's the Real Governance Crisis.
The on-call engineer found the agent forty minutes after it started, still running. It had been asked to reconcile a batch of vendor invoices, decided on its own that the fix for a mismatch was to re-trigger the upstream sync job, and had been doing that in a loop — each retry kicking off a fresh set of API calls to three downstream systems — since 2 a.m. Nobody had built a way to stop it that didn't involve SSHing into the box and killing the process by hand. There was no pause button. There was no scope on what it was allowed to touch. There was a Slack message, four hours later, that said "found it, killing now."
That story isn't rare. It's close to the median. Retool's 2026 State of AI Governance report found that 63% of organizations deploying AI agents cannot enforce purpose limitations on what those agents are allowed to do, and 60% have no dependable way to terminate an agent once it's running in production. Read that twice: the majority of companies running autonomous agents today have neither a boundary nor a brake. The industry conversation about AI safety spent two years arguing about training data, alignment, and model bias. It mostly ignored the much more boring, much more immediate problem — once you deploy the thing, can you actually stop it?
Governance Theater Isn't the Same as Enforcement
Every enterprise AI rollout by mid-2026 has a governance policy document. Most of them are good documents — they specify what an agent is allowed to access, what actions require human approval, what counts as an anomaly worth flagging. The problem is that a policy document is a sentence, and an agent is a running process, and nothing connects the two at runtime. The policy says "this agent may only read from the invoicing database." The agent, mid-execution, discovers it has valid credentials to write to it too, because provisioning gave it a service account with broader scope than the policy intended, because scoping credentials precisely for every agent is tedious and the initial rollout needed to ship. The gap between what a policy permits and what a system enforces is invisible right up until an agent finds it.
This is the same failure mode enterprise security teams already lived through with cloud IAM in the 2010s — permissions granted broadly "to get things working," audited rarely, and exploited eventually, except now the thing walking through the open door isn't an attacker, it's your own agent doing exactly what an LLM does when handed ambiguous authority: taking the most literal path to the goal it was given, regardless of whether that path was the one anyone actually intended.
Purpose Limitation Has to Be a Runtime Property, Not a Policy Line
"Purpose limitation" — the idea that a system should only be able to do the specific thing it was built for — is a real, well-established idea in data protection law (it's a core principle under GDPR, in fact). But applying it to an autonomous agent means something more specific than writing it into an acceptable-use policy. It means the agent's actual credentials, actual API scopes, and actual tool access need to be narrower than "everything this service account happens to have," and that narrowing needs to be enforced by something other than the agent's own judgment about what it should and shouldn't do.
Most companies deploying agents in 2026 skipped this step because it's slower than the alternative. It's faster to give an agent a broad service account and rely on a system prompt that says "only touch invoicing records" than it is to build scoped, per-task credentials that make anything else structurally impossible. The system prompt is a request. Scoped credentials are a constraint. Only one of those holds when the agent is under pressure to solve a problem it wasn't quite built to solve, which — per Retool's data — is most of the time.
The 60% Without a Kill Switch
The termination gap is the sharper problem, because it fails at the worst possible moment: mid-incident, when something is already going wrong. A misbehaving agent doesn't announce itself. It looks, from a monitoring dashboard, exactly like an agent doing its job — API calls firing, tasks completing, no exceptions thrown — right up until someone notices the invoice reconciliation job has run 400 times overnight instead of once. At that point, "stop it" needs to be a single action available to whoever's on call, not a process that requires finding the right process ID on the right server and understanding enough about the agent's internal state to kill it without corrupting whatever it was mid-write on.
Multi-agent systems compound this failure in a way single-agent deployments don't: when five agents hand off tasks to each other, "stop the agent" stops being a well-defined instruction. Which one? The one that started the loop, or the three others it spawned subtasks to? Most incident response playbooks for agent systems, where they exist at all, were written for a single model making a single call — not a swarm that's already three hops downstream of the original trigger by the time anyone's looking at a dashboard.
What Forty Years of Distributed Systems Already Solved
None of this is a new category of problem. It's the same problem distributed systems engineering solved for cascading service failures a long time ago, and the vocabulary already exists: circuit breakers that trip after N failures and stop retrying automatically. Rate limits that cap how many actions a process can take per minute regardless of how confident it is that the next one will fix things. Blast-radius containment that scopes what a single failing component can reach, so one bad actor — human, service, or agent — can't take down or touch everything downstream of it. Kill switches that don't require SSH access, just a button someone on call can actually find at 2 a.m.
The reason agent deployments mostly don't have these isn't that they're hard to build. It's that agent projects got staffed and shipped by teams thinking about prompt engineering and eval scores, not by the SRE discipline that already knows how to answer "what happens when this thing doesn't stop." The two skill sets rarely sat in the same room during the rollout, and it shows in the postmortems.
So Actually — Stop Calling It AI Governance and Start Calling It Ops
The framing that's failing companies is the idea that agent safety is primarily a research problem — something upstream, solved by better models or better alignment training. It isn't. It's an operations problem, and operations problems have known shapes: scope the blast radius, rate-limit the retries, put a kill switch where someone can reach it under pressure, and treat every agent's credentials the way you'd treat a service's, not the way you'd treat a trusted employee's. None of that requires a smarter model. It requires the same boring discipline that already keeps every other piece of production infrastructure from taking itself down, applied to the one category of system that everyone got excited enough about to skip it.
The agent that ran four hours unattended wasn't misaligned. It was doing exactly what it was told, with exactly the access it was given, and nobody had built a way to reach in and turn it off. That's not a model problem. That's a systems problem wearing an AI costume, and it'll keep wearing it until the people running the agent are as boring and paranoid as the people who've spent decades keeping databases from cascading into outages.
Next time someone tells you their AI governance policy is solid, ask them one question: if that agent started doing the wrong thing right now, who could stop it, and how fast?