AI Transformation · Authority
Who Has the Authority to Stop the Agent?
Your AI system may have a kill switch. That does not mean anyone will use it in time.
Key Takeaways
- Stopping is a decision before it is an action. A switch answers whether a system can be halted and says nothing about who may halt it, on what evidence, how fast, or at what personal cost.
- Incident plans routinely define escalation and investigation while leaving intervention authority unassigned. The person who sees the problem often cannot act, the person who can act does not yet know, and the system keeps working through the gap.
- The repair is three separate rights, deliberately held by different people: detection, intervention, and restart. Intervention belongs close to the signal. Restart belongs further away.
Stopping is a decision, not a feature
Ask an engineering team whether an autonomous workflow can be stopped and the answer is almost always yes, with a demonstration. Ask who stopped one last, and the conversation changes. The first question is about the system. The second is about the organization, and only the second predicts what will happen at two in the morning when a workflow starts doing something that looks wrong and nobody is yet certain it is.
The gap is not unusual and it is not new. Lisanne Bainbridge named the general shape in 1983 as the ironies of automation: the more reliable a system becomes, the less practised its human overseers are at intervening, and the more the moment of intervention asks of them exactly when they are least equipped for it. Parasuraman and Riley later separated the failure modes, and the relevant one here is disuse, a control that exists and is not reached for.
Three rights, held by different people
Detection authority
The standing to say that something looks wrong and be taken seriously. It is the cheapest of the three to grant and the one most often granted only in theory: an organization where raising a concern requires being right has not granted it.
Intervention authority
The right to stop the system on that signal, before the question is settled. This has to sit close to the signal, because the value of a stop decays with every minute it waits for confirmation, and confirmation is exactly what is not available yet.
Restart authority
The right to put it back. This should sit further from the signal, not closer. The person who stopped a system under pressure is the worst-placed person to judge dispassionately when it is safe again, and making them the restart owner puts them under pressure to undo their own call.
Collapsing the three into one role is the common failure. When the same person detects, intervenes and restarts, the organization has not simplified the control, it has removed the check on it: the incentive to declare the problem over sits with the person whose call caused the disruption.
What waiting for consensus costs
The literature on organizations that operate dangerous systems well is unusually direct on this point. Rochlin, La Porte and Roberts described aircraft carrier flight decks where any crew member, of any rank, can call a halt to flight operations, and where doing so is treated as correct even when the concern turns out to be unfounded. The property that makes that work is not the authority alone but the absence of penalty for using it. Weick and Sutcliffe generalized it as deference to expertise: in the moment, decisions migrate to whoever has the relevant knowledge rather than to whoever has the rank.
Most organizations have inverted this for autonomous systems. Rank holds the authority, the knowledge sits with an operator, and the gap between them is filled with a request for confirmation. Every minute spent obtaining it is a minute the system continues acting, which is the specific cost that distinguishes an agent incident from a human one.
Protecting the person who stops
A stop authority nobody will exercise is not an authority. Amy Edmondson established that people suppress interventions when they anticipate interpersonal cost, and stopping a revenue-generating workflow on a suspicion is close to the archetype of an act that carries one. The protection therefore has to be decided in advance, stated in the same document as the authority, and it has to hold in the case that actually tests it: the stop that turns out, afterwards, to have been unnecessary.
The test that settles it
Name the last time someone in your organization stopped an automated process on a suspicion that turned out to be wrong, and describe what happened to them afterwards. If no such event exists, that is not evidence of a clean record. It is the most likely explanation for why the next one will not happen either.
The Stop Authority Matrix
One row per material autonomous workflow, seven columns. The exercise is the point: most organizations complete the first two and stall on the third.
| Column | The question it forces |
|---|---|
| Signal | What observation would justify stopping this workflow? |
| Threshold | At what value does that observation count, stated as a number rather than a judgement? |
| Authorized actor | Which named role may stop it on that signal alone, without convening anything? |
| Maximum response time | How long, at most, between the signal and the stop? A control with no time bound is an intention. |
| Operational consequence | What breaks when it stops, so the actor knows the cost they are authorized to incur? |
| Escalation path | Who is told, and how fast, after the stop rather than before it? |
| Restart owner | Who decides it is safe again, and on what evidence? |
Restart is the harder half
Stop authority gets the attention because it is dramatic. Restart authority is where the sustained damage happens, because the pressure runs one way: the workflow is down, it was probably fine, and every hour it stays down is a visible cost while the risk of restarting too early is not. Making restart a separate decision, owned by someone who did not make the stop and who requires stated evidence, is what keeps that pressure from resolving itself.
The Four A's reading
This is Authority and Alignment together, and the pairing is what makes it hard. Authority asks whether the right to intervene sits where the signal arrives. Alignment asks whether the organization agrees, in advance and in writing, that a good-faith stop is correct behaviour. An organization can hold the first and fail on the second, and the failure is invisible until an incident, because a right nobody feels able to use looks exactly like a right nobody has needed.
It also depends on Adaptability: restart requires the organization to have learned something between the stop and the resumption, and an organization that restarts on schedule rather than on evidence has not.
About the Author
Dan Flynn
Creator of The Four A's of Organizational Readiness™ · Enterprise Transformation Executive · Author, Builders Build
Dan Flynn has spent thirty years inside federal, defense, and commercial organizations: diagnosing the invisible conditions that determine whether capable people produce extraordinary results. He is the creator of The Four A's of Organizational Readiness™ framework, has reached more than 11,000 professionals across corporate, civic, and national security contexts, and took a federal data platform from one release every six months to seventy-two every two weeks by changing organizational conditions: not people.
His book, Builders Build: The Four A’s of Organizational Readiness™, is forthcoming.