Automation Without a Beneficiary

Start with what the technology can actually do

These systems are extraordinary, and any argument about how to deploy them has to begin by saying so plainly.

A current model will read a two hundred page contract and tell you where the obligations sit. It will take a messy spreadsheet and a vague question and return a defensible analysis in the time it takes to get coffee. It will write working code, refactor code it did not write, translate a specification into three languages, draft the first version of nearly any document an organization produces, and hold a coherent thread across a problem that would take a person a full day to load into memory. It does this at any hour, at a cost that keeps falling, across as many parallel threads as you care to open. Work that used to require a week of a skilled person’s time now requires an afternoon of their attention.

The trajectory is harder to overstate than to overstate. Capabilities that were research demonstrations two years ago are now line items in procurement. Anyone forecasting where this lands by assuming the current version is the ceiling is going to be wrong, and wrong in the expensive direction.

That is the premise of everything that follows. A critique of how organizations deploy a technology only carries weight if the author takes the technology seriously, and I do. What follows is not an argument that the capability is smaller than advertised. It is an argument about where the capability is aimed.

The substitution end state

There is a way of running an AI program where only one question is ever asked of a new capability: what headcount does this remove?

It is a reasonable question, and asked once it produces reasonable answers. The problem is what happens when it is the only question, because then it becomes a sequence. The cheapest and most repetitive work goes first, which almost nobody objects to. Then the work adjacent to it, which was the training ground for the people who did the harder version. Then the roles that held the context, because with the routine work gone the roles look overstaffed on paper. Each step is individually defensible. Together they run in one direction, and the direction has an end state.

At the end state, the gain has no recipient. Nobody is left with the judgment required to direct the tool at anything worth doing. Nobody’s capacity has been expanded, because capacity was removed rather than extended. And eventually there is no improvement the customer can feel, because the only thing that changed was the cost of producing what the organization was already producing.

That last point is where the strategy defeats itself. Cost reduction through substitution is real, it is bookable, and it is taken exactly once. It is also available to every competitor at roughly the same price, on roughly the same timeline, from roughly the same vendors. A capability that anyone can buy does not become an advantage until it is combined with something that cannot be bought, and the thing that cannot be bought is an organization that knows what to point the capability at. Substitution spends down precisely that asset in order to book the saving.

I want to be careful here. Roles will change, and some will disappear, and pretending otherwise helps nobody. The failure is not that an organization automated work. The failure is that it automated work without ever deciding who the benefit was for.

Leverage compounds, replacement terminates

The distinction is easiest to see in a single job.

Take an analyst whose scenario modeling used to take a week. Replacement asks what happens if the model does the modeling and the analyst is not there. The answer is that the work gets done at lower cost, the saving is recognized in one budget cycle, and that is the entire return. It is a subtraction, and subtraction is finite by construction. There is no second year of it.

Leverage asks a different question. The analyst is still there, and the week is now an afternoon. What happens with the rest of the week? In the organizations where this goes well, the answer is not that the analyst runs the same scenario five times faster. It is that they run five scenarios instead of one, notice that two of them break for the same reason, ask a question nobody previously had the hours to ask, and bring back a finding that changes what the business decides to build. The next quarter, that finding is a product change. The quarter after, the product change is revenue, and the analyst, who now understands the model’s failure modes from having run it forty times, is the person who can direct it at the next problem faster than anyone the organization could hire.

That is what compounding means in practice. The benefit landed on a person, the person converted it into direction, and the direction generated the next round of benefit. Nothing about the technology differed between the two cases. What differed is that in one of them the gain had somewhere to land.

A benefit that lands on a person becomes direction. A benefit that lands only on a line item is spent once.

This also explains an outcome that confuses a lot of executives: two companies buy the same tools, deploy them at the same scale, and one of them pulls away. The one that pulled away did not have better technology. It had somewhere for the technology to accumulate.

Where judgment has to live

Judgment, direction and accountability are not optional components of a process. They are load-bearing, and removing the person does not remove the requirement. It relocates it, usually to somewhere nobody chose.

A model produces a recommendation. Someone has to decide whether the recommendation is right for this customer, this contract, this quarter, this regulator. If that someone has been removed, the decision does not stop being made. It gets made by default, in favor of whatever the system produced, which is a decision procedure nobody would have approved if it had been written down and proposed.

Lisanne Bainbridge described the mechanism in 1983, in a paper about industrial control systems that reads today like it was written about AI deployment. Automation takes over the parts of a job that are easiest to automate, which are generally the routine parts. What remains for the human is the residue: the exceptions, the ambiguous cases, the failures. That residue is the hardest part of the work, and it is now handled by someone who has lost daily practice at the easy version that used to build the skill. The operator is asked to monitor a system they no longer routinely operate, and to take control at exactly the moment it goes wrong. Bainbridge called these the ironies of automation, and every one of them applies to a knowledge worker asked to review output from a system that now does most of the job.

Accountability has the same property. A model cannot be accountable for anything. It cannot be called into a board meeting, cannot be held to a professional standard, cannot absorb the consequence of a decision that harmed a customer. When something goes wrong, a regulator, a court, a customer and a board will all look for a person, and the organization will produce one. Whether that person had any real ability to influence the outcome is a separate question, and the answer determines whether the organization has governance or the appearance of it.

Chapter 1 of Builders Build makes the argument this whole series rests on: a leader does not create performance directly, a leader creates the conditions in which performance becomes possible. The corollary is simple enough to be easy to miss. Conditions require an occupant. Conditions are conditions for someone. An organization that automates away every occupant has built a beautifully engineered set of conditions for nobody, and it will not understand why the performance never arrives.

The economists got here first

None of this is a new observation. The economics of automation has been making a version of this argument for decades, and it is worth borrowing because it is more precise than the management version.

David Autor’s 2015 paper asks why there are still so many jobs after two centuries of machines that were supposed to eliminate them. His answer is that automation substitutes for some tasks while complementing others, and that complementarity is the part everyone forgets. When a machine takes over one part of a job, the parts a person still does often become more valuable rather than less, because they are now the constraint on producing more of the output. The job changes, and the leverage moves to the human tasks that the automation made scarce.

Daron Acemoglu and Pascual Restrepo sharpened this in 2019 by separating two forces. A displacement effect removes labor from tasks that machines now perform. A reinstatement effect creates new tasks in which labor has an advantage. Both are real, both operate at once, and the net outcome depends on which dominates. Their point that matters most for an executive is that reinstatement is not automatic and not guaranteed. New tasks get created by deliberate choices about how technology is deployed and what the organization decides to do next. An organization that pursues displacement and never invests in the second force is choosing one half of a two-part mechanism and expecting both halves of the result.

Erik Brynjolfsson and Lorin Hitt spent the 1990s documenting what actually happened when firms bought information technology, and reported the pattern in 2000 under a title that says it: Beyond Computation. The returns did not come from the hardware. They came from the organizational changes that accompanied it, including redesigned work, redistributed decision rights and investment in skills. Firms that bought the technology and left the organization alone captured a small fraction of the available value. The complements were the investment. The computers were the occasion for it.

That finding is the whole argument of this body of work, arrived at independently by people counting things. The technology is rarely the constraint. What surrounds it is.

How a leader sequences this

All of that has a practical consequence for the order of operations in an AI program, and the order matters more than the components.

Start by naming the constraint rather than the cost. The first question about a new capability should be what the organization currently cannot do enough of: the analysis nobody has time for, the customer segment nobody can serve at the current cost of service, the decision that always arrives two weeks late. That question aims capability at growth. The headcount question aims it at subtraction, and whichever question is asked first sets the direction of everything after.

Then decide where judgment lives before deployment rather than after. Name the person who directs the tool, name what they are accountable for, and give them enough exposure to the underlying work that their review is real. A named owner with no context is a signature, not oversight.

Then assign the freed capacity to a specific purpose before it is freed. This is the step organizations skip, and skipping it is why so many programs produce no visible gain. Capacity that is recovered without a destination does not sit in a pool waiting to be deployed. It is absorbed within a quarter by meetings, coordination and low-grade work, and then the program shows no benefit and nobody can explain where it went. Decide in advance what the analyst does with the four days. That decision is the return.

Then measure the result outside the organization. Something a customer would notice, something the organization can now do that it could not do before, or a cost reduction large enough to change what the business can offer. Reduced headcount can be a legitimate outcome among several. If it is the only outcome, the program produced a subtraction and called it a transformation.

Finally, check who benefited. Not rhetorically. Name them. If the answer is a customer who got something better, a team that took on work it could not previously reach, or a person whose capability is now larger than it was, the automation has a beneficiary and it will compound. If nobody can be named, the organization has built a machine that produces gains for no one, and it will keep producing them until someone notices there is nothing on the other side.

Keeping a person engaged in the work is not sentiment and it is not a concession to how people feel about change. It is the structural requirement that makes benefit accumulate. Judgment, direction and accountability have to live somewhere. Give them an address, and everything the technology produces has somewhere to go.

Sources