From Model Identity to Human Accountability

Four different things called traceability

Identity is which model. Provenance is what it was built from and how it was configured. Authorization is the organizational grant that permitted it to act in this context. Accountability is the named person who answers for the consequence.

The first two have improved enormously. Model cards, proposed by Mitchell and colleagues in 2019, document intended use, performance across conditions, and known limitations. Datasheets do the same for training data. Both are genuine advances, and both document an artifact from the perspective of the people who built it. Neither was ever intended to say who in a deploying organization decided to point it at accounts receivable, under what ceiling, or who is answerable when it starts issuing credits it should not.

The gap has a name and it is older than the technology

Dennis Thompson called it the problem of many hands in 1980, writing about public officials. Helen Nissenbaum applied it to computing in 1996: where many actors contribute to an outcome, each contribution is individually defensible, and no one is answerable for the whole. Responsibility does not get evaded, it dissolves.

That is the agentic case without adaptation. The vendor trained, the platform team configured, the business requested, a risk function approved a category rather than an instance, and an autonomous system acted. Ask afterwards who is accountable and every party can produce a document showing they did their part correctly. All of them are right. The chain that would connect them was never anyone's deliverable.

Mark Bovens supplies the other half of the definition, and it is the useful one for a board. Accountability is a relationship, not a record: an actor obliged to explain and justify conduct to a forum, which can pose questions and pass judgement, and which can attach consequences. A log satisfies none of those three conditions. It is an input to accountability that is frequently mistaken for the thing itself.

The Accountability Chain

Nine links. The first five come from engineering systems and are usually solid. The chain breaks at the sixth, and it breaks the same way in most organizations.

The point of the chain is not that each artifact should exist. Most of them already do. The point is that they are held by different functions in different systems with no key that joins them, so the chain is reconstructible in principle and not in practice. Reconstructible in principle is worth nothing during an incident.

How a board should test it

Not by asking whether logging is in place. By naming one action an autonomous system took last month and asking for the whole chain by close of business. What comes back tells you more than any framework attestation: which links exist, which are reconstructible, which are absent, and how many departments had to be involved to answer a question about one event.

Raji and colleagues set out an internal algorithmic auditing framework built on the same premise, that accountability requires an end-to-end trail rather than a set of point checks, and that the trail has to be assembled deliberately because no single team's normal work produces it.

Vendors and external models

The chain does not shorten when the model is somebody else's. It gains a discontinuity. Links one and two move outside the organization, where they are supplied at the vendor's discretion and change on the vendor's schedule, and links six and seven stay firmly inside. A model upgrade the vendor treats as routine can alter behaviour under an authorization that was granted against the previous version, and nothing in a standard commercial arrangement obliges anyone to tell the accountable human that the thing they are accountable for has changed.

That is a contracting question before it is a technical one: notification of material change, the right to pin a version, and a stated position on who answers when a behaviour shifts. Authority Should Expire takes up the reauthorization side of the same problem.

The Four A's reading

This is Authority in its evidentiary form. The other pieces in this series ask where a decision right sits and whether it still holds. This one asks whether the organization could demonstrate the answer afterwards, to someone entitled to ask, about a specific event. An organization can have well-placed authority and no way to show it, and the second failure is only discovered under the conditions where it is most expensive.

It reaches Alignment too, because a chain that spans four functions is only assembled if those functions agree it is one artifact rather than four. Each function documenting its own link correctly is exactly the state that produces the problem of many hands.