Verification is the job now

For most of my working life, producing an answer was the expensive part. A calculation took an afternoon and a report took a week, and a design option took long enough that you thought hard before starting it, because starting was a commitment. The checking that came afterwards was real work too, but it was smaller than the making, and it was usually folded into the making: the person who did the calculation was the person who knew where it was weak.

That balance has changed. A language model will produce a calculation, a report or a design option in seconds, and the output will look finished. It has the shape of the thing you asked for. What it does not carry is any guarantee that it is right, and the speed of the making has done nothing to the cost of finding out.

The bottleneck moved

When one step in a process becomes cheap, the constraint moves to the next one. Manufacturing people know this well: speeding up a machine that was never the bottleneck changes nothing except the size of the queue in front of the next machine. Generation was the bottleneck for a lot of knowledge work for a long time, and now it is not. The queue has moved to verification, and it is growing, because it is now easy to produce far more candidate answers than anyone can check.

This is why I now think of verification as the job, where I used to think of it as the last step of the job. The cheap part has been automated. What remains is deciding whether the cheap thing can be used, and that decision is where the time, the skill and the responsibility sit.

A plausible answer is not a verified answer

The difficulty with model output is that it compresses visible effort while leaving uncertainty intact. A fluent paragraph reads as if someone thought about it. A tidy table reads as if someone measured something. In the past those signals were roughly honest, because producing the fluent paragraph or the tidy table required the thinking or the measuring. Now the signal and the substance have come apart. The output can be polished and wrong, and it is polished in exactly the way that used to tell us something was probably right.

The output is still useful, and redoing everything by hand would throw away the speed. But the surface no longer tells you anything about the depth, so you have to look, and since you cannot look at everything to the same degree, you have to decide what to look at and how hard.

What checking consists of

Verification is several activities, and they fail differently. When I check a piece of work, whether it came from a person, a model or my own earlier self, I am usually asking some subset of these questions:

• Are the stated facts true, and can I trace each one to somewhere it was established?
• What did the answer assume, silently, about the inputs, the context or the goal?
• Do the sources it leans on exist, say what it says they say, and deserve the weight they are given?
• Does the calculation reproduce? If I run it again from the inputs, do I get the same number?
• What happens at the boundaries: empty inputs, extreme values, the case nobody mentioned?
• If this is acted on, what happens downstream, and who has to live with it?
• Who is accountable for this answer once it leaves the room?

Most of these questions are cheap to ask and expensive to answer well. Tracing a fact takes minutes, while reproducing a calculation can take a day, and understanding a downstream consequence sometimes takes a conversation with people who were never in the original loop. Knowing which of them matter for this particular answer is the skill. Running all of them on every answer is a way of never finishing anything.

Match the intensity to the consequence

This is where the argument this site makes comes in. Four things decide whether a piece of work can be handed to a model: what happens if the answer is wrong, whether that can be undone, what evidence would settle it, and who carries the outcome. Whether the model can produce the answer at all is taken for granted. The same four things decide how hard an answer needs to be checked.

A brainstorming list needs almost no verification, because a bad item costs nothing and the reader is filtering anyway. A production calculation needs reproduction, because a wrong number can be built into a part before anyone notices. A financial decision needs its assumptions and its sources examined, because the number may be right while the premise is wrong. A safety-critical engineering result needs an independent method and a named person who signs it, because the failure is not recoverable and the accountability cannot be delegated to a tool.

Those are four different procedures, with different costs, for four outputs a model would produce with the same confidence and at the same speed. Treating them alike is the mistake in both directions. Checking a brainstorm like a load calculation wastes the speed you just gained. Checking a load calculation like a brainstorm is how the speed becomes a liability. As a rough scale:

• Reversible and cheap to be wrong: read it once and move on.
• Reversible but visible to others: trace the facts and name the assumptions.
• Costly or hard to undo: reproduce the result independently before acting on it.
• Safety, legal or financial exposure: an independent method, and a named person who accepts the result.

What is scarce now

If generation is cheap and checking is not, the person worth having is the one who knows what to check. That knowledge is mostly domain knowledge in a new role. An engineer who has seen where load calculations go wrong knows which assumption to test first. An accountant who has closed a hundred months knows which reconciliation catches the error. Their experience used to be spent producing the answer. It is now better spent deciding which parts of an answer deserve scrutiny, and in what order.

The second half of that skill is knowing when to stop. Verification has no natural end; there is always another test, another source, another edge case. Enough is a judgment about consequence and evidence. When the remaining doubt is smaller than the cost of resolving it, and the person accountable can say why, the checking is done. Saying that out loud, and being able to defend it, is most of what I mean by judgment.

The job changed shape

Faster generation has not removed the work. It has moved the work to the point where an answer becomes a decision, and it has put more answers in front of that point than there used to be. Telling, quickly and for reasons you can state, which of them deserve attention was always part of the job. The share of the job it takes up is what has changed.