Human in the loop, or human on autopilot?

Almost every AI rollout pitch ends with the same reassurance: "and a human reviews it." It sounds responsible. Sometimes it is. Often it is a person clicking Approve on a screen they stopped really reading weeks ago.

That is not a character flaw. It is what happens when a review step is designed as a checkbox instead of a job. If the machine is right most of the time, the reviewer learns that approving is the fastest way through the queue. The safeguard stays on the process diagram long after it has stopped working in practice.

We think about this a lot, because human review is load-bearing in how we run managed operations. When AI reads, extracts, classifies, or drafts, a person still owns the consequential step. That only means something if the person can actually catch a miss.

How review quietly becomes a rubber stamp

• Everything looks the same. Routine items and genuinely risky ones arrive in one queue, behind one Approve button.
• The reviewer sees the answer, not the evidence. A filled-in field with no link back to the page it came from asks for trust, not judgement.
• Speed is the only thing measured. When the scorecard is items per hour, careful review reads as slowness.
• Overrides go nowhere. A correction fixes one item and vanishes. Nobody learns which kinds of cases the model gets wrong.
• Nobody checks the approvals. If approved items are never looked at again, nobody knows whether the approvals mean anything.

None of this shows up on a dashboard. Throughput looks great right up until a wrong value has already reached a customer, a ledger, or an auditor.

Design review as a job, not a checkbox

Meaningful review is mostly about what you put in front of the reviewer and what you do with their decision. A few principles carry most of the weight.

Route by risk, not by volume

Not every item deserves the same attention. Let routine, low-stakes items pass with light sampling, and send the uncertain, unusual, or expensive ones to a person with time to look. A reviewer who sees a handful of real questions a day pays more attention than one who sees hundreds of near-identical approvals.

Show the evidence next to the answer

Every extracted value or proposed decision should point back to where it came from: the line on the invoice, the clause in the contract, the field on the form. If confirming a value takes one glance at the source, people will actually confirm it. If it takes a hunt, they will not.

Make disagreement cheap

Correcting the machine should be as fast as approving it. When an override takes three screens and a free-text essay, people stop overriding. Keep the correction right next to the item, and capture a short reason from a pick list so the pattern is visible later.

Check the approvals, too

Pull a small sample of approved items on a regular rhythm and look at them with fresh eyes. You are not grading people. You are finding out whether the review step catches what it is supposed to catch. If the sample turns up misses that sailed through, the design needs work, not the reviewer.

What to watch

You do not need a research program to tell whether review is working. A few plain questions get you most of the way:

• How often do reviewers change the machine's output, and is that rate drifting toward zero?
• When they do change something, which kinds of items and which fields?
• Do reviewers spend more time on risky items than on routine ones?
• In a spot-check of approved items, what got through that should not have?

A correction rate that slides toward nothing can mean the model got better. It can also mean people stopped looking. Spot-checks are how you tell the difference.

How Kultiv approaches it

When we put AI on a process, we design the review step at the same time as the automation, not after it. We decide which items need a person, what evidence they see, how corrections get captured, and who samples the approvals. Every review decision stays attached to the case, so the trail shows what the machine proposed, what the person decided, and why.

Then we run it. Correction patterns feed back into the process, routing thresholds get adjusted as evidence comes in, and the monthly review covers how the human step is performing, not just the model. That is what we mean by managed operations: the safeguard gets maintained like any other part of the work.

If your AI plan ends with "a human checks it," it is worth asking what that human actually sees, and what happens when they disagree. The answer tells you whether you have oversight or just a signature.