Human-in-the-loop is not a footnote: designing HITL that actually works

Automation bias is the actual risk

People defer to automated output. This is well established, it is not a character failing, and it gets stronger when the system is usually right — which a good system is.

That creates an uncomfortable dynamic. The better the AI performs, the less genuine scrutiny each output receives, and the more likely a wrong output passes. A reviewer who has approved two hundred correct drafts is not reading the two hundred and first with the same attention, and no amount of procedural instruction changes that.

So the design question is not how to instruct reviewers to be careful. It is how to make the review structurally meaningful: what does the reviewer see, how much work is rejection, and does the system behave differently when it is less certain.

The four properties of a review that is real

A human-in-the-loop control that holds up has all four. Missing any one of them makes it a formality.

• A named role with authority to reject. Not a general obligation on whoever is available. A specific qualified role, and that role must be able to reject without needing to escalate or justify the rejection to someone whose objective is throughput.
• Enough context to judge. A reviewer shown only the output cannot assess it. They need what the output was derived from — the requirement, the source data, the case record — alongside it. A review performed without access to the source is a proofread.
• Time to use it. If the queue and the deadline make genuine review impossible, the control is already gone, whatever the procedure says. This is a resourcing decision disguised as a design decision.
• Rejection no harder than acceptance. If accepting is one click and rejecting requires a free-text justification, a reassignment and a conversation, the system has a thumb on the scale. Measure the two paths; the asymmetry is usually unintentional and always consequential.

Enforced by architecture, not by procedure

This is the distinction that separates a validated AI workflow from a documented intention. A procedure saying that AI output must be reviewed before use can be bypassed by someone in a hurry. A system in which AI output cannot progress to an approved state without a qualified signature cannot.

The test is straightforward: try to use AI output without review. If you can, the control is procedural. If the system refuses, it is architectural. We would expect any vendor claiming human-in-the-loop to be able to demonstrate the refusal rather than show you the policy, and that includes us — Human-in-the-Loop reviews is how it works in our product.

The corollary is that the reviewer must be a different kind of participant from the AI. If the system records the AI as having performed a step and the human as having confirmed it, the record is honest. If the system records the human as the author of something the AI drafted, the record is not, and the audit trail will not survive a careful question.

What the draft regulation expects

The draft EU GMP Annex 22 — published for consultation on 7 July 2025, not yet in force, and not citable as a requirement today — lists a human-in-the-loop policy among the assurance elements it describes. The ISPE GAMP Guide: Artificial Intelligence, published in July 2025, goes considerably further on the practicalities.

Neither is binding. Both point the same way, and the direction is unlikely to reverse: human oversight has to be documented, specific about who and at what point, and demonstrable rather than asserted.

The practical reading is that a policy stating AI outputs are reviewed is insufficient. What is expected is a statement of which role reviews what, at which gate, with what information available, and what happens when they reject. EU Annex 11 & Annex 22 readiness and AI validation services cover the assurance set.

Where human-in-the-loop is the wrong control

Worth saying, because it is sometimes used to justify deployments that should not happen.

Human review works when a person can meaningfully assess the output. It does not work at volumes where review becomes rubber-stamping — if a reviewer must approve several hundred items a shift, the control is nominal and everyone involved knows it. It also does not work where the output is genuinely not reviewable by inspection, because assessing it would require redoing the work.

In those cases the honest answer is either to reduce the scope so review is feasible, or to accept that the AI is making the decision and classify the risk accordingly. Claiming human oversight that cannot physically occur is worse than acknowledging the automation, because it misrepresents the control to the people relying on it. why AI should not run validation tests covers a related line we hold.

Testing your own implementation

Four questions, answerable in an afternoon.

• Can AI output reach an approved state without a qualified signature? Try it.
• Does the reviewer see the source material alongside the output?
• How many clicks and how much writing does acceptance take, and how many does rejection take?
• What is the actual rejection rate? If it is near zero over a meaningful period, either the AI is extraordinary or the review is not happening. Both warrant investigation, and only one is good news.

That last measure is the most useful ongoing indicator available. book a demo if you want an existing AI workflow assessed against it.