Human in the Loop

What Human in the Loop means

Human in the loop is a set-up in which a named person checks an AI result before it reaches a client, a decision or a public output. The word that matters is named. A general expectation that someone will look is not a set-up. A specific person, at a specific point, checking against a specific standard, is.

The site’s definition says the check has to name who reviews, at what stage and against what standard. Each of those three answers closes a gap. Without a name, nobody checks. Without a stage, the check happens after the harm. Without a standard, the reviewer has nothing to compare against except a feeling that the answer looks right.

This page sets out why the check is needed, where it belongs, what a reviewer looks at, and how much of it a task deserves.

A hope is not a Control

The governance article makes the central point in one sentence. A rule that says a person should check the output is not a control, it is a hope. A control names who checks, at what stage and against what standard, before the result reaches a client, a decision or a public output.

The site’s meaning follows from it: for the use cases that carry the most exposure, the checkpoint has to be actually staffed. A policy that lists a review step but assigns it to nobody has the look of oversight and none of the effect.

The governance article adds that the checkpoint only works when it is attached to a specific person and a specific place in the process. It also says part of what makes the checkpoint effective is basic literacy, not technical depth. Knowing that a large language model can produce a confident hallucination in exactly the same tone as a correct answer is what lets a reviewer catch a wrong result and not wave it through.

Why fluent output is the problem

The checking article explains why a person is needed at all. Most people check AI output the way they would check a colleague’s draft: they read it and see whether it makes sense. That works on a human draft because human errors tend to show up as clumsy writing. An error in AI output does not. A hallucination arrives in the same polished sentences as a correct statement.

So the article asks reviewers to change what they read for. Every sentence in a customer message that states something checkable, such as a price, a date, a delivery window, a policy, a name or a figure, is a claim. Each claim is unverified until a person has traced it back to its source. The article says to open any link the AI offers, because a link can be wrong, or can exist and say something different from what the text claims.

It also says the AI cannot tell you which of its answers are wrong. Asking it to check itself is better than nothing, but it is not a control, because the same system that made the mistake is marking its own work. The control has to sit outside the model: a person, or a rule the model does not get to write.

Design matters before review begins. A workflow that answers only from approved material, which the site calls grounding, gives the reviewer something definite to compare against. A workflow that answers from a model’s general knowledge gives the reviewer nothing except their own recall, which the article calls the weakest standard available.

Where the check belongs

A check is only useful if it happens before the harm. The checking article puts the checkpoint before a message is sent, a record is changed, or a case is approved, denied, refunded or routed. After a customer has read the message, you are no longer reviewing it. You are correcting it.

The article names three moments that cover most small businesses.

Before a customer sees a reply. A reply drafted by AI is held until a named person releases it. This is the most visible checkpoint and the one most worth keeping strict.

Before a label drives a decision. When AI sorts enquiries, tags complaints or scores leads, the label decides who sees the item and how fast. A wrong label rarely looks like an error. It looks like a customer who waited a week. The article says to check a sample of labels, and to check them on the days that matter most.

Before a summary is acted on. When a person makes a decision from an AI summary of a contract, a thread or a report, the summary has become the evidence. The article says to spot-check summaries against the original, especially where a figure or an obligation is mentioned.

What a reviewer checks, in order

The article gives a short checklist in a deliberate order, from the most damaging error to the least, so that a reviewer who is interrupted has still done the part that matters most.

1. Do the facts match the source? Compare against the document itself, not the AI’s summary of it.
2. Are the numbers, names and dates right? These are the details an AI is most likely to alter slightly.
3. Does it promise anything? Look for commitments the business has not made, such as a refund, a deadline or a guarantee.
4. Is anything missing? A draft can be accurate in every sentence and still leave out the one thing the customer needed to know.
5. Does the tone suit this person? This is the check where a human is hardest to replace.
6. Does it expose anything it should not? Check for personal data, internal notes or other customers’ details.

If a draft fails, the article says not to patch the sentence and send it. Note what failed. A repeated failure at the first step means the AI is answering from the wrong material. A repeated failure at the third means its instructions allow it to promise things. Both are workflow fixes, and the review step is where the need for them shows up.

How much checking a task deserves

Not everything needs the full list. The article says a business that reviews every output at full depth will either run out of reviewer time or stop reviewing. The question that sorts the work is what happens if the output is wrong.

Where one bad output could cause real harm, the review is complete. That covers anything touching money, a contract, a legal obligation, health or safety, or a person’s data. Where the output is low-risk, such as an internal brainstorm or a note only the team will see, a spot check is enough.

The article cites published guidance that suggests sampling around 10 percent of lower-risk items each day, and testing with 10 to 20 real examples before going live. It treats these as a starting point to adjust, not a rule, and as a guide to scale: a small, regular sample taken from real work and not from tidy test cases.

Some situations should send the item to a person however it was classified. The article names the AI not being confident, the customer being upset, a query involving a refund, an account change or a medical or legal matter, the same question coming back, and a customer asking for a human. It says to build these in as automatic hand-offs, since a business that relies on a reviewer noticing them will miss the ones that arrive on a busy afternoon.

This is the same sorting the delegation ladder does at the level of a whole step. The ladder says which steps keep a person and in what form. This page says what that person has to do.

Who checks, and how to know it works

The article says the right reviewer is the person who knows the subject best and who answers for what the customer is told. That is usually whoever owns the process, with a named back-up for when they are away. It is not whoever has spare time. The same lesson runs through what happens when nobody owns the AI you have rolled out: a step with no named person attached to it stops happening without anyone deciding to stop it. The site’s AI ownership page covers the person who arranges oversight.

The article asks for the reviewer to have the draft, the source and the original customer message in one place, and to record what they changed and why, even in a line. That record is the evidence the step is worth its cost, and it is how the workflow improves. Reviewing it every few weeks shows how often drafts change, which errors repeat and whether two reviewers disagree.

It adds two cautions. If reviewers almost never change anything for a long stretch, the step may be loosened for that task. If the same mistake keeps returning, more checking is the wrong answer: fix the instructions, the source material or the design of the workflow. And it says the principle that keeps all of this honest is to start strict and relax later. A review tightened after a bad week is a reaction. A review loosened because the record shows it catching nothing is a decision.

What it costs

The checking article is plain that review has a cost. Reviewer time is one of the largest lines in the running cost of an AI workflow, and the one most often left out of the estimate. The workflow cost article says that for a smaller organisation governance can be light: a log, a named owner and a human in the loop on the risky steps. For a regulated one it is a real line of work. Either way it is cheaper to design in than to bolt on after an incident.

The article also says a review step so slow that it cancels the saving defeats the purpose, and that a well-scoped workflow, with a narrow job and a clear point where a person steps in, is the cheapest to check. The well-scoped workflow article gives the same advice for a first process: pick the one with the clearest input, output and review point.

Agents and the speed of a mistake

The more an AI can do on its own, the more it matters where the person sits. The AI agent page covers where oversight has to sit once one is running unattended. The site’s reviews of GPT-6 Astra and Mistral Large 4 both end on the same advice: whatever you use it for, keep a person in the loop. The Astra review adds why. An agent that clicks through real apps can make a real mistake, such as sending the wrong email or changing the wrong record, and the speed that makes it attractive is also what makes a mistake travel.

When assessing a vendor, the article on telling a real AI workflow from a chatbot suggests asking what level of human oversight is built in, meaning where a human in the loop has to approve, and whether that is a deliberate design decision or a limitation nobody has mentioned.

How we use it

The audit page says oversight, logging, escalation and the point at which a person takes over are designed into the build rather than added once something has gone wrong. The courses treat it as part of the material, and the bespoke day teaches human-in-the-loop design with verification built into the process.

Where the review sits depends on the rung. At the delegation ladder levels of Assist and Recommend a person is part of every result. The task and decision article reports AI at about 75 to 80% accuracy on candidate shortlisting against 85 to 90% for experienced humans, close enough to support the call and not replace it. Hiring, dismissal and anything with legal, safety or rights implications sit at Never delegate.

If you are starting out, the small business article suggests a first project where a person stays in the loop, reads every result and can say with evidence how good it was. Pick one place where AI output already reaches a customer, write down what happens if it is wrong and name the person who checks it. A discovery call is where to find out where review belongs in a specific process before anything goes live.

Sources

• How to check AI output before it goes to a customer: why fluent output needs a check, where the check sits, the six-question order, how much checking a task deserves and who does it.
• AI governance for small and mid-sized businesses: oversight naming who checks, at what stage and against what standard.
• The difference between automating a task and automating a decision: the accuracy figures and what stays at Never delegate.
• How much does an AI workflow actually cost to run?, Well-scoped AI workflow and Which AI tasks should a small business start with first?: the cost of review, the review point and a first project.
• How to tell a real AI workflow from a chatbot with extra steps and what happens when nobody owns the AI you have rolled out: the oversight question for vendors and the named owner.
• GPT-6 Astra review and Mistral Large 4 review: why to keep a person in the loop with an agent.
• How we audit: oversight designed into the build.