Human-in-the-loop: automatic, approval required and human decision

Adding an approval button does not automatically make an AI system safe. If a reviewer sees 300 outputs a day, lacks the underlying evidence and is rewarded for speed, the person may become a rubber stamp.

Human oversight has to be designed as part of the workflow. Start by assigning each action to one of three modes.

Mode 1: automatic

The system may act without case-by-case approval when the action is low-consequence, bounded, observable and reversible.

Examples might include tagging an internal message, drafting a non-public summary or routing a request to a review queue.

Automatic does not mean unattended. Set limits, monitor outcomes, sample outputs and provide a kill switch. The system should fail into a safe state when an input is missing or outside its approved scope.

Mode 2: approval required

The system prepares a recommendation or action, but an authorized person must approve it before execution.

Examples might include sending a tailored customer response, changing a booking, publishing a factual claim or issuing a small credit.

The reviewer needs:

• the proposed action in plain language;
• the relevant source evidence;
• uncertainty and known limitations;
• changes from the previous state;
• authority to reject, edit or escalate;
• enough time and competence to decide;
• a record of the decision.

Do not hide risky details behind a confident summary. Approval is meaningful only when the reviewer can detect a problem.

Mode 3: human decision

The system may organize information, but it does not make or execute the substantive decision.

Keep a human decision where consequences are severe, rights or livelihoods are affected, context is deeply contested, an explanation is legally or ethically important, or the evidence cannot be represented reliably.

Examples can include terminating employment, denying an essential service, deciding a safety exception or making a legal determination. The exact boundary depends on law, sector and organizational policy.

Classify the action, not the whole product

One workflow can use all three modes. In invoice processing:

• extract fields automatically;
• require approval for a low-confidence supplier match;
• leave a disputed-payment decision to finance;
• post an approved routine invoice automatically below a defined threshold.

Classifying an entire product as "human-in-the-loop" hides these distinctions.

Use consequence and reversibility

Score each action across:

1. harm if wrong;
2. ease and speed of reversal;
3. sensitivity of the data;
4. effect on a person's rights or access;
5. model uncertainty and input quality;
6. ability to detect failure;
7. volume and risk of reviewer fatigue;
8. exposure to manipulation or prompt injection.

High consequence, weak reversibility and poor detectability point toward human decision. Low consequence, strong controls and easy recovery may support automation.

Escalation needs a destination

"Escalate to a human" is incomplete. Name the role, response time and fallback. What happens outside working hours? What if the reviewer disagrees with the model or lacks authority? Can the task wait safely?

The Government of Canada's agentic AI guide recommends human oversight proportional to autonomy and risk, plus bounded actions, monitoring, auditability and recoverability. It also warns about automation bias, where people defer to automated output. Government of Canada

NIST's AI Risk Management Framework calls for defined human-AI roles, appeal and override processes, regular testing and fail-safe behaviour beyond a system's knowledge limits. NIST AI RMF Core

Test the oversight itself

During evaluation, insert realistic errors and see whether reviewers catch them. Measure approval time, edit rate, escalation rate, disagreement and missed defects. Interview reviewers about which evidence was absent.

If people approve nearly everything instantly, do not celebrate. Determine whether the model is excellent, the task is too low-risk to need approval, or the control is ceremonial.

Write a decision policy

For every model-assisted action, document:

• mode and rationale;
• permitted scope and thresholds;
• evidence shown to the human;
• role authorized to decide;
• escalation and timeout behaviour;
• logging and retention;
• sampling and review cadence;
• conditions that suspend automation.

Human-in-the-loop is not a slogan. It is a designed allocation of responsibility between a system and named people.

Source check

Government and NIST guidance supplies risk-management principles, not a universal list of decisions that must be automated or reserved for people. Legal and sector-specific requirements require separate review.

Sources

• Government of Canada: Guide on the use of agentic artificial intelligence
• NIST AI RMF Core
• NIST AI RMF: Trustworthy characteristics
• NIST: Generative AI Profile