"A human is in the loop" is the most common sentence in AI policies and the least often true. Oversight means a named person who understands the system, sees its outputs in time, has the authority to change or stop them, and actually does so sometimes. If nobody has ever overridden the tool, you do not have oversight; you have a rubber stamp.
What the laws require
- EU AI Act, Article 14 (providers) and Article 26 (deployers): high-risk systems must be designed so that natural persons can oversee them, and deployers must assign oversight to people with the competence, training and authority to do it. Deployers must also monitor operation and tell the provider about problems.
- GDPR Article 22 and Quebec Law 25: people have a right not to be subject to decisions based solely on automated processing that significantly affect them, or at least to obtain human intervention and to contest the decision.
- Colorado SB 26-189 (from 1 January 2027): after an adverse outcome, consumers can request "meaningful human review and reconsideration". The Attorney General's proposed rules define what "meaningful" means; expect it to require a reviewer with authority and information, not a click-through.
- Canada's Directive on Automated Decision-Making: impact level III and IV decisions cannot be made without specific human intervention points.
- NYC Local Law 144: a tool that "substantially assists or replaces" discretionary decisions triggers the law; genuine human judgement at the end changes that analysis.
Designing an oversight point
For every high-risk or consequential entry in your inventory, write down:
- Who reviews: a role, then a name.
- What they see: the output, the confidence, the key inputs, and the option to disagree.
- When: before the decision takes effect (pre-decision review) or after, on request (appeal). Pre-decision is required where the law says "solely automated" is not allowed.
- Power: they can change the outcome and stop the system. Write down who can hit the off switch, and that it has been tested.
- Evidence: the override rate. A reviewer who agrees with the machine 100 percent of the time is a signal that oversight is nominal.
Automation bias
People defer to machines, especially under time pressure. The EU AI Act names this risk explicitly. Counter it with: a small daily sample of decisions reviewed blind (before seeing the AI's answer), rotation of reviewers, and a rule that reviewers must write one line of reasoning for agreement as well as disagreement.
Customer-facing systems
For chatbots and assistants, oversight means an escalation path a customer can trigger ("talk to a person"), a person who reads a sample of transcripts weekly, and a way to correct the knowledge the bot draws on. Air Canada lost Moffatt v. Air Canada not because its chatbot was wrong once, but because nothing in its process would have caught the error or honoured the promise when the customer raised it.
Write it down
Oversight you cannot show did not happen. Keep the reviewer roster, the sampling schedule, and the override log with the system's record file (next lessons cover records and monitoring).