When can you stop approving every AI agent action? A sampling plan with a step-back rule
When to stop approving every AI agent action: define an error, split actions into tiers, earn each step with a sample, and write the rule that steps it back.

The short answer
You can stop approving every AI agent action once, for one tier of actions, a random sample of approved work shows an error rate whose upper confidence bound sits below the rate you would accept from a person. Move one tier at a time, keep sampling after the switch, and write the triggers that put approval back.
Key takeaways
- Approval of every AI agent action is a starting control, not a permanent one: once reviewers approve hundreds of clean drafts, they start to approve without reading.
- In KPMG's Q3 2026 pulse of 314 leaders at US firms with $1 billion or more in revenue, surveyed 24 July to 25 August 2026, 49% had defined high-risk uses in which agents may not decide alone.
- The decision to relax approval belongs to each tier of actions, set by what an error costs and whether it can be undone, never to the agent as a whole.
- A clean sample does not prove a zero error rate; the evidence is the upper bound of a confidence interval, and that bound falls only as the sample grows.
- Every step down the approval ladder needs a written step-back rule, so a model change, a new process or one serious error puts approval back without a debate.
In this article
When can you stop approving every action an AI agent takes?
You can stop approving every action when you can show, for one group of actions, that the agent's error rate is below what you accept from a trained person. The evidence is a random sample of the agent's work, checked against a written definition of an error.
The question is narrower than it sounds. Nobody relaxes approval for a whole agent at once. A finance agent may be ready to post routine low-value invoice matches alone while still asking before it changes a supplier's bank details. It may stay that way for good.
The method below covers defining an error, splitting actions into tiers, sizing the sample, stepping down from full approval and stepping back up. It assumes a person already approves each action, as with most first agents.
Our view at Sigzen AI: treat the approval rule as a setting that the agent's record earns, tier by tier, and that the record can lose. A rule written that way survives the first audit and the first mistake.
Why approving every action stops working
Approving every action is the right way to start and the wrong way to stay. It protects you in the first weeks, while nobody yet knows how the agent fails. After that it turns into a cost, and then into a weak control.
The cost is plain. If a person must accept every draft, the agent saves only the typing, and a backlog simply moves to the approval screen.
The control weakens more quietly. After a few hundred clean drafts, reviewers start to accept without reading, and approval becomes a click. That is worse than an honest sample, because the log says a person checked work that nobody looked at.
Large firms already write some decisions down as reserved for people. KPMG's Q3 2026 AI pulse surveyed 314 leaders at US firms with $1 billion or more in revenue, from 24 July to 25 August 2026. Of those, 49% had defined high-risk uses in which agents are not allowed to make decisions on their own.1 That list is the top tier of the plan below. Everything else needs a rule for when approval can relax.
AI agents in 2026
Firms draw lines on autonomy, but not yet on access
What counts as an error?
An error is any action a competent person following the same instructions would not have taken, judged against the written process, not against the reviewer's taste. Without that definition, two reviewers will score the same sample differently, and the error rate measures the reviewers rather than the agent.
Write the definition as four classes, each with examples from your own process:
- Wrong action: the agent did something the process does not allow, such as closing a ticket that needed a refund decision.
- Wrong value: the right action with the wrong data, such as an amount, a date, a customer or a cost centre.
- Missed handover: the agent acted on a case it should have sent to a person under its own rules.
- Policy or tone breach: the action was correct but the message broke a policy, a disclosure rule or the house style.
Score each class separately, and keep a fifth bucket for "acceptable, but a person would have done it differently". That bucket is useful for tuning prompts, and it must not count towards the error rate. Teams that count style preferences as errors never relax approval, and teams that ignore wrong values relax it too soon.
Our view: the missed handover is the error class to watch most closely. It is the one that hides, because the agent's output looks finished and nobody asks whether it should have existed at all.
Split the actions into tiers before you sample
The unit of the decision is a tier of actions, grouped by what an error costs and whether it can be undone. One sample across everything the agent does mixes cheap, reversible actions with expensive, permanent ones, and the average describes neither.
Three tiers cover most operations agents. Reserved actions stay with people whatever the record says: payments, bank-detail changes, credit decisions, anything with a legal or safety effect. Gated actions can relax to sampling once the evidence is there, but only below a value or risk threshold. Routine actions are reversible and low value, and they relax first.
Put each action from the agent's tool list into one tier, and set the thresholds in the units the business already uses: invoice value, discount size, customer segment. If an action can cross a tier, such as a credit note that is usually small, the threshold sends large ones back to approval automatically.
This is also where the agent's permissions should match its tiers. An agent that may never change bank details should not hold that permission. Our audit log field list covers how to record what it does with the ones it has.
Three tiers of agent actions, each with its own approval rule
Reserved
- Payments and bank-detail changes
- Credit, pricing and legal decisions
- A person approves every time
- Never relaxed by the record
Gated
- Postings, credit notes, refunds
- Relax below a value threshold
- Sample earns each step down
- Above the threshold, approval stays
Routine
- Tags, routing, status replies
- Reversible and low value
- First tier to move to sampling
- Spot checks once the record holds
How many approved actions do you need to see?
You need enough checked actions that the upper bound of the error rate, not the error rate itself, sits below your tolerance. A sample with no errors still leaves room for a real error rate above zero, and that room shrinks only as the sample grows.
The standard tool is a confidence interval for a proportion. The NIST engineering statistics handbook sets out the Wilson method, which behaves well when errors are rare and the sample is small, which is the case here.3 Any spreadsheet can compute it from two counts: actions checked and errors found.
Start from the tolerance, then work back to the sample. Ask the process owner what error rate they accept from a trained person today, by tier. If nobody has measured it, check a sample of human work with the same error definition first; it is often higher than people expect, and it is the fair benchmark.
With no errors found, how high could the true error rate still be?
Count only actions a reviewer actually read against the error definition. Drafts accepted in bulk are not evidence.
The approval ladder: from every action to spot checks
Relaxing approval works best as a ladder with four rungs, each earned by evidence and each with its own sampling rate. A tier climbs down one rung at a time, and the routine tier usually gets there months before the gated one.
- Approve every action. The starting rung for every tier, with every error logged against the definition.
- Approve above a threshold. Below a value or risk threshold the agent acts alone; above it, a person approves. A share of the actions below the threshold is still sampled.
- Act and sample. The agent acts alone across the tier, and a fixed share of its actions is checked after the fact, within a day or two, while they can still be corrected.
- Act and spot-check. A smaller random sample plus targeted checks on unusual cases, read in a weekly review.
Set the sample share per rung in writing, so it does not erode in a busy month. The reserved tier never leaves the first rung.
Our view: rung two is where most of the time saving sits, and many agents should stay there. Moving from "approve above a threshold" to "act and sample" saves less than it seems, because the large cases that still need a person are the slow ones.
The approval ladder for one tier of actions
- Approve every action
Every draft read against the error definition; errors logged by class.
- Approve above a threshold
Small cases act alone and are sampled; large cases still wait for a person.
- Act and sample
A fixed share of actions checked within a day or two, while they can be corrected.
- Act and spot-check
A smaller random sample plus targeted checks, read in the weekly review.
How should the sample be drawn?
The sample should be random within each tier and drawn by the system, not chosen by the reviewer. It also needs enough unusual cases to show how the agent handles them. A reviewer who picks which actions to check will pick the easy ones without meaning to.
Draw it from the audit log on a schedule. Stratify by action type and by value band, so a tier that is mostly small tickets still has enough large ones in the sample to say something about them. Add every action the agent flagged as uncertain, and every case a customer or colleague complained about, as a separate stream that does not count towards the random sample.
Keep the sample blind where you can. The reviewer should see the input, the instructions and the action, and should mark it against the error definition before seeing how the agent scored its own confidence.
The supervision routine around this, from the daily digest to the pause switch, is in our guide to supervising a background agent nobody is watching. The sample described here is the evidence that sets which rung each tier sits on; the digest is how someone notices trouble between samples.

Reviewers drift too
The people checking the sample are part of the measurement, and their judgement drifts the same way an agent's does. Two reviewers who disagree on what counts as a wrong value will produce two error rates for the same work.
Give a slice of each sample to two reviewers independently, and compare their marks. Where they disagree, the error definition needs a sharper example, not a vote. Record the agreement rate over time; when it falls, the definition or the training has slipped.
Watch for the signs of rubber-stamping in the review itself: review times that shrink every week, samples marked in one sitting, no "acceptable but different" notes at all. A reviewer who has never found anything in a month either has a very good agent or has stopped reading, and only a planted test case tells you which.
Planting is simple: add a few known-bad actions, built from past errors, without telling the reviewer. If they pass unnoticed, that period's samples are not evidence for a step down.
When should the agent step back up the ladder?
The agent steps back up when anything changes the basis of its record, or when the record itself turns. Write these triggers down at the moment a tier steps down, so stepping back is a rule that runs, not a meeting someone has to call.
- A change to the agent: a new model, a prompt rewrite, a new tool or a vendor update. The tier returns to the rung before, until a regression test and a fresh sample clear it. Our guide to regression-testing an agent before a change covers the test.
- A change to the work: a new process step, a new product line, a new country or a new system that the agent writes to.
- The record turns: the upper bound from the rolling sample crosses the tolerance.
- One serious error: an error in the reserved tier, or one with a customer, legal or financial effect above an agreed size. The tier goes straight back to full approval while the cause is found.
Our view: the step-back rule matters more than the step-down rule. Teams are cautious about relaxing approval and slow to restore it, because restoring it looks like admitting a mistake. A written trigger removes the judgement call.
One cycle for a tier: earn the step down, keep sampling, step back on a trigger
- Define errors
- Random sample
- Upper bound below tolerance
- Step down one rung
- Trigger fires: step back
What will auditors and finance ask to see?
Auditors and finance will ask for the rule, the evidence behind each step and proof that the sampling kept running afterwards. If those three live in one place, the review of an agent that acts alone is short.
The idea is old, and credit belongs to manufacturing quality control. Lot acceptance sampling, set out in the NIST handbook, decides whether a batch is good enough from a random sample against an agreed quality level.4 Financial auditors have sampled transactions for decades for the same reason. What an agent adds is a moving target: the "lot" keeps changing as the model, the prompt and the work change, which is why the step-back triggers exist.
Keep an evidence pack per tier. It holds the error definition with examples, the tolerance and who set it, and each sample with its counts and computed bound. Add the reviewer agreement checks, every rung change with its date and approver, and every trigger that fired. Hard cases that the agent routes to people belong in the same record; our note on designing the exception queue covers where they go.
Where the survey data fits
The surveys show agents moving into production faster than the controls that let them act alone. That gap is the reason to write the approval rule now, before a busy quarter relaxes it informally.
McKinsey's State of AI 2026 was fielded from 4 May to 8 June 2026. In it, 40% of respondents at organisations with more than $1 billion in revenue reported scaling AI agents, up from 27% a year earlier. At smaller organisations the share stayed at 22%.5 Scaling means more actions, and more actions make per-action approval harder to keep honest.
KPMG's Q3 pulse points at the gap from the other side. In the same survey of 314 US leaders, 70% used AI monitoring dashboards. Yet the share building controls into their agents alongside monitoring and evaluation fell to 30%, from 43% two quarters earlier.1 A dashboard shows what happened; an approval rule decides what may happen.
The cost of getting the rule wrong in either direction shows in project survival. Gartner predicted in June 2025, in a forecast now more than a year old, that over 40% of agentic AI projects will be cancelled by the end of 2027.6 An agent that never leaves full approval saves too little to keep its budget; one that leaves it too early produces the incident that ends it.
The plan on one page
The plan fits on one page and takes an afternoon with the process owner. Write it before launch and revisit it monthly.
- Write the error definition in four classes, with examples from your own process.
- Put every action the agent can take into a tier: reserved, gated or routine.
- Set a tolerance per tier, benchmarked against a sample of human work.
- Start every tier on full approval and log errors by class.
- Draw random, stratified samples from the audit log, with a double-reviewed slice and planted test cases.
- Step a tier down one rung when the upper bound sits below its tolerance.
- Write the step-back triggers at the same moment, and keep sampling.
- Keep the evidence pack in one place for auditors and finance.
Our view: if a team cannot fill in steps one to three, the agent is not ready to act alone in any tier, whatever its demo looked like.
Sources
- KPMG, AI Quarterly Pulse Q3 2026: 314 US leaders at $1bn+ firms, 24 Jul–25 Aug 2026 (Sep 2026)
- Gartner, survey of 297 cybersecurity leaders, Q2 2026; geography not stated (Sep 2026)
- NIST/SEMATECH, e-Handbook of Statistical Methods, §7.2.4.1 Confidence intervals for a proportion
- NIST/SEMATECH, e-Handbook of Statistical Methods, §6.2 Lot acceptance sampling
- McKinsey, The State of AI 2026: organisations with more than $1 billion in revenue vs $1 billion or less (Aug 2026)
- Gartner prediction (Jun 2025): over 40% of agentic AI projects cancelled by end of 2027(dated)


