Human in the Loop

“Human in the loop” is often used as a general assurance that someone remains involved. That is too vague to govern real systems. The meaningful questions are which decision the person owns, what evidence they receive, and whether they can intervene in time.

Map decisions by consequence

Teams should classify actions by reversibility, production impact, data sensitivity, security, financial effect, and external obligation. Low-risk changes can use automated approval. High-consequence actions need named authority.

This produces a risk-based control model instead of requiring the same manual review everywhere.

OpenAI’s internal harness experiment goes further: human pull-request review is optional, with minimal blocking merge gates. This book retains explicit authorization for consequential changes as a risk-policy choice. That choice should depend on the system’s consequences and recovery evidence, not on copying another team’s merge policy.

Authorization also needs enforcement. Anthropic’s containment account explains why model behavior and approval prompts cannot replace environment boundaries. Its auto-mode design treats automated permission screening as one layer alongside sandboxing. A written authority table must become actual identity, filesystem, network, and tool controls.

Design a real decision surface

A reviewer needs the objective, proposed action, alternatives, evidence, failed checks, uncertainty, and rollback path. A large diff without this context creates ceremonial approval.

The system should make the safest valid action easy, including the action to stop.

Prevent automation bias

People can over-trust polished outputs, especially when review is repetitive. Independent evidence, adversarial checks, sampling, and rotation of review work help maintain attention.

The reviewer must understand that approval transfers accountability. It is not a courtesy click.

Learn from interventions

When a human corrects or blocks an agent, the reason should improve future specifications, policies, tests, or context. Otherwise the same failure returns to the same manual gate.

At the broadest level, interventions come in three kinds, and each implies a different repair. Some supply knowledge that could have been written down, such as a reviewer explaining again that an API must keep accepting a legacy date format. Capture it as maintained context and a check, so the next change no longer depends on that reviewer. Some are judgment under uncertainty, such as whether a simplification justifies disrupting a valuable customer’s workflow. Better evidence and clearer alternatives can narrow that choice; what remains is escalated to its owner. Some are authority: whose interests take priority and which risk the company accepts. That stays with a named person, however good the recommendation becomes.

The goal is not to remove humans indiscriminately. It is to improve the system around recurring judgment.

Put authority in a table before the incident

For the fictional Northstar case, use a deliberately narrow authority model:

Approval applies to a particular action and revision. It is not a reusable blessing for whatever the agent does next. If the patch or governing decision changes materially, the relevant approval must be revisited.

Give the reviewer a refusal path

The packet includes failed checks and unresolved assumptions. A reviewer can approve the bounded action, ask for evidence, return it for revision, or stop it. “Approve” should not be the only easy button.

The Northstar packet states that external payment side effects were excluded. A reviewer responsible for a real payment workflow must reject an attempt to treat this sandbox evidence as sufficient release evidence. That rejection preserves the contract instead of obstructing progress.

Design for an unavailable human

Name a backup owner and an escalation window appropriate to the risk. If neither can respond, the default for an unapproved consequential action is to wait safely. The agent may preserve state and prepare evidence, but it cannot convert urgency into authority.

An emergency path should be defined before the emergency, with bounded actions, observable state, and retrospective review. Do not invent an emergency exception in a prompt because the normal owner is asleep.

Control admission so reviewers can exercise judgment. A human nominally assigned to hundreds of patches is not a meaningful control. Use risk tiers, limited work in progress, independent checks, and rotation to preserve attention.

The absence ends, and the return deserves the same design. As a proposal: when the owner comes back, the system presents a short handoff that separates verified results from claims, names each decision waiting on that person, and lists work it deliberately deferred. In the fictional Northstar case, a sandbox regression investigation that ran while Mara was away would arrive as one patch with the checks actually executed, one open question touching the accepted key semantics that only she can decide, and two related signals set aside as duplicates. It would not arrive as every log line the agent produced.

Judge the unattended period by how ready the next working session is, not by how much it produced. A night that ends with more pull requests than reviewers can responsibly assess has moved the wait rather than removed it. The handoff is itself a claim, so the owner checks its evidence before acting on it.

Learn without making refusal disappear

Classify interventions: missing context, incorrect implementation, policy conflict, weak evidence, or changed intent. Improve the corresponding part of the workflow. Do not optimize for fewer refusals as an isolated metric; a system can achieve that by hiding risk.

Run a tabletop exercise in which the agent asks for a production migration while the primary owner is unavailable. Identify who receives the request, what remains paused, and what evidence survives the wait. Repair the authority map wherever the team has to improvise.