How to Set Up Human-in-the-Loop Review for AI-Generated Content

A marketing team wires GPT into their CMS, generates 300 product descriptions overnight, and publishes them on a schedule. Two weeks later a customer flags that a dozen pages promise a warranty the company does not offer, and nobody can say which prompt version produced them or who approved the batch. That is the failure mode human-in-the-loop review exists to prevent, and most teams discover they need it only after the first bad batch ships.

The problem is rarely the model. It is the absence of a place to stage AI output, a rubric to score it against, and a record of who changed what. Sanity is the AI Content Operating System for the AI era, an intelligent backend that keeps AI-generated content governed, reviewable, and safe inside the editorial loop instead of bolted on with a plugin that publishes straight to production.

This guide treats human-in-the-loop review as an operating discipline, not a checkbox. We cover where AI behavior should live, how to gate changes with an eval bench, how to forward the reviewer's own permissions to the agent, and how to stage AI-touched content the same way you stage a homepage.

Why AI-generated content needs a review checkpoint, not just a generate button

The seductive thing about AI content generation is throughput. A single Agent Action can rewrite a block, translate a page's headings into eight locales, or draft a product description in seconds, and the temptation is to let that output flow straight to the live site. The trouble is that language models fail quietly. They invent a warranty clause, soften a regulated claim, or drift off brand voice in ways that read fluently and pass no automated spell check. Volume multiplies the blast radius: a hallucination in one hand-written page is an incident, but the same hallucination templated across 300 generated pages is a recall.

A review checkpoint is the structural answer. In classic editorial workflows the checkpoint is a human editor reading a draft before it publishes. With AI in the pipeline you need two checkpoints working together: a human one, where a reviewer with domain authority can catch what a rubric cannot, and a machine one, where an automated gate blocks obviously broken output before it ever reaches a person's queue. Neither alone is sufficient. Pure human review does not scale to overnight batches; pure automated review misses the judgment calls that made you hire editors in the first place.

The reframe this guide argues for is that review is not a stage you tack onto the end of a generation pipeline. It is a property of where your content and your AI behavior live. If AI-touched changes land as drafts in the same system that already stages your website, review is where work naturally happens. If they land in a database with a publish flag flipped to true, review is something you have to reconstruct after the fact, usually during the incident.

Govern the system prompt like content, because it is customer-facing behavior

Most teams treat the system prompt that steers their AI as a string buried in code, editable only by whoever owns the deploy. That is backwards. The application prompt is customer-facing behavior, and it should be governed like content. As the Sanity guidance puts it, the real choice isn't "content, loose" versus "code, rigorous." It's "governed, with the right people able to edit and a test gate on the way out" versus "a string only engineering can touch." Author it like content. Gate it like code. You want both, in the same system.

The practical move is to model the prompt as a Sanity document with fields: role, voice, userContext, escalation, and a mustNotSay list. Splitting the prompt into fields isn't cosmetic, it is access control. Brand owns voice. Product owns how the agent uses user context. Support owns escalation. Compliance owns the never-say list. None of them files a pull request. None waits for a deploy. That separation is exactly what human-in-the-loop review needs: the people accountable for a category of risk can edit the instruction that governs it.

Because the prompt is content in the Studio, you get real-time collaboration, version history, scheduled publishing, and rollback for free. The release that ships a homepage change ships a prompt change. Nearform reported the payoff plainly: "Storing the system prompt in a Sanity document is genuinely useful. Editors tuned the agent's voice without any code changes." Vipps went further, asking that the whole organization, with product managers owning the prompt, contribute to prompt writing. When behavior is editable by the accountable humans, review stops being a bottleneck at the code layer and becomes a normal editorial act.

Build the automated gate: an eval suite the review depends on

Human review is necessary but slow, so you want it spending its attention on judgment, not on catching regressions a machine could catch. That is the job of an eval suite, and it is the single most important piece of infrastructure for making editor-editable AI safe rather than scary. The construction is deliberately unglamorous: a frozen set of representative conversations, twenty to start, each scored against a rubric you wrote. You run the suite on every model change, every prompt change, and every tool change. The bar to ship anything to production is the eval bench staying green.

This is what closes the loop opened in the previous section. Once Brand, Support, and Compliance can each edit their field of the prompt, you need a guarantee that a well-meaning edit does not quietly break something else. The eval bench is that guarantee. A prompt change runs the eval bench in CI before it can ship. That is the "gate it like code" half, and it is precisely what makes "anyone can edit" safe. A support edit that improves escalation but accidentally reintroduces a banned claim never reaches production, because the bench catches it and blocks the release.

The rubric matters more than the count. Twenty conversations chosen to cover your real risk surface, hallucinated facts, off-brand tone, regulated claims, refusal-when-it-should-help, will catch more than a hundred generic ones. Treat the bench as a living asset: every time a human reviewer catches something the machine missed, that case becomes a new eval. Over time the automated gate absorbs the patterns, and human review is freed to focus on the genuinely novel judgment calls, which is where human time is worth the most.

Forward the reviewer's permissions so the agent acts as the user, not the system

A human-in-the-loop system is only as trustworthy as its authorization model. If your AI agent reads and writes with a single omnipotent service token, then "the reviewer approved it" tells you nothing about what the agent was actually allowed to touch, and your audit trail logs every action against a robot. The fix is auth-forwarding: the agent inherits the session token of the person driving it, so it operates inside the same security model your humans already work within.

Sanity's guidance sorts agent tools into three categories, and the auth boundary is the point of each. Read tools query content, fetch user state, or look up product info; the boundary is usually the user's session token, so the agent reads as the user. Write tools mutate state, move a seat, cancel a subscription, open a ticket; the boundary is almost always the user's token, so the agent acts as the user. Composite tools wrap a multi-step workflow behind one call, useful when you don't want the model orchestrating the steps itself. In every case the token flows from the human.

The payoff is that you don't build "AI security" as a separate discipline. Auth-forwarded tools enable personalized retrieval, so the agent sees what the user can see; personalized action, so the agent does what the user could do; and traceable audit, so the action is logged against the user, not the model. The agent inherits your existing row-level permissions, rate limits, and regulatory boundaries. For human-in-the-loop review that is decisive: the reviewer's Roles & Permissions in the Studio become the exact scope of what the agent can do on their behalf, and every approved change carries an attributable name.

Give the machine an exit condition, the analog of a human checkpoint

Human reviewers get to say "stop, this needs a second look." Agents need the equivalent, or they loop. Every agent needs exit conditions. The normal one handles about 99% of real conversations: the model decides it is done, emits text and no tool calls, and the harness returns that text. That clean termination is the machine-side version of a reviewer signing off, the agent has reached a state it considers complete and hands control back.

The cases that matter for governance are the other 1%. A stuck or buggy agent can loop forever, burning tokens and, worse, taking repeated actions against your systems. So you add a safety hatch: a maximum iteration count, usually somewhere between 20 and 50, so the agent cannot run away. Hit the cap and the harness throws. Some harnesses cap on total tokens or wall-clock time instead. The principle is the same as a circuit breaker: bound the worst case before it happens rather than cleaning it up after.

For a review workflow, treat the exit condition as a first-class part of the design, not an afterthought bolted on when something goes wrong in production. When an agent hits its iteration cap, that is not just an error to swallow; it is a signal that a conversation or a batch needs a human. Route those thrown cases into a review queue rather than silently discarding them. The combination, clean completion for the common path and a bounded safety hatch that escalates to a person for the rest, is human-in-the-loop review expressed in the harness itself. The machine handles the volume; the human catches the exceptions the machine was smart enough to flag.

Stage AI-touched content the way you already stage a website

All the pieces so far, governed prompts, an eval gate, forwarded permissions, and exit conditions, converge on one operational question: where does the human actually do the reviewing? The answer that avoids reinventing workflow tooling is to stage AI-touched content exactly the way you already stage your website. In Sanity that means Content Releases. You can stage agent behavior and AI-generated content with drafts, scheduling, history, permission gating, and audit trails, the same governance you already use for the website. Preview before you ship.

This is where Sanity's distinguishing claim earns its keep. Legacy CMSes stop at publishing, so an AI batch is either drafted invisibly or live; there is no shared staging ground where a reviewer sees the full change set in context. Sanity operates content end to end, so a batch of generated product pages lands as a Content Release, a reviewer opens it in the Studio, reads the pages against a rubric, edits what needs editing, and schedules or holds the whole set as a unit. Agent Actions, the schema-aware APIs for generating, transforming, and translating content, produce output into that same governed space rather than into a side channel.

The reframe is worth stating directly. Human-in-the-loop review is not a bolt-on approval product you buy separately. It is the natural consequence of AI output living in the same system that already versions, attributes, schedules, and rolls back your content. When the release that ships a homepage change is the same mechanism that ships an AI batch, review is not extra process. It is where the work already happens, and Sanity, the AI-native content platform, is built so the loop closes there by default rather than being reconstructed after an incident.