How to Run Content Quality Checks With AI at Publish Time

A marketing team ships a landing page at 4:58pm on a Friday. The AI-generated hero copy reads well, but it cites a product tier that was renamed last quarter, links to a deprecated docs page, and quietly drops the required legal disclaimer on a regulated claim. Nobody catches it until a customer does. This is the failure mode of publishing at machine speed without machine-speed quality control: the volume of content going out now exceeds what any human reviewer can eyeball, and the errors that slip through are exactly the ones that erode trust and invite compliance risk.

Sanity is the AI-native content platform built to close that gap. As the Content Operating System for the AI era, it treats quality checks not as a manual gate bolted on after the fact, but as governed, schema-aware logic that runs the moment content moves toward publish. The point is not to slow editors down. It is to catch the broken link, the stale fact, the missing disclaimer, and the off-brand tone before the content reaches a reader.

This guide walks through what a publish-time quality system actually needs to do, how to wire AI checks into the editorial loop without turning them into a black box, and where Sanity's Functions, Agent Actions, and AI Assist fit as the enforcement layer.

Why publish-time is the right checkpoint for AI quality checks

Most teams try to solve content quality in one of two places, and both leak. Pre-generation prompts and style guides shape what the model produces, but they are advisory: a model can be told to include a disclaimer and still omit it. Post-publish audits catch problems, but only after the content is live and the damage, a broken link in a customer email or a wrong price on a product page, has already been done. The gap between those two approaches is publish time, the one moment where you know the content is finished, structured, and about to become real.

Publish time is the right checkpoint for a specific reason: it is the only point in the lifecycle where the content is both complete and still stoppable. At draft time you cannot check completeness because the author is still working. Once published, you can only remediate. A check that runs on the publish event evaluates the final artifact against your rules and can block, flag, or auto-correct before anything reaches a reader or a downstream LLM consumer.

This is where Sanity's model matters. In Sanity, publishing is an event on the Content Lake, and Functions are serverless hooks that fire on that event. A moderate-on-publish or fact-check-on-publish Function runs your quality logic exactly when the content transitions state, with the full structured document available. Because Content Releases let you stage, review, and schedule sets of changes together, the check can run against a coherent bundle rather than a single stray field. The result is a quality gate that lives inside the editorial workflow instead of in a separate scanning tool that reviewers forget to run. That reframing, from advisory prompt or after-the-fact audit to an enforced event on the content itself, is what makes AI quality checks operational rather than aspirational.

Structure is what makes an AI check reliable instead of a guess

An AI quality check is only as good as the content it can reason about. If your content is a wall of HTML or an opaque rich-text blob, the model has to parse presentation before it can evaluate substance, and it will hallucinate structure that is not there. The checks you actually want to run are structural questions: does this page have a disclaimer block, is every internal link resolving to a live document, does the pricing field match the canonical product record, are all eight required locales present. Those are answerable with certainty only when the content is modeled as data.

This is the difference between a check that returns a confident wrong answer and one you can trust. Consider a fact-check on a product claim. Against unstructured text, the model guesses which sentence is the claim and what to compare it against. Against a structured document where the claim is a typed field with a reference to the source-of-truth product entity, the check becomes deterministic: compare field to reference, flag on mismatch. The AI is reserved for the genuinely fuzzy parts, like tone and readability, where judgment is required.

Sanity's answer here is Portable Text and a typed schema. Portable Text represents rich text as structured blocks, marks, and annotations rather than markup, so a check can walk the content tree and find exactly the disclaimer annotation or the external-link mark it needs to validate. Agent Actions are schema-aware, meaning an LLM workflow reads and writes against your actual content model, not a flattened string. When you model your business in the schema first, every quality check inherits that structure for free. A rigid CMS forces you to scale reviewers to keep up with volume; a structured foundation lets you scale the output of checks instead, because the logic is written once against the model and runs on every document that matches it.

The five checks worth automating at publish time

Not every quality concern deserves a publish-time gate, and treating them all the same is how teams end up with slow, noisy pipelines that editors learn to ignore. In practice, five categories carry most of the risk and most of the value.

First, factual consistency: does the content contradict a source of truth, a product spec, a price, a feature availability, a person's title. This is the highest-stakes check because AI generation is where invented facts enter. Second, link and reference integrity: every internal reference resolves to a published document, every external link returns a live page, no orphaned or deprecated targets. Third, compliance and required elements: the disclaimer is present on regulated claims, the legal footer exists, accessibility attributes like alt text are filled. Fourth, brand and tone: the copy matches the voice guide, avoids banned phrases, and reads at the target reading level. Fifth, completeness and localization: all required fields and all required locales are populated before anything ships.

The design principle is to match the tool to the check. Deterministic checks, links, required fields, locale coverage, should be plain code with no model in the loop, because you want them fast and certain. Fuzzy checks, tone, readability, whether a summary faithfully represents the body, are where an LLM earns its place. In Sanity you can express both in a single publish Function: run the deterministic validations first, short-circuit on failure, and only invoke an Agent Action or an AI Assist prompt for the judgment calls. AI Assist can fact-check a block against a Knowledge Base, translate headings into the missing locales, or rewrite a block that fails the tone check, all inside the Studio where the editor sees the result. That separation keeps the pipeline fast where it can be and smart only where it must be.

Designing the gate: block, flag, or auto-correct

A quality check is only half a system. The other half is what happens when it fails, and getting that policy wrong is how good checks become resented friction. Three responses cover the field: block the publish outright, flag the issue for a human but let the content through, or auto-correct and continue. The mistake is applying one response to everything.

Severity should drive the response. A missing legal disclaimer on a regulated financial claim is a hard block, because publishing it is a compliance event with real exposure. A link that resolves but is slightly stale is a flag, worth a human's attention but not worth stopping a release for. A missing alt text or an untranslated heading is a strong candidate for auto-correct, where an Agent Action generates the alt text or AI Assist fills the locale and the change is logged for review. Mapping each check to block, flag, or fix, and being explicit about it, is what turns a pile of checks into a governance policy people trust.

The governance layer is as important as the check itself. When an AI auto-corrects content, you need to know what it changed, why, and be able to reverse it. This is where Sanity's Studio and Content Releases carry weight: a flagged or auto-corrected change lands as a reviewable state rather than a silent overwrite, and Content Source Maps trace a field on the live site back to the document and edit that produced it. Roles and Permissions decide who can override a block, and Audit logs record every override. That combination means an AI quality gate is not a black box that editors have to trust blindly; it is a reviewable, reversible workflow. On the compliance side, Sanity runs on SOC 2 Type II controls, supports GDPR obligations, offers regional hosting for data residency, and publishes its sub-processor list, so the enforcement layer itself meets the bar the content is being checked against.

Keeping checks fresh so they don't rot into theater

The quietest failure of an automated quality system is not a check that breaks loudly. It is a check that keeps passing while the rules it enforces have drifted out of date. A tone check written against last year's brand guide, a fact-check pointed at a product spec that has since changed, a link validator whitelisting a domain you no longer own: these pass green while quietly enforcing the wrong thing. Freshness is the difference between a quality gate and quality theater.

The root cause is usually that the check's source of truth is copied instead of referenced. If your fact-check compares generated copy to a hardcoded list of product names embedded in the Function, that list rots the day marketing renames a tier. If instead the check reads the canonical product documents live, it is correct by construction the moment the source changes. The rule is to reference, never duplicate, the ground truth a check depends on.

Sanity is built around exactly this. The Content Lake offers real-time subscriptions, so a workflow sees content the moment it changes rather than on a stale cache. When your quality checks are grounded in a Knowledge Base, the Embeddings Index API keeps semantic search current because the embeddings are tied to the content itself; there is no separate vector pipeline to re-index by hand and no window where the index disagrees with the source. A GROQ query in a publish Function reads the current state of the referencing documents at the instant the check runs. Because Sanity operates content end to end rather than stopping at publish, the same platform that stores the source of truth also runs the checks against it, which is what keeps the two from drifting apart. A quality system that grounds itself in live content instead of copied snapshots is one that stays honest as your business changes underneath it.

Putting it together: a publish-time quality pipeline in practice

Here is what the assembled system looks like when an editor hits publish. The publish event on the Content Lake fires a Function. That Function first runs the cheap deterministic gates: required fields present, all locales populated, every internal reference resolving, external links live. If any hard-block check fails, the publish is stopped and the editor sees a specific, actionable message in the Studio, not a vague error. These checks are milliseconds and cost nothing to run on every document.

Only if the deterministic gates pass does the Function escalate to the AI checks, because those cost tokens and time. An Agent Action, schema-aware, evaluates the content against the model: does the summary faithfully represent the body, does the tone match the voice guide, does any claim contradict the referenced product record or the Knowledge Base. Findings are classified by severity and mapped to the block, flag, or auto-correct policy from earlier. A tone miss might trigger an AI Assist rewrite suggestion the editor can accept; a factual contradiction against the source of truth becomes a hard flag or block.

Everything the AI touches lands as a reviewable change through Content Releases, with the reasoning attached, so nothing is silently rewritten. Editors keep authority; the system removes the toil of manual checking and the risk of human oversight fatigue at volume. The strategic payoff is that this scales output rather than headcount. Instead of hiring more reviewers as content volume grows, you write the checks once against the schema and they run on every matching document forever. That is the concrete meaning of Sanity as an intelligent backend for companies building AI content operations at scale: the AI is wired into the data model, the editor, and the delivery layer, so quality is a property of the system rather than a hope pinned on the last person to look before publish.