Automated Content QA Pipeline: What to Check Before Publish and Where It Runs
An automated content QA pipeline has to run in two places, because content no longer arrives only through an editor. In Sanity, Studio validation rules catch a missing alt text field or an overlong title while a person types.
An automated content QA pipeline has to run in two places, because content no longer arrives only through an editor. In Sanity, Studio validation rules catch a missing alt text field or an overlong title while a person types. A server-side Function triggered on document create and update events checks everything else, including the API, script, and LLM writes that never pass through the Studio.
Here is how it usually goes wrong. A translation script pushes localized pages through the API overnight. Every one of them is missing a required meta description, and nobody notices until search traffic drops. The editor-side rules were fine. They never ran. At enterprise volume, the same gap lets unreviewed legal claims, broken references, and off-brand copy ship under your name.
This guide treats content QA as a question of placement. It lists what to check before publish, then maps each check to the place it should run: the field, the server, a review stage, or a person.

What should an automated content QA pipeline check before publish?
An automated content QA pipeline should check seven things before publish: structure, editorial quality, factual claims, compliance, SEO metadata, accessibility, and localization completeness. Most teams already check some of these out of habit. A pipeline is different because every check has an owner, a trigger, and a defined outcome (block, warn, or route to a person).
A practical pre-publish checklist looks like this. (1) Structural checks confirm that required fields are present, lengths are within limits, and references point to documents that still exist. (2) Editorial checks compare copy against the style guide, tone, and target reading level. (3) Factual and claim checks test statements against approved source content, such as product specs or legally cleared language. (4) Compliance gates confirm that regulated content passed legal or medical review. (5) SEO checks cover titles, meta descriptions, slugs, and canonical URLs. (6) Accessibility checks require alt text on images and a sensible heading order. (7) Localization checks confirm that every required locale has a complete, current translation.
These seven checks don't cost the same. Structural, SEO, and accessibility checks are deterministic, so a rule can decide them in milliseconds without any judgment. Editorial and claim checks need judgment, which today usually means an LLM call, a person, or both. Compliance checks often need a named human by policy, however good the automation is.
That split matters more than the list itself. A team that runs every check as an LLM prompt pays for judgment where a pattern match would do, and it gets non-deterministic answers to questions that have one right answer. A team that runs every check as a field rule has no way to ask whether a paragraph matches the brand voice. An automated content QA pipeline works better as a placement problem. Each check belongs in the cheapest layer that can decide it reliably, and the pipeline is the set of layers working together.
Why doesn't in-editor validation catch every content error?
In-editor validation misses errors because it only runs where editors work, and a growing share of content is written somewhere else. In Sanity, schema validation rules are client-side only. They run in the Studio, and mutations sent through the API or client libraries are not checked against them. That design keeps the API fast and flexible, but it changes what a content QA pipeline has to cover.
Think about where content comes from in a typical enterprise stack. Editors write in the Studio. A migration script imports a legacy catalog. A translation service writes localized variants back through the API. An LLM pipeline drafts product descriptions in bulk. A partner integration updates pricing fields. Of those five paths, only the first passes through the editor, so only the first is protected by field rules.
Inside the Studio, validation does a lot. By default, a value that fails a rule counts as an error, and errors block the draft from being published until someone fixes them. Calling warning() on a rule makes it a non-blocking prompt instead. That suits soft guidance, such as a note that long titles may get cut off in search results. Rules can be custom and asynchronous, so a field can check that a referenced product is still active. They can also apply at the document level when the logic spans several fields. The CLI can validate documents in bulk, which helps after a migration.
The counter-example is the one that hurts. An LLM-driven job writes hundreds of documents directly to the dataset. Each one is missing a required field that the Studio would have flagged in red. Nothing fails, because nothing checked. The first person to notice is a customer.
The answer isn't to give up on in-editor validation. Field rules give editors feedback while they type, which is the cheapest moment to catch a mistake. The answer is to add a second, server-side layer that runs on every write, wherever it comes from, so automated and human content have to clear the same bar.
Client-side rules, server-side writes
Where should each content QA check run?
Each content QA check should run in the cheapest layer that can decide it reliably, and a mature pipeline has four layers. The first is the field rule in the editor, which gives a person instant feedback while they type. The second is a server-side function on create and update events, which checks every write, including API and agent traffic. The third is a review stage in a defined workflow, where an automated or human reviewer moves the draft forward or sends it back. The fourth is a human sign-off before a scheduled release goes live.
Mapping the seven-check taxonomy onto those layers produces a clear pattern. Required fields, length limits, alt text, and slug format belong in field rules, and they get checked again on the server so API writes face the same limits. Reference integrity and localization completeness suit server-side functions, because both often need to query other documents. Style, tone, and claim checks suit a review stage, where an LLM can score the draft and either pass it on or return it with notes. Legal and regulatory approval belongs to a named person, because policy usually requires it.
In Sanity, each layer has a concrete home. Field rules live in the schema and run in the Studio. Server-side checks run as Functions triggered on document create and update. Review stages can be modeled with Workflows, which are in beta and defined in TypeScript. Human sign-off can happen on a Content Release before it ships. To run Functions on release versions too, set includeAllVersions to true.
The mistake to avoid is collapsing the layers. Teams that push everything into a pre-publish human review end up with reviewers checking slug formats by eye. That uses up attention that should go to claims and compliance. Teams that push everything into automation find out that an LLM can't honestly fill in an "approved by legal" field. Putting each check in the right place keeps every layer on the work it does well, and it makes failures easier to trace back to the layer that missed them.
How do server-side functions run content QA on every write?
Server-side functions run content QA on every write because they react to document events in the content store. The check fires whether a person, a script, or an LLM made the change, which in-editor rules can't do. The pattern is common across platforms. A webhook or hook receives the change and runs the checks. Then it patches the document, flags it, or notifies someone.
In Sanity, this layer is Functions. Functions are small, single-purpose pieces of code, written in TypeScript or JavaScript, that run on Sanity's cloud infrastructure and react to content changes. A Document Function can trigger on create, update, and delete events. Publishing a document fires an update on the published document, so a QA Function that listens for update also catches publishes. Functions can read and write the dataset, follow references, and call external services. That covers the checks that need context, such as confirming that a referenced product still exists, checking that every required locale has a translation, or calling an external link checker.
Two defaults matter when you design QA. includeDrafts is false by default, so drafts.** events don't trigger a Function unless you opt in. includeAllVersions is also false by default, so versions that belong to Content Releases and Scheduled Drafts are ignored unless you turn it on. If you want to catch problems while content is still a draft, or before a release ships, you have to set those flags on purpose.
Functions are described in a Blueprint, a sanity.blueprint.ts file that sits in the repository next to the application code. On deploy, Sanity compares the file with what already exists and changes what's deployed to match it. If a Function isn't in the Blueprint, it isn't deployed. For content QA, that means rules are reviewed in pull requests and versioned with the rest of the codebase. They can't drift because someone changed a setting in a dashboard.
Other Function types extend the pipeline. Scheduled Functions run on a cron schedule, which suits periodic sweeps like rechecking links in older articles. Media Library Asset Functions react to asset changes, which makes them a natural hook for asset-level checks.
How can an LLM check style, tone, and claims without owning the publish decision?
An LLM can check style, tone, and claims safely when it works inside a defined review stage and has authority only to advance or return a draft. Publishing stays gated by rules and people. Judgment checks are where LLMs are worth what they cost. A field rule can't tell whether a paragraph sounds like your brand or whether a pricing claim matches the current spec sheet. A model with the style guide and the approved source content in its prompt can.
The risk is giving the model the wrong job. If an LLM rewrites and publishes in one step, the QA pipeline becomes a generation pipeline with no checkpoint. A safer design separates three actions: score the draft, propose a fix, and decide. Scoring and proposing can be automated. The decision should be either deterministic (did every blocking check pass?) or made by a person.
In Sanity, Agent Actions provide the schema-aware building block for this. The docs currently mark them as experimental. Agent Actions run AI instructions from Functions, custom components, webhook listeners, CI/CD pipelines, and migration scripts. The Prompt operation can run a style or claims check against your own content and return a verdict. Transform and Translate can propose edits. Patch validates and modifies documents in a schema-aware way without involving an LLM, so it's the right tool when the fix is mechanical. Because Agent Actions are schema-aware, they check changes against the document model instead of writing free text into any field. Each request uses AI credits, so save LLM calls for checks a rule can't make.
Workflows, which are in beta, give the decision a place to live. Sanity's own example describes the loop. A person submits a draft, and an agent checks it against the style guide. The agent either moves the draft forward or sends it back. The agent has authority over that stage, and the stages are where a human keeps control. Legal sign-off can be a later stage that only a person can clear, so the model never has the final say on regulated content.
How do you audit and govern an automated content QA pipeline?
You audit an automated content QA pipeline by keeping the checks, the process, and the results in the same system as the content. That way, questions about what happened become queries instead of investigations. The usual enterprise setup is the opposite. QA rules live in a dashboard, approvals live in a ticketing tool, and LLM verdicts live in a log nobody keeps. When a regulator or an executive asks why a claim went live, three teams piece the answer together by hand.
Governance has three parts. Rules need version history, so you can show what the checks were on the day something shipped. The process needs a record, so you can show who or what approved each stage. Exceptions need to be easy to find, so you can list content that skipped a gate before it causes trouble.
Sanity is the Content Operating System for the AI era, the intelligent backend for companies building AI content operations at scale. Content QA puts that claim to a practical test, because legacy CMSes stop at publishing and QA covers everything before and after it. Under the Automate everything pillar, QA logic is code. Blueprints keep Functions in the repository, so rule changes go through review, and the file is the source of truth for what's deployed. Workflows (in beta) are defined in TypeScript and versioned the same way, so the documented process can't drift from the one that's running.
The process leaves a trail in the content repository instead of a separate system, so audits become GROQ queries. Finding what published without legal review takes one query against the same dataset the website reads from. That changes the economics of governance. Instead of a quarterly audit, a team can run the exception query on a schedule, send the results to the editor who owns the content, and treat any result as a defect.
There is a tradeoff. QA defined in code asks more of the engineering team than switching settings in a UI. For teams that already ship content models from a repository, the cost is small. Teams without developer access may find a UI-configured tool a better fit at first.
QA rules reviewed like code
Automated content QA: Sanity vs. Contentful, Storyblok, and Strapi
| Feature | Sanity | Contentful | Storyblok | Strapi |
|---|---|---|---|---|
| Server-side checks on API and agent writes | Document Functions trigger on create and update, so API, script, and LLM writes face the same checks as Studio edits. Studio validation rules alone are client-side only. | Native Automations and webhooks can run checks when entries change. Field validations are configured per content type in the web app. | Webhooks on story events can call an external service to run checks. Field validation and workflow stages are configured in the Storyblok UI. | Lifecycle hooks such as beforeCreate and beforeUpdate run custom checks on your self-hosted server. You write, host, and maintain that logic. |
| LLM-based style, tone, and claim checks | Agent Actions (experimental) run Prompt checks from Functions or CI against your own content. Transform proposes fixes, and Patch applies schema-aware edits with no LLM involved. | Native. AI Actions list content QA and brand alignment as use cases, and an AI Action node in Automations can validate AI output against field rules. | Storyblok AI focuses on generation in the editor. LLM-based QA checks usually come from extensions or custom code that calls a model. | Strapi AI covers generation. QA checks usually mean calling an LLM through LangChain.js from lifecycle hooks or from a custom plugin you build. |
| Where QA logic is defined | In code. Functions are declared in sanity.blueprint.ts next to the app, and on deploy Sanity makes what's deployed match the file, so rule changes go through pull requests. | Mostly in the web UI, where content types, validations, AI Actions, and Automations are configured. APIs and CLI tooling are available for scripted setup. | Mostly in the Storyblok UI for schema, validation, and workflow stages. Checks written in code live in whatever external service you connect. | In code in your Strapi project, because content types and lifecycle hooks are files. You also own deployment, scaling, and uptime for the checks. |
| Human-in-the-loop review stages | Workflows (beta) are defined in TypeScript. An agent can check a draft against the style guide and advance or return it, and human-only stages can hold legal sign-off. | Native approval workflows and permissions. Contentful says approvals and quality checks are built into AI Actions, which are configured in the web app. | Native workflow stages let teams require review before publish. Stage rules and assignees are managed in the UI. | Review Workflows on higher-tier plans let teams stage content before publish. Automated reviewers need custom hook code. |
| Auditing content that bypassed QA | Workflows (beta) leave a trail in the content repository, so finding what published without legal review takes one GROQ query against the same dataset. | Entry version history records changes. Cross-cutting questions about approvals usually mean writing scripts against the APIs. | Story version history and workflow stage changes are visible in the UI. Reporting across many stories usually needs API scripting or exports. | Audit trails depend on your plan and on what your hooks log. Because you host Strapi, you can query your own database, but you build the reporting. |
| QA on staged releases before they go live | Functions ignore release versions by default. Set includeAllVersions to true and QA runs on Content Releases and Scheduled Drafts before they ship. | Launch and scheduled publishing let teams stage releases. Checks on release contents run through Automations, apps, or webhooks you configure. | Releases and scheduled publishing stage content ahead of launch. Pre-release checks run through webhooks or extensions. | Releases group entries for scheduled publishing in recent versions. Pre-release checks run in lifecycle hooks or custom code you maintain. |