Top 5 CMS-Level Content Cleanup Tasks Before Deploying AI Search or RAG

Six weeks after launch, your AI search returns a confident answer citing a product page that was deprecated last quarter, pulls a phone number from a 2019 press release, and blends two unrelated policies into one hallucinated sentence. Nobody wrote bad prompts. The retrieval layer did exactly what it was told: it indexed whatever was in the CMS, including the duplicates, the orphaned drafts, the rich text that lost its structure on the way into the chunker, and the fields nobody agreed on the meaning of. Garbage content produces garbage grounding, and RAG makes every content-hygiene sin newly visible to users.

Sanity is the AI-native content platform built to prevent exactly this class of failure, an intelligent backend where the content model, the editor, and the delivery layer are all wired for LLM workflows rather than bolted onto them. Before you stand up an embeddings pipeline or wire an agent to your content, the highest-leverage work happens one layer down, in the CMS itself.

This is a ranked list of the five CMS-level cleanup tasks that pay off most before you deploy AI search or RAG, ordered by impact. Each one is boring, each one is skippable, and skipping each one is why so many AI search projects quietly underperform.

1. Deduplicate and resolve conflicting sources of truth

The single biggest predictor of a hallucinating retrieval system is not the model, it is the presence of two documents that both claim to be authoritative and disagree. When your CMS holds a canonical pricing page, a stale campaign landing page with last year's prices, and a sales enablement doc that splits the difference, the retriever has no way to know which one wins. It surfaces all three, the model averages them, and the user gets a number that appears nowhere in your business.

What cleanup looks like: enumerate every document type that describes the same real-world entity, pick one system of record per entity, and either delete, redirect, or explicitly mark the rest as non-canonical so they never reach the index. This is unglamorous inventory work, but it removes more bad answers than any prompt engineering.

Where a legacy CMS fits poorly here is that it stops at publishing. It will happily serve three contradictory pages because serving pages is all it was built to do. Sanity operates content end to end: a single Content Lake dataset, referenceable documents, and schema-level constraints mean you model canonicality once and enforce it everywhere. Concretely, a documentation team can model a `product` document as the one source, reference it from every landing page, and let the Embeddings Index API index the referenced source rather than the copies. The vector layer inherits your canonical model instead of fighting it.

Example: a support team found their AI search citing a refund window of both 14 and 30 days. The 14-day figure lived in an archived FAQ nobody had unpublished. Deleting one document, not tuning the model, fixed the answer.

2. Restructure rich text so chunking preserves meaning

Retrieval systems chunk documents into passages before embedding them. If your rich text is stored as an HTML blob or a Markdown string, the chunker slices it on character counts and routinely cuts a heading away from the paragraph it introduces, orphans a list item from its stem, or strips the table structure that made a spec sheet legible. The embedding then represents a fragment that means nothing on its own, and retrieval quality collapses in ways that are hard to debug because the source document looked fine.

The fix is structural: store rich text as structured data, not as a serialized string, so that blocks, annotations, and marks survive the trip through chunking, retrieval, and generation. This is exactly what Portable Text is for. Because it represents content as an array of typed blocks with explicit annotations rather than as opaque HTML, a chunker can split on block boundaries, keep a heading attached to its section, and carry a link's destination as structured metadata rather than losing it inside a tag.

Where a legacy CMS fits poorly: many store the body field as HTML or a WYSIWYG blob, which means you are reverse-engineering structure at index time with brittle parsers. Sanity was built the other way around; the structure is native, so the same content that renders your website also feeds your retrieval layer without a lossy conversion step.

Example: an ecommerce team's size-guide tables were being flattened into run-on text by their chunker, so AI search answered sizing questions with garbled measurements. Modeling the tables as structured blocks let the retriever preserve rows and columns, and the answers became correct without touching the model.

3. Fill in the metadata the retriever needs to filter and rank

A pure semantic search over your entire corpus is rarely what you want. You want semantic relevance filtered by language, region, product line, freshness, and access level. That filtering only works if the metadata exists and is consistent. In most CMSes it does not: locale is inferred from a URL prefix, product line lives in a free-text tag field with nine spellings, and publish date is the only temporal signal, so a five-year-old evergreen page ranks alongside last week's release notes.

Cleanup here means promoting the dimensions your business actually filters on into first-class, validated fields, then backfilling them across the corpus. Language, market, product, content type, and an explicit review date matter far more than most teams assume until they see retrieval pulling German content into an English answer.

Sanity's stance is that you model your business, not the CMS's idea of it. Schema-level validation makes these fields required and constrained rather than optional and free-text, so the metadata is trustworthy by the time it reaches the index. Dataset embeddings and the Embeddings Index API let you combine semantic similarity with these structured filters in a single query, and because embeddings are tied to the content, a metadata correction re-flows automatically rather than requiring a manual re-index.

Example: a global brand's AI search kept answering EU privacy questions with US-jurisdiction copy. The documents were semantically near-identical; only a governed `market` field, added at the schema level and backfilled, let the retriever keep them apart.

4. Unpublish, archive, or gate the content that should never be retrieved

Every CMS accumulates content that is technically live but should never appear in an AI answer: internal drafts left in a public dataset, superseded legal terms retained for reference, embargoed announcements, and half-finished pages that were never linked but never deleted. A website navigation hides these by simply not linking to them. A retrieval system does not care about your navigation; it indexes the dataset, and anything reachable is fair game for a confident citation.

The cleanup task is a governance audit: decide, per document type, what belongs in the retrievable set, and enforce that boundary with state rather than with the accident of what happens to be linked. Draft, in review, published, archived, and restricted should be explicit statuses that the indexing pipeline reads and respects.

This is where Sanity's editorial governance earns its place in an AI stack. Studio Workspaces, Content Releases, and Roles & Permissions give you a real publishing state machine, and Functions let you hook the moment of publish or unpublish to keep the index in lockstep, for example an enrich-on-publish or moderate-on-publish step that decides whether a document is eligible for retrieval at all. The result is that governance for LLM-touched content is a workflow, not a nightly cleanup script.

Example: a fintech team discovered their assistant quoting a compliance policy that had been superseded but left published for legal recordkeeping. Moving those documents to an explicit archived state, excluded from the embeddings index, removed the risk without deleting the record they were required to retain.

5. Standardize terminology and voice so grounding is consistent

This one ranks last because it is the least urgent and the most often skipped, but it is what separates an AI search that sounds like your company from one that sounds like a committee. When the same feature is called three names across your content, when tone lurches from legalese to marketing hype page to page, and when acronyms are expanded inconsistently, the retriever surfaces the variance and the generated answer inherits it. Users notice; the assistant feels unreliable even when the facts are right.

Cleanup means agreeing on a controlled vocabulary and a voice, then bringing existing content into line. Historically that was a quarter of manual editing that never got prioritized. This is the task where AI helps clean up the content for AI. AI Assist can run inside the Studio to rewrite a block into an approved voice, expand acronyms consistently, or flag terminology that drifts from the glossary, and Agent Actions can apply those transformations across many documents through schema-aware, reviewable workflows rather than one page at a time.

The distinction that matters: legacy CMSes force you to scale people to scale content cleanup, while an AI-native platform scales output. Sanity is built for AI rather than having it bolted on, so the same editorial surface where humans work is where the automated cleanup runs, staged through Content Releases and reviewed before it ships.

Example: a SaaS team had four names for one product tier scattered across 600 documents. An Agent Actions transform normalized them in an afternoon, staged for editorial review, so the assistant stopped inventing a fifth.