How to Use AI to Backfill Missing Metadata on Old Pages

You inherit a site with 40,000 pages, and half of them are missing meta descriptions, alt text, or Open Graph tags. Search crawlers deprioritize them, social shares render as gray boxes, and the AI answer engines that now sit between your content and your audience skip past pages they cannot parse. Nobody is going to hand-write 20,000 descriptions, so the gap just sits there, quietly bleeding traffic.

Sanity, the AI-native content platform, reframes this from a data-entry backlog into an automated content operation. As the Content Operating System for the AI era, it treats the missing-field problem as two jobs it already does well: find the documents that are incomplete, then generate the right value against the actual schema. This guide walks the enterprise version of that workflow. First the failure modes that make naive backfill produce garbage, then the mechanics of finding gaps, grounding the model in your content's real shape, and governing what ships. The point is not a bigger model. It is giving a capable model the context and the guardrails to write metadata you would have approved by hand.

Why missing metadata is an operations problem, not a content problem

Missing metadata rarely comes from carelessness. It accumulates from migrations that dropped fields, schema changes that added a metaDescription nobody backfilled, imports from a legacy system that never had alt text, and years of editors publishing under deadline. By the time anyone notices, the gap is measured in tens of thousands of pages, which puts it firmly outside what a content team can clear by hand.

The stakes have shifted, too. Meta descriptions and structured data used to be an SEO nicety. Now they are how machines decide whether your page is worth surfacing. Answer engines and AI overviews lean on clean, structured signals to decide what to retrieve and quote. A page with no description and no alt text is not just ranked lower, it is harder for an LLM to parse, summarize, and cite correctly. The cost of the gap compounds as more of your traffic arrives through an AI intermediary.

The reason this stays unsolved is that most CMSes stop at publishing. They give you a field to fill and a form to fill it in, and the moment the work exceeds human throughput, you are on your own. What the problem actually needs is a content backend that can query for its own gaps and act on them at scale. That is the frame for the rest of this guide: backfill is an automation pipeline, and the CMS should be the thing running it, not a passive store you script around from the outside.

Step one: find the gaps with structural queries, not a crawl

Before you generate anything, you need an exact list of what is missing, and 'missing' has to be defined structurally. A meta description that is present but empty, an alt text that is a single space, a hero that references a media asset with no accompanying caption: each is a different failure that a naive crawler flattens into 'looks fine.' You want a query that understands your data model well enough to distinguish them.

This is where a content backend earns its keep. In Sanity, GROQ lets you filter documents on the presence, absence, or shape of any field directly against the schema. You can select every article where metaDescription is not defined, or where altText is an empty string, or where an og:image reference points at a deleted asset. The result is a precise worklist, not an approximation, and it updates the instant content changes because it runs against Content Lake rather than a stale export.

Structural filtering is exactly where pure semantic tools fall down. Vector search and 'AI-powered search' products are built to answer 'find me something like a trail runner.' They are not built to answer 'find every document where this field is missing, in this content type, published before 2022.' As the research puts it, pure embeddings work for fuzzy semantic match and fall over for anything structural. Metadata backfill is a structural problem first: you cannot generate what you cannot reliably enumerate. Getting the worklist right is half the job, and it is the half that a keyword-plus-embeddings search bolt-on cannot do on its own.

Step two: ground the model in the real shape of your content

Here is where most backfill projects quietly produce garbage. You point a model at a page, ask it for a meta description, and it writes something fluent and wrong because it does not understand what it is looking at. The failure is almost never the model. It is context.

Sanity's own schema-exploration work makes this concrete. Running retrieval against Sonos's catalog, an honest nightmare of a dataset, the team reached about 83% accuracy on a mix of difficulties, and getting there meant teaching the retrieval step the things a schema alone will not tell you. Counter-intuitive field names: a field called body that is actually a slug. A hero that is a reference to a mediaAsset, not an image. Second-order reference chains the schema does not connect. Data-quality issues the schema cannot reveal, like a features array that is always empty, so you use support-product instead. None of that is a model problem. The model needed to know the shape of the data, not just its types.

For metadata backfill the implication is direct. To write a good description for a product page, the model needs the actual product name, its real category, the specs that live three references away, and the knowledge that the field it might naively summarize is empty for half the catalog. Give it that context and a mid-sized model writes metadata you would approve. Give it a bare page and the largest model available and you get plausible nonsense. Backfill quality is a context problem, and the content backend is where that context lives.

Step three: generate against the schema with Agent Actions

Once you have the worklist and the context, generation should run through something that knows your schema, not a raw call to an LLM API that returns free text you then have to parse and reshape. This is the distinction between an AI feature and an AI-native architecture.

Agent Actions, now the Agent API, are schema-aware APIs for generating, transforming, translating, and validating content with LLMs, exposed over HTTP anywhere you can run code. For backfill that means you can ask the platform to generate a metaDescription for a specific document, and the operation understands that metaDescription is a string with a length constraint, that altText belongs on the image object inside the hero, and that the value has to validate against the real model before it can land. You are not gluing a chat completion to a hand-rolled field mapper. The schema is the interface.

This maps cleanly to two of Sanity's pillars. 'Automate everything' is the pipeline itself: Functions can trigger enrichment when a document changes or a schema field is added, so new gaps close automatically instead of accumulating into the next backlog. 'Power anything' is the output: structured, semantically clear content that GROQ delivers to every frontend, feed, and agent downstream. The difference from the bolt-on approach is architectural. Legacy CMSes bolt AI on after the fact, while here the generation primitive speaks the same schema language as the store it writes to, which is what makes bulk backfill reliable instead of a one-off script you babysit.

Step four: govern the output before it ships

A backfill pipeline that writes straight to production is a liability, not a feature. Twenty thousand machine-generated descriptions published without review is how you turn a metadata gap into a metadata incident: subtly wrong claims, mismatched tone, hallucinated specs, all live and indexed before anyone notices. AI-touched content needs the same governance as everything else on the site.

Sanity handles this by staging AI output the same way you stage a website. With Content Releases and the Studio you get drafts, scheduling, history, permission gating, and audit trails. Backfilled metadata lands as drafts you can preview, review, and bulk-approve, or reject, before it goes anywhere near production. An editor can spot-check a representative sample, a lead can gate who is allowed to publish machine-written fields, and the audit trail records what was generated, by which operation, and who approved it. That last part matters when someone asks in six months why a description reads the way it does.

This is the practical answer to the trust problem that stalls most AI content initiatives. The blocker is rarely whether the model can write a decent description. It is whether the organization is willing to let unreviewed machine output touch the live site. By treating backfill as staged content rather than a fire-and-forget write, you keep a human in the loop where it counts and remove the human from the ten thousand cases where review is a formality. That is how scaling output stops meaning scaling headcount.

Doing this on a stack where AI is bolted on

You can approximate this workflow on almost any CMS, and it is worth being honest about how. The differences are not about whether AI generation is possible. It is about how much of the pipeline you own versus how much the platform owns.

On Contentful, developers use the App Framework plus React and an AI assistant to build sidebar apps. Generation exists, but schema-aware bulk backfill against version-controlled models is harder, and the deeper 'AI chatbot for Contentful' products are third-party bolt-ons. Strapi leans on official LangChain.js and Next.js tutorials, which means enrichment is a build-it-yourself orchestration exercise wired to an external LLM stack rather than a native, schema-aware content operation. Payload has the community payload-ai plugin, real capability with completions, embeddings, and moderation, but it is a plugin you install and maintain, not a first-party governed pipeline. Directus offers a first-party OpenAI integration wired into Flows, a genuinely good low-code story, though enrichment runs through flow steps rather than schema-aware transform APIs.

None of these are incapable. The distinction, and it is the one that matters at 20,000 pages, is that in each case the structural query layer, the freshness pipeline, the schema-aware generation primitive, and the governance layer are separate systems you integrate and keep in sync. When a description updates, a field changes, or a document is deleted, the index has to know. Building that yourself, incremental indexing, re-embedding on change, deletion handling, backfill for schema changes, is a real project and a class of bug all its own. Wiring retrieval and generation into the content backend removes it as a roadmap line item.

Freshness: why backfill is a loop, not a one-time job

The tempting way to think about metadata backfill is as a project with an end date: clear the 20,000 gaps, close the ticket, move on. That framing is wrong, and it is wrong in a way that costs you a second backlog eighteen months later. Gaps regenerate. Every schema change that adds a field, every import, every new content type reopens the hole you just closed.

The durable version treats backfill as a standing loop, and that only works if retrieval and generation stay current with the content automatically. Content Lake keeps the search and retrieval index fresh by default. When a product description updates, when a price changes, when an article publishes, when a record is deleted, the index knows. That is precisely the freshness pipeline the bolt-on approach forces you to build and babysit, and having it as a property of the backend rather than a service you operate is what lets you wire enrichment to a Function that fires on change. A new document missing a description gets one before it is ever indexed as incomplete.

This is the sense in which Sanity is the intelligent backend for companies building AI content operations at scale. The metadata problem is a symptom of a deeper property: content that is structured, queryable, and fresh can be operated on by machines continuously, while content that is not has to be operated on by people episodically. For the retrieval-heavy end of this, grounding agents in your content for RAG and answer generation, the deeper treatment lives on agent-context.org; here the point is narrower and it is enough. Own the freshness and the query layer, and backfill stops being a project you finish and becomes a state your content maintains.