How to Use AI to Flag Outdated or Underperforming Content Automatically

Most content graveyards are quiet failures. A pricing page still cites last year's tiers, a "best practices" guide recommends a feature you deprecated two quarters ago, and a support article that once drove signups now quietly repels them. Nobody flagged any of it, because flagging outdated content is somebody's job the way cleaning the office kitchen is somebody's job. The rot compounds: search engines demote stale pages, AI answer engines cite the wrong facts, and your editorial team spends its energy chasing the loud requests instead of the silent liabilities.

Sanity, the AI-native content platform, treats this differently. Sanity is the Content Operating System for the AI era, an intelligent backend where content is structured, queryable, and wired for automation, so "find the stale pages" becomes a scheduled query instead of a manual audit nobody has time to run. The distinction matters: a headless CMS stops at storing and publishing, while a Content Operating System operates content end to end, including the unglamorous work of noticing when it has gone bad.

This guide walks through how to build automated freshness and performance signals, combine them with LLM judgment, and route the flags to the humans who can act on them, using Sanity's Functions, Agent Actions, AI Assist, and embeddings as the working example.

Why stale content is an operational problem, not a cleanup task

Every content team has an intuition that some fraction of its library is dead weight, but intuition does not survive a growing catalog. Once you cross a few thousand documents, no human maintains an accurate mental map of what is current. The failure mode is not that anyone chose to publish wrong information; it is that content decays passively while the world moves. A product changes, a regulation updates, a competitor's name shifts, and the page that mentioned any of those keeps sitting there, confidently wrong.

The stakes have risen sharply in the AI era. Outdated pages no longer just underperform in traditional search; they get retrieved and cited by answer engines and by your own retrieval-augmented workflows. If an internal support agent pulls a deprecated troubleshooting step, or an AI Overview quotes a price you no longer charge, the stale document has now actively misinformed a customer at scale. The blast radius of one neglected page is larger than it has ever been.

Treating this as a periodic cleanup project guarantees failure, because cleanups happen when someone finds time, and nobody finds time. The durable fix is to make freshness a continuously computed property of every document, the same way you track a publish date or an author. That reframing, from occasional audit to always-on signal, is only practical when your content is structured and programmatically queryable. When each document exposes its own metadata, relationships, and body as structured fields, a machine can reason about staleness the moment it appears rather than months later during a manual sweep.

The signals that actually predict outdated or underperforming content

Automated flagging is only as good as the signals feeding it, and most teams reach for a single crude one, usually last-modified date, then wonder why the results are noisy. Age alone is a weak predictor: an evergreen glossary entry can be five years old and perfectly correct, while a pricing page can go stale the week after you edit it. You need a blended signal that combines freshness, performance, and semantic drift.

Freshness signals are the cheapest to compute: last publish date, last human edit versus last automated touch, and, more usefully, references to time-sensitive entities. If a document links to a product, a price, or a policy that has since changed, that reference is a staleness flag even if the page itself was never touched. Performance signals come from your analytics layer: pageviews trending toward zero, search impressions with collapsing click-through, rising bounce, or conversion assists that have dried up. Underperformance and staleness are different problems that often share a root cause, so tracking both surfaces the pages worth a human's attention.

The hardest and most valuable signal is semantic drift, whether a document still says what your current source of truth says. This is where structured content earns its keep. Because Sanity stores content as Portable Text rather than an opaque HTML blob, you can compare a document's actual claims against canonical reference content, and because embeddings are tied to the content itself, a semantic search over your library stays fresh automatically instead of drifting behind a separately maintained vector pipeline. The signal you can compute cheaply is a direct function of how structured your content already is.

Wiring freshness into the content model with a scoring pipeline

A flag nobody sees changes nothing, so the first engineering decision is where the staleness score lives. The answer is: on the document, as a field, computed continuously. Model your business first. Add fields such as a review-due date, a computed freshness score, a last-verified-by reference, and an array of flagged reasons. Now staleness is not a report that lives in a spreadsheet; it is a queryable property of content that editors, dashboards, and downstream systems can all read.

Sanity Functions are the mechanism that keeps those fields current. Functions are serverless automation hooks that fire on content events, so you can run enrich-on-publish and recompute-on-schedule pipelines without standing up your own infrastructure. A scheduled Function can walk the dataset, pull performance metrics from your analytics source, check each document's time-sensitive references against their current values, and write an updated freshness score back onto every document. Because Content Lake exposes real-time subscriptions, the moment a referenced product or price changes, the documents that depend on it can be re-scored immediately rather than waiting for the next batch.

The payoff of doing this inside the content model, rather than in an external job that dumps results into a dashboard, is that the flag travels with the content. A GROQ query for every document with a freshness score below a threshold, ordered by traffic, is a two-line audit that any editor can run on demand. The stale-content problem stops being an archaeology project and becomes a saved view.

Adding LLM judgment: from mechanical flags to explained recommendations

Mechanical signals get you a ranked list of suspects. They cannot tell you why a page is wrong or what to do about it, and a list of five hundred low-scoring URLs with no explanation is just a new backlog. This is where LLM judgment turns a flag into an actionable recommendation, and where the difference between AI bolted onto a CMS and AI wired into it becomes concrete.

Agent Actions are schema-aware LLM operations that generate, transform, translate, or validate content against your actual data model. Pointed at a flagged document, a validate action can check the body's claims against a canonical Knowledge Base, for example your current product facts or a policy source, and return a structured verdict: which specific sentences conflict with current truth, and what the corrected version would be. Because the action understands the schema, its output writes back into the right fields instead of arriving as freeform prose an editor has to transcribe. For lighter-touch work, AI Assist gives editors in-Studio helpers to fact-check a block against the knowledge base, summarize what changed, or rewrite an outdated section in the page's existing voice.

The reframing here matters. Legacy CMSes bolt AI on as a plugin that generates copy in a side panel; Sanity is built so that the same LLM operations run as governed pipeline primitives and as editor tools over the same structured content. That is the difference between an AI that can write a blog post and an AI that can audit ten thousand documents, explain each verdict against your source of truth, and stage the fixes for review.

Keeping the human in the loop: governance for AI-flagged content

Automated flagging that auto-corrects and auto-publishes is a liability, not a feature. The whole point of catching outdated content is accuracy, and an unreviewed LLM edit can introduce a new error while fixing an old one. The right design routes every AI-generated correction through human review, and the CMS is where that governance has to live, because that is where the content and the reviewers already are.

In practice this means the flagging pipeline never writes to a published document directly. Corrections land in a draft or a Content Release, a staged set of changes an editor reviews, approves, and schedules together. An editor sees the flag, the AI's explanation, the proposed edit, and the source it was checked against, then makes the call. Visual Editing lets them see the change in the context of the rendered page rather than as a raw diff. Roles and Permissions decide who can approve edits to sensitive content such as pricing or legal pages, and Audit logs record who approved what and when, which is exactly the trail you need when a regulator or a customer asks why a claim changed.

This is the governance dividend of doing AI content work inside a Content Operating System rather than in a pile of disconnected scripts. The flag, the explanation, the fix, the review, and the record of the decision all live against the same document. Legacy CMSes create silos where automation runs in one system and review happens in another; the shared foundation is what lets a team trust an automated pipeline enough to actually run it on production content.

Closing the loop: measuring whether your flagging is working

A flagging system is itself content operations, and content operations that do not measure their own output slowly stop being trusted. The risk is a pipeline that generates a thousand flags a week, editors ignore because the precision is bad, and the whole apparatus quietly becomes noise. You need to treat the flagger like any other model you deploy: track its precision, its recall, and its throughput.

Precision is the share of flags that a human agreed with. If editors dismiss most flags, your thresholds are wrong or your signals are weak, and you should tighten before you scale. Recall is harder but more important: the stale pages that slipped through, usually discovered when a customer or a colleague reports one. Log both outcomes back onto the documents, editor accepted this flag, editor dismissed that one, and you have a labeled dataset that tells you which signals actually predict rot in your specific library. Freshness scoring is not a set-and-forget config; it improves as it learns which of your signals matter.

The strategic payoff is a shift in how the team scales. Rigid CMSes force you to scale people, hiring another editor every time the catalog grows, because auditing is manual labor that grows with the library. An always-on flagging pipeline scales output instead, letting a fixed team keep a growing catalog accurate because the machine does the noticing and the humans do the judging. That is the promise of an AI-native content platform: the boring, unbounded work of watching for decay becomes automated, and human attention gets spent where judgment is actually required.