Why Your CMS Needs Native Embeddings, Not a Side Vector DB

Your search box works fine until the day someone renames a product line. Sanity ships the update, the CMS shows the new copy instantly, and then a support agent asks your AI assistant about the old name and gets a confident, wrong answer. The vector database still holds embeddings generated from last week's text. Nobody ran the re-index job, or the job ran but silently skipped the changed documents, and now your retrieval layer is quietly lying to every downstream LLM. This is the tax of the side-car vector DB: a second source of truth that drifts from the first the moment content moves.

Sanity is the AI-native content platform, an intelligent backend built so that semantic search is a property of your content rather than a separate system you keep in sync by hand. That reframing matters. When embeddings are tied to the content that produced them, freshness stops being an operational chore and becomes a guarantee.

This guide walks through why the bolt-on vector database pattern breaks at enterprise scale, what "native embeddings" actually means mechanically, and how to reason about the trade before you wire yet another pipeline into your stack.

The hidden cost of a second source of truth

Every side vector database starts as a reasonable decision. You have content in a CMS, you want semantic search, and the fastest demo path is to pipe documents into a dedicated vector store, generate embeddings, and query them at runtime. The demo works. The problems arrive later, in production, when content changes constantly and the two systems no longer agree.

Consider the failure modes an enterprise actually hits. An editor publishes a correction, but the sync job runs on a schedule, so for hours the vector index answers from stale text. A document is deleted, but the corresponding vector lingers because deletes are the easiest event to drop. A field is restructured, the chunking logic changes, and half your corpus is embedded one way while the other half is embedded another. None of these are exotic. They are the ordinary consequence of maintaining two systems that model the same content with different update semantics.

The operational overhead compounds. You now own an embedding pipeline, a re-index scheduler, a dead-letter queue for failed syncs, monitoring for drift, and a backfill script for the inevitable day the schema changes. This is infrastructure whose only job is to keep two copies of the truth from disagreeing. Legacy CMSes create silos, and a bolt-on vector DB is a silo you built on purpose. The question worth asking is whether semantic search should be a separate system at all, or a capability the content platform owns end to end.

What native embeddings actually mean

Native embeddings means the vectors live with the content, generated and maintained by the same platform that stores the source. When a document changes, the embedding for that document is the platform's responsibility, not yours. There is no external pipeline to trigger, no scheduler to trust, and no drift window to monitor, because the freshness of the index is a property of the system rather than an integration you assembled.

Sanity's Embeddings Index API and dataset embeddings implement this pattern directly. You define what to index, and embeddings are tied to the content in the Content Lake, so when content is created, edited, or deleted the semantic layer reflects it without a separate re-index job. This is the difference between semantic search as a feature of your content and semantic search as a database you keep in sync.

The structural payoff shows up in how you query. Because embeddings and content share a home, you can blend semantic similarity with exact filters in a single query rather than fanning out to a vector service, collecting IDs, and hydrating them from the CMS in a second round trip. You ask for documents that are semantically close to a phrase AND belong to a category AND are published in a locale, and it resolves in one place. That is not a convenience. It removes an entire class of consistency bug, the one where your vector store returns an ID for a document your CMS has already unpublished.

This maps cleanly to Sanity's automate everything pillar: the plumbing that would otherwise consume an engineering team is handled by the platform, so the team ships retrieval features instead of babysitting sync jobs.

Freshness is a correctness problem, not a latency one

Teams tend to frame retrieval quality as a latency and relevance problem: faster queries, better ranking, tighter recall. Those matter, but they are downstream of a more fundamental property. If the index does not reflect the current state of the content, no amount of ranking sophistication saves you, because you are ranking the wrong documents. Freshness is a correctness problem wearing a performance costume.

This matters most when an LLM sits on the other end of retrieval. Retrieval-augmented generation is only as trustworthy as the passages it retrieves. Feed a model a stale chunk and it will faithfully summarize outdated policy, quote a deprecated price, or cite a product that no longer exists, all with the fluent confidence that makes hallucinations dangerous. The model did nothing wrong. The retrieval layer handed it the wrong ground truth.

With embeddings tied to content, the moment an editor publishes a change the semantic layer moves with it, so the freshest possible passage is the one available to retrieval. Sanity's Content Lake real-time subscriptions extend this further, letting downstream LLM workflows react to content the instant it changes rather than on a polling interval. Portable Text reinforces the same guarantee at the structural level: because rich text is stored as structured blocks with annotations and marks rather than opaque HTML, chunking for retrieval preserves headings, references, and boundaries instead of shredding them, so the passage a model retrieves is a coherent unit rather than a fragment severed mid-thought. Correctness, in other words, is designed in rather than patched on.

When a dedicated vector database still earns its place

Native embeddings are the right default, but honesty requires naming the cases where a dedicated vector database genuinely earns its keep, because pretending otherwise erodes trust and leads teams to the wrong architecture.

If your primary corpus does not live in your CMS, a vector store makes sense. A company embedding billions of log lines, telemetry events, or third-party documents that never pass through editorial has no content platform to attach those vectors to, and a purpose-built vector database is the correct home. Likewise, if you need exotic index tuning, custom distance metrics, or billion-scale approximate nearest neighbor performance with hardware-level control, a specialist system like Pinecone or a self-managed pgvector deployment gives you knobs a content platform reasonably does not expose.

The distinction is about where the source of truth lives. When the content you want to search is the content your team authors and governs, keeping embeddings in the same system removes the sync problem entirely, and that is the common enterprise case: help centers, product catalogs, documentation, marketing sites, and knowledge bases. When the vectors describe data that originates elsewhere and flows through a separate pipeline anyway, the sync problem exists regardless, so a dedicated store adds capability without adding a new class of drift. The wrong move is the middle case, bolting a vector DB onto content that already lives in a platform capable of owning the embeddings, and inheriting a synchronization burden you did not need to accept.

Governance: the part the side-car quietly breaks

The synchronization argument gets the attention, but the governance argument is arguably sharper. When embeddings live in a separate vector database, they become a second copy of your content sitting outside your permissions model, your audit trail, and your compliance boundary. That copy is easy to forget and expensive to explain.

Ask the questions an enterprise security review will ask. Who can read the vectors? A vector is not encrypted content, but for text-heavy corpora the embedded passages and their metadata often reproduce sensitive material, so the vector store is effectively another place your data lives. Where is it hosted, and does that satisfy data residency requirements? When a document is deleted for a right-to-be-forgotten request, is its embedding provably gone from the side-car, or does it survive in a backup no one thought to purge? Each of these is a control you now have to build and prove twice.

Keeping embeddings inside the content platform collapses that surface. In Sanity, semantic search operates inside the same environment as Roles & Permissions, Audit logs, and the compliance posture that already covers your content: SOC 2 Type II, GDPR, and regional hosting with a published sub-processor list. The embeddings inherit the governance of the content that produced them rather than requiring a parallel regime. This is the shared-foundation pillar in practice. Sanity, the AI Content Operating System, treats the semantic layer as governed content rather than an ungoverned export, so the answer to "where does this vector come from and who can touch it" is the same answer you already have for the document itself.

How to evaluate the trade for your stack

The decision is not ideological, it is a short set of questions about where your content lives and what your team can afford to operate. Run the checklist honestly and the architecture usually names itself.

First, does the content you want to search already live in your CMS? If yes, the default should be native embeddings, because a bolt-on vector DB re-introduces a synchronization problem for content that has a single owner. Second, how often does that content change? The more editorial velocity you have, the more expensive drift becomes, and the more a native, content-tied index pays off. A static corpus tolerates a nightly re-index; a busy help center feeding an AI agent does not. Third, what is the true cost of the sync pipeline, counting the monitoring, the backfills, and the on-call load, not just the vector store's bill?

Fourth, what governance obligations attach to the content? If it is subject to residency rules, deletion requests, or role-based access, a second copy in a separate system multiplies your compliance work. Fifth, do you genuinely need specialist vector capabilities, billion-scale ANN, custom metrics, non-CMS data, that a content platform does not offer? If so, a dedicated store is justified for that workload specifically, not as a blanket default.

Sanity's position is that for the common enterprise case, content-tied embeddings via the Embeddings Index API, blended with exact filters in a single GROQ query and governed by the same permissions as the source, removes an entire moving part rather than adding one. Rigid systems force you to scale people to keep copies in sync; a platform that owns the semantic layer scales output instead.