How to Set Up Automated Evaluations for RAG and Agents (Tests in CI)

Your RAG pipeline passed every test on Friday. On Monday, a content editor renamed a field, someone re-chunked the knowledge base, and the model quietly started answering pricing questions with last quarter's numbers. Nobody noticed until a customer did. This is the failure mode that separates demo-grade AI from production AI: quality regressions are invisible until a human catches them in the wild, because most teams ship LLM features with no automated way to know when retrieval, grounding, or answer quality degrades.

Sanity is the AI Content Operating System, an intelligent backend built so that the content feeding your agents stays governed, versioned, and testable inside the editorial loop rather than drifting silently underneath your prompts. That framing matters here, because most eval advice treats the model as the only thing worth testing. The content layer is where the real regressions start.

This guide reframes evaluation as a CI discipline you already know: write assertions, run them on every change, block the merge when they fail. We will cover what to measure for RAG and agents, how to build a golden dataset, how to wire evals into your pipeline, and why the CMS, not just the model, belongs under test.

Why RAG and agents fail silently in production

Traditional software fails loudly. A null pointer throws, a test goes red, a page 500s. LLM systems fail quietly, and that is precisely what makes them dangerous to operate at scale. A retrieval step that returns the wrong chunk still returns something. A model that hallucinates a return policy still produces fluent, confident prose. There is no exception, no stack trace, no red build. The output looks exactly as plausible when it is wrong as when it is right.

The failure surface is also wider than most teams model. An agent answer can degrade for at least four independent reasons: the retriever pulled irrelevant context, the context was relevant but stale, the model was grounded correctly but reasoned poorly, or the underlying content itself changed and nobody re-evaluated. Each of these lives in a different part of the stack, and each demands a different assertion. Testing only the final answer tells you something broke without telling you where.

Consider the stalest of these: freshness. A support agent trained and evaluated in March against a knowledge base snapshot will happily keep citing that snapshot in June, even after the source articles have been rewritten twice. If your embeddings are computed by a separate batch job disconnected from your content, the index and the truth drift apart on their own schedule. This is why evaluation cannot be a one-time launch gate. It has to run continuously, on every content change and every code change, the same way unit tests run on every commit. The rest of this guide treats that as the baseline, not the aspiration.

What to measure: retrieval, grounding, and answer quality

Good RAG evaluation decomposes the pipeline and tests each stage, because a single end-to-end score hides which layer regressed. Start with retrieval. Given a query, did the system fetch the chunks that actually contain the answer? The standard metrics are context recall (did we retrieve everything we needed) and context precision (did we retrieve without drowning the answer in noise). You can compute both if your golden dataset labels which source documents are relevant to each question.

Next, grounding, sometimes called faithfulness. Does the generated answer stay inside the retrieved context, or does it invent claims the sources never made? This is the metric that catches hallucination directly. A common technique is to have a judge model extract each factual claim from the answer and check whether the retrieved context supports it. Answers that assert unsupported facts fail the grounding assertion even if they sound authoritative.

Finally, answer quality against the reference: relevance to the question, correctness versus a known-good answer, and completeness. For agents specifically, add trajectory checks. Did the agent call the right tools in a sensible order, and did it stop when it had enough information rather than looping? Structure helps enormously here. When your content is stored as Portable Text rather than a wall of HTML, the boundaries between blocks, marks, and annotations survive chunking and retrieval, so a judge can reason about what was actually retrieved instead of guessing at where one idea ended and the next began. The cleaner the structure going in, the sharper every downstream metric becomes.

Building a golden dataset that reflects real usage

Evaluation is only as honest as the dataset behind it. A suite of twenty questions a founder wrote in an afternoon will pass forever and protect nothing. The golden dataset is the contract that says what good looks like, so it deserves the same rigor as production code and the same lifecycle: versioned, reviewed, and owned.

Source your cases from reality, not imagination. Mine real user queries from logs, sample the questions your support team actually fields, and deliberately include the ugly ones: ambiguous phrasing, questions with no good answer, multi-hop questions that require combining two documents, and adversarial prompts that try to pull the agent off task. Each case needs a question, the set of source documents that should be retrieved, and a reference answer or a rubric describing an acceptable answer. Aim for coverage of your content's breadth rather than raw volume; a well-chosen hundred beats a careless thousand.

The hard part is keeping the dataset alive as your content changes, and this is where treating content as governed data pays off. When your reference material lives in a system with structured schemas, review workflows, and version history, you can trace exactly which content a test case depends on and flag cases for review when that content is edited. Content Releases let you stage a batch of content changes and evaluate the agent against them before they go live, so a rewrite of your pricing page gets tested against your golden dataset in a review environment rather than discovered in production. Governance and evaluation stop being separate disciplines and become the same review gate.

Wiring evaluations into CI so regressions block the merge

An eval suite that runs when someone remembers to run it is theater. The value comes from automation: every pull request, every content release, every scheduled interval, the suite runs and the result is a gate, not a suggestion. This is the move that turns evaluation from a research activity into an engineering discipline.

Mechanically, wire a job into your existing CI (GitHub Actions, GitLab CI, or whatever you run) that executes the eval harness against a fixed golden dataset and emits per-metric scores. Set thresholds: grounding must stay above your floor, context recall cannot drop more than a point or two from the main branch baseline, and no previously passing case may regress. When a threshold is breached, the job exits non-zero and the merge is blocked, exactly like a failing unit test. Because LLM outputs are stochastic, pin what you can (temperature, seeds, model version) and run enough samples that a single unlucky generation does not flip the build; assert on aggregates and small deltas rather than exact strings.

Code is only half the trigger, though. Content changes need to run the suite too. With Functions, you can hook evaluation into content lifecycle events so that publishing a knowledge base update kicks off the relevant test cases automatically, and the Live Content API means downstream workflows react the moment content changes rather than on a nightly batch. The result is a system where a bad edit to a source document is as likely to trip an alarm as a bad edit to your retrieval code.

Making the CMS a first-class participant, not a silent dependency

Most eval tooling stops at the model and the prompt, treating the content store as an inert bucket the retriever reads from. That is the gap this guide exists to close. In a real RAG or agent system, the content layer is an active source of regressions, and if it is not under test, your evaluation has a blind spot exactly where the most insidious failures live.

Sanity closes that gap by making content a governed, queryable, LLM-ready substrate rather than a passive datastore. Legacy CMSes stop at publishing; the intelligent backend operates content end to end, which is what evaluation actually requires. Embeddings tied to content through the Embeddings Index API and dataset embeddings mean your semantic index stays consistent with the source instead of drifting on a separate schedule, so freshness regressions largely stop happening by construction. Agent Actions give you schema-aware LLM operations (generate, transform, translate, validate) that run against the same typed content your tests assert on, so what you evaluate and what you ship are the same thing. And GROQ lets a test harness query exactly which documents exist, when they changed, and what a retriever should have seen, turning content state into something you can assert against directly.

The practical consequence is that governance and evaluation converge. Studio Workspaces, Content Releases, and version history give humans the review loop; Functions and the Live Content API give machines the trigger; the Content Lake gives both a single shared foundation instead of the silos legacy systems create. AI is wired into the data model, the editor, and the delivery layer, so the thing you test is the thing your agent reads.