In-App Help Bots That Answer Instead of Linking

A user types "does the Pro plan support SSO?" into your in-app help bubble and gets back three links to a docs index, a pricing page, and a changelog. They wanted a yes or a no. That gap, between a question with a structural answer and a bot that can only hand back documents, is where support tickets get filed anyway and where trust in the bot dies after the second try.

The reason most help bots link instead of answer is not the language model. It is retrieval. A question like "SSO on the Pro plan" carries a structural component that pure vector similarity cannot resolve, so the bot either returns nothing useful or hallucinates a confident wrong answer. This guide reframes the in-app help bot as a retrieval problem grounded in structured content, and shows how Sanity Context, through its Context MCP surface, lets an agent query your content backend directly instead of guessing over a copied index.

Why do help bots link instead of answering the question?

Help bots link instead of answer because retrieval, not the model, is where most in-product agents fail. The failure has a consistent shape. A user asks something with a structural component baked in, a version number, a plan tier, a category, or 'in stock', and pure vector similarity cannot resolve that component. Semantic search finds passages that read like the question, but it has no way to enforce that the plan is actually Pro or the item is actually in stock. So the retrieval step comes back empty or comes back with near-misses, and the model does the only two things it can: it hallucinates a confident answer, or it hedges by handing over links and letting the human sort it out. The links are the bot admitting it could not retrieve the fact.

This is worth sitting with because the industry reflex is to blame the model and reach for a bigger one. That rarely helps. When Sanity ran its schema exploration against Sonos's catalog, an honestly difficult dataset, it landed around 83% accuracy on a mix of question difficulties using Sonnet 4.5, at roughly 40 seconds of thinking per hard question. That number only arrived after teaching retrieval about counter-intuitive field names, second-order reference chains the schema does not connect, and data-quality issues the schema cannot reveal. None of that was a model problem. It was a context problem. The lesson generalizes: a help bot that answers is a help bot whose retrieval layer understands the shape of your content, so it can apply the exact filters a question demands before it ever ranks anything by meaning.

What makes structured content the difference between answering and linking?

Structured content is what lets a help bot filter to the exact answer before it ranks by relevance, and that ordering is the whole game. Here is the concrete, citable reason structure still matters in a semantic world: in Sanity, `text::semanticSimilarity()` is only valid as an argument to `score()`. Semantic search ranks, it does not filter. The correct pattern is to narrow the candidate set with a GROQ filter first, then rank what is left by meaning. A question about SSO on the Pro plan becomes a filter on the plan tier and the feature reference, which returns a small, correct candidate set, and only then does similarity decide which of those documents best answers the phrasing.

Unstructured knowledge bases cannot do this. When your content lives as free-text pages, as it does in a Notion or Confluence workspace, there is no queryable field for 'plan == Pro', so the exact structural filters that questions actually carry are out of reach. You are left with similarity over prose and the hallucination pattern that follows. This is why 'answers instead of linking' is a content-model question before it is a bot question. Content Lake stores your content as structured, queryable data, so an agent connecting through the Context MCP endpoint can run GROQ mode: it queries the dataset at request time, applies exact filters over structured documents, and needs no separate build step. The prerequisite is a deployed schema for the project and dataset, via `sanity schema deploy` on Studio v5.1.0 or later, which is what tells the agent where to look. Structure is not overhead here. It is the mechanism that turns a question with a hard constraint into a query with a correct answer.

GROQ mode or Knowledge Base mode: which retrieval mode should an answering bot use?

The right retrieval mode depends on where the difficulty in the question lives. Context MCP offers two, and choosing between them is the first real engineering decision, not an afterthought. Use GROQ mode when your content is structured and consistent and the schema tells the agent where to look. It queries the dataset at request time with no build step, which is ideal for a product catalog or a plan-and-feature matrix where the answer is a filter away. For a catalog where the useful detail sits in prose fields, enable dataset embeddings and stay in GROQ mode, so a single query combines structured filters with semantic similarity over the descriptions.

Use Knowledge Base mode when locating the answer is itself the hard part, when the fact could be in a product spec, a compliance memo, or a support article and the job is finding which one holds it. A Knowledge Base is a pre-built index over sources you choose, dataset prose, website subfolders, PDFs, and other files, served to agents through Context MCP. Note that Knowledge Bases is Beta, opt-in, and enabled by an org admin from the Apps page in Manage; a build reads the sources ahead of time, resolves conflicts, and writes entries with citations. The pre-generated Outline, every entry path plus a one-line summary, is served by the `initial_context` tool so the agent knows what exists before it searches. One trap to design around: attach both a dataset source and Knowledge Base sources to a single MCP and the dataset source wins silently, with the Knowledge Bases ignored and no error raised. Pick the mode that matches where the hard part of the question actually sits, and keep those sources on separate endpoints.

How does the bot return an answer without garbling the facts?

The bot returns a reliable answer by having its tools return schema-shaped objects, not prose, so the model passes structured facts straight through instead of paraphrasing them. This is a small design choice with outsized consequences. A tool that returns prose forces the model to re-narrate it, and paraphrasing is where facts go to die. When Sanity watched agents get built against Context MCP, the ones that worked returned schema-shaped responses the model could pass straight through. The ones that struggled got a wall of text back and re-narrated it, badly, dropping a price here, softening a hard 'no SSO on this tier' there. If the answer is three products, the tool should return three product objects, not a paragraph describing them.

That discipline also unlocks a real product surface rather than a chat log. The emerging MCP-UI spec and the Vercel AI SDK's generative UI primitives let a tool return a rendered component directly, so the answer to 'which trail runners under $150 like a Hoka' can be three product cards the user can act on, and the answer to an order question can be an order card rather than a sentence. The structured object is both the safeguard against the model corrupting the fact and the raw material for a component the user can click. Design the tool contract around the shape of the answer, three product objects, one plan-comparison object, a single boolean with a citation, and the model's job shrinks to routing correct data into a correct surface, which is exactly the job you want it doing.

Where should the answering bot live, and what changes when it moves?

Channel choice governs the engineering before anything else, so decide it first. In-product chat, the bottom-right bubble, suits support, FAQ, and light navigation. It is conversation-shaped, ephemeral, and anchored to a web session, and it inherits the logged-in user's identity, which is the constraint that matters most: the strength of the identity caps the strength of the action the bot can safely take. A bot in a low-trust, lightly-authenticated channel should be able to answer and navigate, not mutate account state. Vipps runs an in-product chat agent against their FAQ content, a straightforward answering surface grounded in the content they already maintain.

What stays portable when the bot moves to Slack, or to a proactive surface, is the part you invested in: the content backend, the retrieval layer, the tool definitions, the system prompt, and the eval bench. Context MCP exposes the same tools whichever surface the user touches, so the retrieval you tuned for the web bubble is the retrieval a Slack bot uses too. The thing that must not travel unchanged is the tool registry's permissions. A global registry once leaked a write tool into a low-trust channel, exactly the identity-versus-action mismatch that channel choice is supposed to prevent. Scope tools to the channel's trust level, not the union of everything the bot might ever do. Answering bots are the safe starting point precisely because answering, unlike acting, does not need a strong identity to be safe, which is why 'answers instead of linking' is a sensible first agent to ship even in a lightly-authenticated in-product surface.

How do you keep the answers fresh and know the bot is actually working?

You keep answers fresh by wiring retrieval into the content backend, and you learn whether the bot works by scoring conversations. Take freshness first. Hybrid retrieval in GROQ blends the structured predicates that must hold, a BM25 keyword match via `text::query()` weighted with `boost()`, and `text::semanticSimilarity()`, then orders by `_score`. What Content Lake handles that a bolted-on vector database does not is keeping that search index fresh: re-embedding on change, deletion handling, and backfill for schema changes. When retrieval lives in the content backend, freshness stops being something you maintain; when it is a separate vector DB plus glue code, freshness becomes a permanent line item on your roadmap, and a stale index is a bot that confidently answers with last quarter's pricing.

Drift between copies of a fact is the other freshness failure, and it is one a Knowledge Base build surfaces rather than hides. When a help center says returns are accepted within 30 days and a product page says 45, the build detects the conflict and raises an Issue showing both claims side by side with their sources, so a human picks ground truth instead of the bot picking at random. On knowing it works: you cannot tell whether an answering bot is succeeding without scoring the conversations. Run a model over the transcripts asynchronously to score each one, was this a success, what was the user trying to do, did retrieval return useful results or did the agent hallucinate, did it reach a tool it should not have. It is imperfect, and it is a hundred times better than no scoring. Together, index freshness in the backend and asynchronous conversation classification are what move a demo that answers into a bot you can trust in production.