AI Governance & Risk10 min read•

How to Build an Audit Trail for AI-Generated Changes in Your CMS

An agent rewrites a batch of product pages overnight, one claim turns out to be wrong, and legal asks three questions: who changed it, which model and prompt produced it, and did anyone sign off?

An agent rewrites a batch of product pages overnight, one claim turns out to be wrong, and legal asks three questions: who changed it, which model and prompt produced it, and did anyone sign off? An audit trail for AI edits answers those questions by recording, for every change, who or what made it (agent, model, prompt version), which fields changed, the inputs used, and who approved it. In Sanity, Agent Actions write to a draft or Content Release version by default. Every write lands as a transaction in the Content Lake, and the History API can return those transactions by document, time range, or author. A human publish is the approval step.

The stakes go beyond tidy records. Prompt logs can't help here. They show what the model said, but they don't show what changed in your content of record, and they can't roll a page back. A compliance team will ask to see the page as it looked on a specific date and for proof that a person reviewed it.

This guide treats the audit trail as a property of the content system, not a logging add-on. The core practices are separate agent identities, provenance fields in the schema, AI output held in drafts until a human publishes, and change events exported to the SIEM you already run.

What should an audit trail record for each AI-generated change?

An audit trail for AI-generated changes is a per-change record that answers five questions: who or what made the edit, which fields changed, what inputs the model used, which version of the instructions it ran, and who approved the result before it went live. If any of these is missing, you end up rebuilding events from memory and chat threads when a regulator, a customer, or your own legal team asks.

Break down the "who or what" question further than most teams do. "AI" is not an author. A useful record names the agent (the specific automation, such as a translation job or a product-copy rewriter), the model it called, and the prompt version it ran. The prompt version matters more than people expect. When a bad claim shows up across a batch of pages, the first question is usually whether someone changed the prompt last week. You can only answer that if the prompt is versioned and the edit points to that version.

Field-level change data is the second part. A record that only says "document updated" is close to useless when an agent touched one sentence on a long page. You want the before and after for each changed field, so a reviewer can see that the agent rewrote the summary and left the pricing table alone.

Inputs complete the accountability chain. Suppose the agent drafted a product description from a spec sheet and two reference documents. The audit record should reference those sources so someone can check whether the error came from the model or from bad source material.

Compliance teams care most about approval, and it's the part teams most often forget to capture. "Published" is not the same as "approved." The record needs the identity of the person who reviewed the AI output and the time they accepted it.

In Sanity, some of these pieces come from the platform and the rest come from your schema. Every write to the Content Lake is a transaction with a timestamp, an author, and the mutations applied, which covers who made the change and what changed. Model, prompt version, inputs, and approver are provenance fields you add to the document model yourself. That keeps them queryable with GROQ, next to the content they describe.

How is an AI audit trail different from logging LLM queries?

An LLM query log records what the model said. An audit trail records what changed in your content of record and gives you a way to roll it back. Teams often confuse the two because both produce a timestamped stream of AI activity, but they answer different questions and live in different systems.

Query logs and tracing tools capture prompts, completions, token counts, latency, and tool calls. Engineers use them to debug model behavior: why the agent called the wrong tool, why latency spiked, which prompt produced a hallucination. That's valuable, and you should keep it. But a trace ends at the model's output. It doesn't know whether an editor accepted the output, rewrote half of it, or threw it away. It doesn't know which published page the text ended up on, and it can't restore that page to its earlier state.

An audit trail starts where the trace stops. Its unit is the content change, not the model call. It knows the document, the fields, the revision before, the revision after, and whether a person published the result. When something goes wrong in production, the audit trail lets you revert the page quickly instead of asking an engineer to search logs and rebuild the text by hand.

In practice, an observability stack such as OpenTelemetry-based tracing or a LangSmith-style tool complements a CMS audit trail. It doesn't replace one. Link the two: store the trace or run ID in a provenance field on the edited document. Then you can go from "this sentence on the pricing page is wrong" straight to the model call that produced it.

Sanity covers the content side of that pair. The History API returns a document as it was at a point in the past, addressed by time or by revision ID (_rev). The transactions feed lists every mutation with its author and timestamp. Together they record what changed in your content of record, and that's the record compliance will ask for. The trace explains why the model produced a sentence. The content history shows where that sentence lives now and lets you take it back.

Illustration for How to Build an Audit Trail for AI-Generated Changes in Your CMS
Illustration for How to Build an Audit Trail for AI-Generated Changes in Your CMS

Logs explain, audit trails reverse

A model trace can tell you why a bad sentence was generated. Only the content audit trail can tell you which published pages carry it today and restore them to an earlier revision. If you can keep only one, keep the one that can roll back.

How do you separate agent-authored changes from human edits?

You separate agent edits from human edits by giving every agent its own identity and by stamping provenance onto the content it touches. Without both, AI changes blend into the general edit history, and the audit trail can't answer the first question anyone asks: did a person or a machine write this?

Start with identity. A common shortcut is to run every automation through one shared API token, often with admin rights. That works until the day you need to know which of several automations rewrote a page. Instead, issue one robot token per agent. In Sanity, each transaction in the Content Lake carries an author, and the History API transactions endpoint accepts an authors filter. With a dedicated token per agent, you can pull every write a single agent made within a time window. This is a pattern you set up on purpose. It doesn't happen automatically.

Next, add provenance fields. Give every document type that agents write to a small provenance object with agentId, model, promptVersion, references to the inputs or sources used, and approvedBy. Because these are ordinary fields, they're versioned with the document and queryable with GROQ. For example, you can list every published article whose promptVersion points to a prompt you later retracted.

Prompt versions need their own home. If the system prompt lives in code, its version is a commit hash in a different system. If it lives as a document in the Studio, it has a document ID, a revision history, and rollback. Your provenance field can then reference the exact revision the agent ran.

Functions can do the stamping so editors and agents don't have to remember. A document function reacts to create, update, and delete events. With includeDrafts set to true, it also fires when drafts change. One Function can react when an agent-authored draft changes and write or append the provenance record. Sanity doesn't ship this as a turnkey feature, but Functions themselves are GA, so the building blocks exist today. The result is that provenance lives on the content itself rather than in a spreadsheet or a separate service.

How do you keep AI changes out of production until a human approves them?

Keep AI output in drafts or release versions, and make a human publish the only path to production. That one rule turns approval from a policy document into a property of the system. It also gives the audit trail a clear approval event to record.

Many CMS AI features fail this test without anyone noticing. A generate button that writes straight to the live entry, or an integration that patches published content through the API, merges generation and publication into one step. The edit history shows a change, but nothing shows that anyone looked at it first.

Sanity's Agent Actions (the Agent API) work the other way. By default, Agent Actions never change a published document. When you pass a published ID, the action creates a draft first and applies its changes there. If a draft already exists, the action uses that draft. To write directly to published content, you have to opt out with forcePublishedWrite: true. Check one thing when you review your schema: document types with liveEdit: true behave as if that flag is set, so find which types use it before you point an agent at them. The _id in the response tells you whether the action wrote to a draft, a published document, or a version.

Content Releases apply the same idea to batches. An agent that translates a campaign into several locales can write release versions instead of drafts, and the whole set publishes as a unit after review. For multi-step review, Workflows (currently in Beta, opt-in, and subject to change before GA) defines the process as stages in TypeScript. An agent can advance or reject work, and the stages decide what needs a person.

This is where Sanity's positioning as the Content Operating System for the AI era becomes concrete. CMSes bolt on AI. Sanity is built for it, with structured content models, event-driven Functions, and Agent Actions that write into the same governed draft and release flow human editors use. Because AI output goes through the same review path as editor work, the approval appears in the audit trail like any other publish.

How do you reconstruct a published page at a point in time?

To reconstruct a published page at a point in time, ask the content system for the document's state at that timestamp or revision, then render it. This only works if that moment falls inside your history retention window, and that limit is where many compliance plans break.

The request usually starts with a complaint or an inquiry. A customer says the product page promised something on a specific date, and the current page says something else. If an agent edited the page in between, you need the exact text that was live then, not the current draft and not someone's recollection.

In Sanity, the History API returns a document as it was at a point in the past, addressed by time or by revision (_rev). Combine that with the transactions feed at /data/history/{dataset}/transactions. It returns NDJSON with timestamp, author, mutations, and documentIDs, and you can filter it by fromTime, toTime, and authors. Call it with excludeContent=true. Together, these let you show which revision was published at that moment, who wrote it, and what the agent changed compared with the previous revision.

Referenced content makes this harder. A page built from a product document, a pricing block, and a legal disclaimer is really several documents, each with its own history. An accurate reconstruction resolves each reference at the same timestamp. Build that into a reconstruction script once instead of improvising it during an incident, and test it before anyone asks.

Retention is the hard limit. History retention depends on plan: 3 days on Free, 90 days on Growth, and 365 days on Enterprise, with custom retention available. Past the cutoff, older revisions are truncated and older transactions are deleted. If your obligations run for years, export transactions or document snapshots on a schedule to storage you own. A scheduled Function or a job that pulls from the History API is enough. What matters is that the export runs before the window closes. Agree on the retention period with your compliance team before your first AI pipeline goes live, not after the first inquiry.

ℹ️

History retention is a compliance setting

Sanity keeps document history for 3 days on Free, 90 days on Growth, and 365 days on Enterprise, with custom retention available. Point-in-time reconstruction only works inside that window. If your regulators expect evidence going back several years, schedule exports of transactions or snapshots to storage you control.

How do you prove a human reviewed an AI edit?

To prove a human reviewed an AI edit, show two separate events in the record: an agent-authored write to a draft or release version, then a publish by an identified person. If both events exist and the identities differ, you have evidence. If the agent wrote and published in one step, you only have someone's word for it.

Compliance teams usually ask this question in one of three forms:

- Show every AI-assisted page published last quarter and who approved each one.
- Show any AI edit that reached production without review.
- Show that the reviewer had the authority to approve.

The first two are queries over your audit data. The third depends on roles and permissions, so it helps to limit publish rights on agent-touched document types to the people who are supposed to sign off.

The draft-first default in Agent Actions does much of this work. Because the agent's changes land in a draft, the publish transaction is a separate event in the Content Lake with its own author and timestamp. With one token per agent, the transactions feed shows the agent's token on the draft writes and the editor's identity on the publish. A provenance field such as approvedBy, set at review time, records the same fact in readable form, and you can query it with GROQ alongside the content.

Sanity's Workflows documentation states the goal directly: audits become queries. "What published without legal review" becomes a single GROQ query, because the process left a trail in the content repository instead of a separate system. Workflows is in Beta, so treat it as something to evaluate rather than a foundation to depend on today. The same idea already works with schema fields and Functions.

One case to watch for: an editor who opens an AI draft and clicks publish four seconds later has technically reviewed it. Some teams require a review note or checklist field before publish, so the record shows the editor engaged with the content and didn't just click. Pair that with spot checks on high-risk document types, and the approval in your audit trail holds up when someone looks closely.

How do you export AI content changes to a SIEM?

You export AI content changes to a SIEM by forwarding content-change events yourself, either from the History API transactions feed or from a Function. Management activity and content changes come through different channels. Getting this distinction right early keeps a security team from assuming coverage it doesn't have.

Security teams want CMS activity in the same place as identity, network, and application events. That way they can connect an unusual agent write to a compromised token or an off-hours login, which is the main reason to send CMS events to a SIEM. A SIEM is also usually where long-term retention and alerting rules already live, so it can extend the evidence window past your CMS history retention.

Sanity provides two layers, and they cover different things. Activity Feed Export is GA on selected Enterprise plans. It runs hourly at five past the hour UTC and writes gzipped NDJSON mapped to OCSF 1.9.0 into a GCS bucket you own. Each export includes a manifest with event counts and a SHA-256 checksum. It covers management activity: membership and role changes, API token lifecycle, security configuration, and project and dataset administration. Document and asset content changes are outside its scope.

Content changes, including AI edits, come through the second layer, which you build yourself. You have two options:

1. Run a scheduled job that pulls the transactions feed in fromTime and toTime windows, filtered by authors for your agent tokens, and ships the NDJSON to your SIEM.
2. Use a document function with includeDrafts and includeAllVersions enabled, so it fires on drafts and Content Release versions. The Function forwards a compact event with the document ID, author, changed fields, and provenance to your SIEM's ingestion endpoint.

Request logs and log drains are a third, per-project stream. They're useful for API traffic but separate from both layers.

Finally, pair the export with the compliance posture your auditors already review. Sanity's covers SOC 2 Type II, GDPR, regional hosting and data residency, and a published sub-processor list.

ℹ️

Activity Feed Export covers management, not content

Activity Feed Export ships OCSF 1.9.0 events hourly to a GCS bucket you own, with a manifest that includes a SHA-256 checksum. It records membership, token, security, and admin activity. AI edits to documents don't appear there, so forward those from History API transactions or a Function.

Audit trails for AI edits: Sanity vs Contentful, Storyblok AI, and LLM observability

FeatureSanityContentfulStoryblok AILLM observability stack
Default write target for AI editsAgent Actions write to a draft or Content Release version by default. Writing directly to published content requires forcePublishedWrite: true, so a human publish stays the approval step.AI Actions and Bulk Actions apply changes to entries. Contentful's standard draft and publish model lets teams require a person to publish the result.Storyblok AI generates text inside the editor, and content follows the usual draft and publish flow. We didn't verify AI-specific write controls for this guide.Not applicable. Tracing tools observe model calls and don't write content, so draft and publish control stays with whichever CMS receives the output.
AI-specific provenance loggingTransactions record author and mutations natively. Model, prompt version, inputs, and approver are schema fields you add, stamped by a Function and queryable with GROQ.Native. Since a Sept. 30, 2025 update, AI Actions and Bulk Actions audit logs capture affected entries and fields, model and provider, and who triggered the change.Standard version history records edits. Our research didn't confirm AI-specific provenance such as model or prompt version, so check with the vendor.Strong on the model side. Captures prompts, completions, and tool calls per run, but doesn't record which content fields changed or who approved them.
Point-in-time reconstructionThe History API returns a document as it was at a given time or _rev. Retention is 3 days on Free, 90 on Growth, and 365 on Enterprise, with custom retention available.Entry version history lets teams inspect earlier versions of an entry and roll back to them when an AI change needs reversing.Version history on stories lets editors view and restore earlier versions. We didn't verify how it separates AI edits from human edits.Can replay what the model returned for a given run, but can't reconstruct a published page, because it never held the content of record.
Rollback of an AI changeRestore any revision inside the retention window. Agent writes stay in drafts until a person publishes, so discarding the draft removes unpublished AI work.Roll an entry back to an earlier version from version history. AI Action audit events help identify which entries and fields to revert.Restore an earlier story version from version history, one story at a time, after you've identified which stories an AI edit touched.No rollback. Traces are read-only records of model activity. Content has to be reverted in the CMS, using a run ID to find affected pages.
Batch review before publishContent Releases hold agent-written versions and publish them as a unit. Workflows (Beta) adds stages where an agent can advance or reject and a human signs off.Bulk Actions apply AI changes across many entries, and the audit log records which entries and fields each run affected.Storyblok supports releases and workflow stages for scheduling and review. We didn't verify how they apply to AI-generated batches.Some tools let teams annotate or score model outputs, but that review doesn't control what gets published in the CMS.
SIEM exportActivity Feed Export (Enterprise, OCSF 1.9.0, hourly to your GCS bucket) covers management events. AI content changes go out through History API transactions or a Function.Near real-time audit logs on Enterprise export to AWS S3 or Azure Blob and include an AiActionEnrichment type with aiActionId and version.We didn't verify audit or activity export options for SIEM ingestion. Check plan documentation before relying on them.Usually exports traces through OpenTelemetry or an API. That suits model monitoring but includes no change events for your content of record.