How to Measure RAG Groundedness (2026)
A retrieval-augmented agent can produce an answer that is fluent, relevant, well-cited — and still made up. Groundedness is the metric that catches this: it asks whether every claim in the answer is actually supported by the documents the agent retrieved. In domains where a confident fabrication costs real money or real harm, it is the metric that matters most.
Quick answer: Groundedness measures whether every claim in an answer is supported by the retrieved sources. Score it by decomposing the answer into atomic claims and verifying each one against the retrieved context, then reporting the proportion supported. Claim-level scoring localises the failure; a single answer-level pass/fail hides which sentence fabricated.
What is groundedness, and what does it not measure?
Groundedness is a faithfulness metric: given a retrieved context and a generated answer, how much of the answer is entailed by that context? It deliberately says nothing about whether the answer is true in the world, or whether it addresses the question. Those are different metrics with different failure modes:
- Groundedness — is the answer supported by the sources?
- Answer relevance — does the answer address the user's question?
- Retrieval quality — did we fetch the right documents in the first place?
- Citation accuracy — do the specific citations point to passages that support the claim? (See how to measure citation accuracy.)
Keeping these separate is not pedantry. An agent can be perfectly grounded in the wrong documents, or highly relevant while inventing details. If you collapse them into one score, you will spend weeks fixing the wrong component.
What are the failure modes worth catching?
The dangerous ones are the confident, plausible failures:
- Unsupported elaboration — the sources say a drug reduces risk; the answer adds a dosage the sources never mention.
- Fluent extrapolation — the answer stitches two source facts into a causal claim neither supports.
- Fabrication with real-looking citations — the answer cites a real document, but the cited passage doesn't say what the answer claims.
- Confident guessing on empty retrieval — nothing useful came back and the agent answers anyway.
The last one is why abstention belongs in the same eval set: an agent that never says "I don't know" will score well on happy-path cases and fail exactly where it matters — see how to test RAG agent abstention.
How do you score groundedness in practice?
The workable approach is claim decomposition:
- Split the answer into atomic claims — each independently verifiable, one assertion per claim.
- Verify each claim against the retrieved context — is it supported, contradicted, or absent?
- Score the proportion supported, and report which claims failed.
Claim-level output is the whole point. "This answer scored 0.6" tells you nothing actionable; "these two claims were unsupported" tells you what to fix. Use an LLM judge for the entailment step where the language is free-form, but validate the judge against human labels periodically — an unvalidated judge drifts, and a drifting judge quietly corrupts every number downstream.
How do you build a golden set that catches real failures?
Mix real production queries with deliberately hard cases. Your set needs, at minimum: questions the corpus fully answers (the baseline), questions it partially answers (where over-claiming shows up), questions it doesn't answer at all (where abstention shows up), and questions where two sources conflict (where fabricated synthesis shows up). A set built only from questions the corpus answers well measures almost nothing.
Keep the set out of public CI logs and blog posts. A groundedness eval that leaks stops discriminating once models have seen it — see why AI agent evals go stale.
How do you know your groundedness eval actually works?
Run the candidate eval against agents whose quality you already know before you trust its verdicts on an agent you don't. A known-good agent (grounded, cites correctly, abstains appropriately) should score high; a known-broken one (loose citation discipline, occasional extrapolation) should score visibly lower; a known-sabotaged one (fabricates citations on purpose) should score near zero. If the eval can't separate those three, it isn't measuring groundedness — it's measuring something correlated with it, like answer length. See the RAG groundedness benchmark for this discriminating-power check on a real reference panel.
How do you gate it in CI?
Treat groundedness like a unit test on your retrieval pipeline. Any change to prompts, model version, chunking strategy, embedding model, or retrieval config triggers a run against the golden set; a regression past your threshold blocks the merge. Track groundedness and abstention rate as separate lines over time — a rise in groundedness paired with a collapse in abstention usually means the agent got more cautious about wording, not more honest.
Related
- How to test RAG agent abstention — the other half of the same problem.
- How to measure citation accuracy — the pointer-level metric that pairs with groundedness.
- RAG groundedness benchmark — how these evals score against known-good, broken, and sabotaged reference agents.
- General-knowledge RAG groundedness pack · Medical · Legal
- Capability packs — the full category.
FAQ
Groundedness measures whether every claim in an answer is supported by the retrieved source documents. An answer can be fluent, relevant, and still ungrounded if it asserts things the sources never said.
Decompose the answer into atomic claims, then verify each claim against the retrieved context and score the proportion that is supported. Claim-level scoring localises the failure; a single answer-level pass/fail hides which sentence fabricated.
Groundedness asks whether the answer is supported by the sources. Answer relevance asks whether it addresses the question. An answer can be perfectly grounded and useless, or highly relevant and fabricated — you need both metrics.
Yes. In high-stakes domains, abstaining is the correct behaviour when retrieval returns nothing useful. Include no-evidence cases in your eval set and score abstention explicitly, or you reward confident guessing.
No, and separating them matters. If retrieval returned the wrong documents, the generator may be perfectly grounded in bad context. Measure retrieval quality and groundedness separately or you will fix the wrong component.
Yes. Gate every change to prompts, models, chunking, or retrieval config against a golden set, and block merges that regress past your threshold. Keep the case set private so it does not decay through public CI logs.