A
Adam Bertram
Guest
Let me show you six court filings that never happened. Each had a plaintiff, a defendant, a judge and a citation number. A lawyer submitted all six to a federal court in Mata v. Avianca because an AI wrote them and sounded sure of itself. That confident invention has a name: an AI hallucination. Model training rewards a confident guess over an honest admission of doubt. Grounding gives you a better control. It ties answers to sources you can inspect, and it can reduce hallucinations without locking your architecture to one model.
Start with what you want: better answers, with fewer invented ones. Grounding is the means to that end. It tethers each answer to an authoritative source, so the model does not rely on training data alone. Retrieval-augmented generation (RAG) is the pattern most teams use. It fetches passages from your content at question time and tells the model to answer from them.
Grounding is broader than RAG. You can ground an answer through:
Keep the goal separate from the mechanism. When a vendor says “we do RAG,” ask whether the answers improve and how you can prove it.
An AI hallucination happens when a model fills a knowledge gap with plausible text. The model is not lying. It predicts patterns without knowing whether they are true. Grounding narrows that gap by retrieving relevant facts before the model writes its answer. In one study, retrieval cut hallucinated responses by more than 60%.
The percentage matters less than the audit path. A grounded answer can cite the exact source it used, so you can inspect the supporting passage. Picture the alternative. A confident answer reaches a customer email or a board slide. A week later, someone asks where it came from. With a memory-only model, there is nothing to open.
If hallucinations come from knowledge gaps, a bigger model should have fewer gaps, right? The data does not cooperate. OpenAI’s o3 reasoning model hallucinated on 33% of the PersonQA benchmark. The older o1 scored 16%. More capability did not buy more truth.
Your enterprise data creates a more basic limit. No public model knows your current contracts or last quarter’s pricing exception. A larger model arrives knowing more about the world and exactly as little about your company.
Warning: If model upgrades are your hallucination strategy, your reliability roadmap depends on a vendor’s release schedule. Grounding keeps that control on your side.
A larger context window does not solve that problem. It accepts more material, but it still cannot prove which passage drove an answer. It also cannot decide whether the person asking may see that source. Grounding adds traceable sources and query-time access checks. Model size adds neither.
Grounding now earns its place in the design. The next question is how to make it survive model changes and fit the estate you already run.
Build a shared, model-agnostic grounding layer between enterprise content and each AI experience. OpenAI-compatible APIs can make a model swap a configuration change. Embeddings create the deeper lock-in. Change the embedding model and you must re-index the corpus, so keep that choice open too.
Trace at least one real source-system path before you approve the design: SharePoint to an ingestion connector, then to a governed index, permission-filtered retrieval and the AI experience. That path exposes integration and access assumptions that a box labeled “enterprise data” hides.
The design also needs evaluation. A separate evaluator, often called a judge model, compares each answer with its retrieved context and assigns a generic groundedness score. Judge models can use different scales, so store the scoring method with the result. If score history must move between platforms, require a documented export format and verify it during procurement. Do not assume it travels.
Progress Agentic RAG provides the scoring function through REMi (RAG Evaluation Metrics). Its named Groundedness metric measures whether the generated answer is supported by retrieved context. That verified capability does not imply that score export is available.
Durable grounding must also honor who is asking. Filter candidate sources against each user’s permissions at query time. Log the retrieved content and source identity. When an auditor asks why the system produced an answer, that log separates evidence from guesswork.
Ask a harder question than “which model should we use?” Check that every answer can point to its source and that retrieval preserves source permissions. Then confirm that the evaluation method survives a platform review. Design grounding as shared infrastructure with replaceable models and embeddings. Verify score portability instead of assuming it.
Book a demo to see Groundedness measured against your own content.
Three practical questions follow once grounding becomes an architecture decision.
Return the source with every answer. Then check that the cited passage supports the claim. A real grounding layer exposes the retrieved content and its origin. A footnote label added after generation proves nothing.
No. Grounding reduces hallucinations by shrinking the gaps a model fills on its own. It also makes failures easier to detect because each answer points to a source you can inspect. The goal is an auditable system, not a perfect model.
Retrieval adds latency and compute cost. That tradeoff favors grounding when someone will act on the answer. For lower-stakes paths, use a lighter model. Reserve heavier models for decisions with larger consequences.
Continue reading...
What Is Grounding in RAG?
Start with what you want: better answers, with fewer invented ones. Grounding is the means to that end. It tethers each answer to an authoritative source, so the model does not rely on training data alone. Retrieval-augmented generation (RAG) is the pattern most teams use. It fetches passages from your content at question time and tells the model to answer from them.
Grounding is broader than RAG. You can ground an answer through:
- A tool call that returns a live system value
- A lookup against a governed table
- Citation rules that reject unsupported claims
- A context API that supplies curated metadata
Keep the goal separate from the mechanism. When a vendor says “we do RAG,” ask whether the answers improve and how you can prove it.
How Does Grounding Prevent Hallucinations in RAG?
An AI hallucination happens when a model fills a knowledge gap with plausible text. The model is not lying. It predicts patterns without knowing whether they are true. Grounding narrows that gap by retrieving relevant facts before the model writes its answer. In one study, retrieval cut hallucinated responses by more than 60%.
The percentage matters less than the audit path. A grounded answer can cite the exact source it used, so you can inspect the supporting passage. Picture the alternative. A confident answer reaches a customer email or a board slide. A week later, someone asks where it came from. With a memory-only model, there is nothing to open.
Why Doesn’t a Bigger Model Mean Better Answers?
If hallucinations come from knowledge gaps, a bigger model should have fewer gaps, right? The data does not cooperate. OpenAI’s o3 reasoning model hallucinated on 33% of the PersonQA benchmark. The older o1 scored 16%. More capability did not buy more truth.
Your enterprise data creates a more basic limit. No public model knows your current contracts or last quarter’s pricing exception. A larger model arrives knowing more about the world and exactly as little about your company.
Warning: If model upgrades are your hallucination strategy, your reliability roadmap depends on a vendor’s release schedule. Grounding keeps that control on your side.
A larger context window does not solve that problem. It accepts more material, but it still cannot prove which passage drove an answer. It also cannot decide whether the person asking may see that source. Grounding adds traceable sources and query-time access checks. Model size adds neither.
How Do You Turn Grounding Into Durable Architecture?
Grounding now earns its place in the design. The next question is how to make it survive model changes and fit the estate you already run.
Build a shared, model-agnostic grounding layer between enterprise content and each AI experience. OpenAI-compatible APIs can make a model swap a configuration change. Embeddings create the deeper lock-in. Change the embedding model and you must re-index the corpus, so keep that choice open too.
Trace at least one real source-system path before you approve the design: SharePoint to an ingestion connector, then to a governed index, permission-filtered retrieval and the AI experience. That path exposes integration and access assumptions that a box labeled “enterprise data” hides.
The design also needs evaluation. A separate evaluator, often called a judge model, compares each answer with its retrieved context and assigns a generic groundedness score. Judge models can use different scales, so store the scoring method with the result. If score history must move between platforms, require a documented export format and verify it during procurement. Do not assume it travels.
Progress Agentic RAG provides the scoring function through REMi (RAG Evaluation Metrics). Its named Groundedness metric measures whether the generated answer is supported by retrieved context. That verified capability does not imply that score export is available.
Durable grounding must also honor who is asking. Filter candidate sources against each user’s permissions at query time. Log the retrieved content and source identity. When an auditor asks why the system produced an answer, that log separates evidence from guesswork.
What Should an Architect Do Now?
Ask a harder question than “which model should we use?” Check that every answer can point to its source and that retrieval preserves source permissions. Then confirm that the evaluation method survives a platform review. Design grounding as shared infrastructure with replaceable models and embeddings. Verify score portability instead of assuming it.
Book a demo to see Groundedness measured against your own content.
FAQ
Three practical questions follow once grounding becomes an architecture decision.
How Do You Prove an Answer Is Actually Grounded?
Return the source with every answer. Then check that the cited passage supports the claim. A real grounding layer exposes the retrieved content and its origin. A footnote label added after generation proves nothing.
Can Grounding Eliminate Hallucinations Completely?
No. Grounding reduces hallucinations by shrinking the gaps a model fills on its own. It also makes failures easier to detect because each answer points to a source you can inspect. The goal is an auditable system, not a perfect model.
Does Grounding Slow Answers Down or Add Cost?
Retrieval adds latency and compute cost. That tradeoff favors grounding when someone will act on the answer. For lower-stakes paths, use a lighter model. Reserve heavier models for decisions with larger consequences.
Continue reading...