AI hallucination prevention in customer support starts with one rule: the agent should only answer from what it can retrieve, and stay silent when it cannot. Grounding, retrieval design, and a willingness to say "I don't know" are more effective than any model upgrade. This article covers why support agents hallucinate and what you can do about it.
What is an AI hallucination in customer support?
A hallucination is when the AI generates a response that sounds correct but is not backed by your actual content. It might invent a return policy that does not exist, cite a shipping timeline you never published, or confidently fabricate a product feature. The answer reads well. It is just wrong.
In general-purpose chatbots, hallucinations are an annoyance. In customer support, they are a liability. A wrong answer about a refund window or a warranty claim can cost real money and break real trust. Unlike a search engine that returns "no results," a language model will almost always produce an answer, even when it has no basis for one.
The Vectara Hallucination Evaluation Framework tested leading models across 12,000 factual queries in 2026 and found that top models now hallucinate on fewer than 1% of general knowledge questions. That sounds reassuring, but general knowledge benchmarks are not your help centre. Your product details, your pricing, your policies exist nowhere in the model's training data. Without grounding, the model is guessing from patterns, not answering from facts.
Why does grounding alone not solve the problem?
Grounding connects the language model to your actual content through retrieval-augmented generation (RAG). Instead of answering from its training data, the model answers from passages retrieved from your knowledge base. This is the right architecture. But it is not a complete solution.
RAG systems have several failure modes that lead to hallucinations even with grounding in place:
- Retrieval misses. The right passage exists in your content, but the retrieval step does not find it. The model then answers from whatever passages it did retrieve, even if they are only tangentially related.
- Chunk boundary problems. If your content is split at arbitrary points, a definition might end up in one chunk and the details that support it in another. The model sees half the picture and fills the rest.
- Conflicting sources. Your FAQ says one thing. Your terms page says another. The model picks one, or worse, blends both into something neither source actually says.
- Over-confidence in thin evidence. The model finds a single sentence that partially relates to the question and builds a full paragraph around it, adding context that does not come from the source.
Research on retrieval systems confirms this pattern: retrieval failures lead the generator to "give incomplete answers, fabricate information to fill gaps, or abstain unnecessarily." Grounding reduces hallucinations, but it does not eliminate them.
How do you design content that retrieval can actually use?
The single biggest lever you have is your knowledge base. Most help centres were written for humans who bring context the page does not contain. They skim, they scroll past introductions, they infer meaning from screenshots. An AI agent does none of that. It works with whatever the retriever returns.
Content that prevents hallucinations has a few properties:
One question per article. If a page covers three different topics, the retriever may return it for any of them, but the model will see all three and may blend them into a single answer. Split the content so each article answers one question clearly.
Explicit titles. "Getting started" tells the retriever nothing. "How to request a refund for a Shopify order" tells it exactly what the page covers. Titles are the strongest retrieval signal you have.
No buried answers. If the actual answer is in paragraph four after a three-paragraph introduction, some retrieval systems will return the introduction instead. Put the answer first.
State what you do not support. If you do not offer phone support, say so. If your refund window is 30 days with no exceptions, write "no exceptions." Absent statements are invisible to retrieval. The model cannot find what you never wrote.
Version and date your content. Stale articles cause a specific kind of hallucination: technically accurate answers to a policy that changed six months ago. A refresh cadence matters. If your pricing changed in March, the article from January is now a hallucination source.
We wrote a full guide on building a knowledge base for AI support agents if you want to go deeper on structure and maintenance.
What is refusal behaviour and why does it matter?
The most underrated hallucination prevention tool is teaching the agent to say "I don't know." In customer support, a confident wrong answer is always worse than an honest "I'm not sure, let me connect you with the team."
Refusal behaviour means the agent has explicit rules about when not to answer:
- Low retrieval confidence. If the retriever returns passages with low similarity scores, the agent should not attempt an answer.
- Out-of-scope questions. A customer asking about your competitor's pricing or a regulatory question outside your domain should trigger a refusal, not a guess.
- Conflicting evidence. When retrieved passages disagree, the agent should escalate rather than arbitrate.
This is where the architecture of your AI agent matters. A well-configured agent treats silence as a feature, not a failure. It declines when the evidence is thin and hands off to a human through a clean handoff process instead of fabricating an answer.
How do you test for hallucinations before they reach customers?
You need a regression test set. This is a list of questions paired with expected answers, sourced from your actual content. Run the test set regularly and check for:
- Invented facts. The agent states something specific (a number, a date, a feature) that does not appear in your content.
- Source mismatch. The agent attributes an answer to the wrong article or combines information from unrelated pages.
- Stale answers. The agent gives an answer that was correct last quarter but is not correct now.
- Confident refusals that should not be refusals. The answer exists in your content, but the agent says it does not know. This is a retrieval problem, not a hallucination, but it shows up in the same test.
Start with 50 to 100 question-answer pairs covering your most common ticket types. Expand the set every time you find a hallucination in production. Treat the test set as living documentation.
How Keloa approaches hallucination prevention
Keloa's AI agents are grounded exclusively in your content. Every answer includes a citation pointing back to the source passage, so your team can verify what the agent said and where it found it. When the agent cannot find a confident answer, it declines and hands the conversation to a human in your unified inbox with the full context attached.
The citation requirement is not decorative. It forces the retrieval system to produce evidence and the model to stay within it. If there is no citable passage, there is no answer. That constraint is more effective than any prompt trick at keeping hallucinations out of customer conversations.
Frequently asked questions
What causes AI hallucinations in customer support? The most common cause is a gap between what the customer asks and what the knowledge base contains. When the retrieval system cannot find a strong match, the language model fills the gap with plausible-sounding but unsourced content. Stale or conflicting articles also contribute.
Can RAG completely prevent AI hallucinations? No. RAG reduces hallucinations significantly by grounding answers in retrieved content, but retrieval failures, chunk boundary issues, and conflicting sources can still cause the model to fabricate or blend information. Refusal behaviour and content quality matter just as much.
How often should I audit my AI agent's answers? Weekly sampling is a good baseline for a team handling moderate volume. Review 20 to 30 AI-handled conversations per week, focusing on topics where your content is thin or recently updated. Expand the cadence during product launches or policy changes.
What is the difference between a hallucination and a wrong answer? A hallucination is a specific kind of wrong answer where the model generates content that is not supported by any source. A wrong answer might come from outdated content, which is a content problem, not a model problem. Both need fixing, but through different paths.
Should I let the AI agent say "I don't know"? Yes. A well-configured support agent should decline to answer when retrieval confidence is low. Customers prefer an honest "I'm not sure, let me check with the team" over a confident wrong answer. Pair the refusal with a clean handoff so the customer is never left without a path forward.