How to Keep an Arrow AI Assistant From Making Things Up
A website assistant that says “I’m not sure, here’s the contact form” is working correctly. One that invents a 30-day return window that does not exist is worse than no widget at all.
Hallucination in document-grounded assistants comes from three places: thin source coverage, conflicting documents, and an instruction that rewards answering. Fix all three.
Why does a grounded assistant invent answers?
Because retrieval returns something. If a visitor asks about gift wrapping and the top result is a shipping policy, a language model will stitch an answer from the shipping policy.
There is no “nothing relevant found” state unless you build one. Set the assistant to answer only when retrieved content directly covers the question, and to hand off otherwise.
How do you structure the knowledge base?
Write short, single-topic documents. One document per policy, not one big FAQ page.
- Return policy: window, condition, who pays shipping, exceptions. Four headings, one page.
- Pricing: current plan names, limits, overage rates. Date it.
- Onboarding: the five steps, in order.
Long documents produce diluted retrieval. A 4,000-word help page gets chunked into pieces that lose context, and the assistant answers from a fragment.
Around 20-40 focused documents covers most support volume for a small business.
What instructions reduce invented answers?
Add these to the system persona:
- Answer only from provided sources. If the source does not cover it, say so.
- Never quote prices or dates that are not in the source documents.
- Keep answers under 80 words, and link the source page.
- For legal, refund or account-specific questions, hand off to a human.
The 80-word limit matters more than it sounds. Long answers contain more sentences, and more sentences means more chances to add something unverified.
How do you find the gaps after launch?
Read the transcript log weekly. Look for:
- Questions with a fallback reply — those are missing documents.
- Visitors repeating the same question twice — the first answer did not land.
- Sudden drops in session length — people gave up.
Tag the top ten unanswered questions each week and write those documents. After three months the fallback rate usually settles under 10%, which is a reasonable target.
If the volume of documents grows past a few hundred and retrieval gets unreliable, the pattern to consider is a proper knowledge engine. AnswerRocket handles structured warehouse data better than document retrieval does; use each for what it is good at.
What should you not try to fix with the assistant?
Anything that needs account data. That is an integration project, not a prompting problem.
Anything legally sensitive. Refund disputes, warranty claims, medical or financial advice — route those to humans with a clear handoff message.
Anything where the answer changes daily. Inventory levels, delivery dates for specific orders. The assistant will confidently serve yesterday’s numbers. Link to a live status page instead.
If your content itself is the bottleneck — pages out of date, policy contradictions across pages — fix that first. Builder.io gives you one place to edit the pages the assistant reads from, which cuts the conflicting-document problem at the root.
Quick Summary
- Hallucinations come from thin or conflicting documents, not from the model alone.
- Use 20-40 short single-topic documents; long help pages chunk badly.
- Instruct the assistant to answer only from sources, cap answers at 80 words, and hand off on refunds and legal topics.
- Review transcripts weekly; every fallback reply is a missing document.
- Target under 10% fallback rate after three months of tuning.
