The most common reason an internal AI assistant disappoints has nothing to do with the model. It is that the material it draws on is incomplete, contradictory or two years old. Preparing that material is the project; connecting it is the easy part.
Collect the questions before the documents Do not start with a folder. Start with a list of the questions people actually ask — pulled from support tickets, sales emails, and the internal chat messages that begin with "quick question". Sixty real questions is a better foundation than six hundred pages, because it tells you what has to be correct.
Write short, self-contained answers Retrieval works on passages, not on whole documents. An answer that assumes the reader has already read the three sections above it will be retrieved without that context and will mislead. Each entry should stand alone: state the situation, give the answer, note the exception, and stop.
Resolve contradictions before you index anything Two documents describing different refund windows will produce two different confident answers, and nobody will know which one a customer received. Force the decision now. This exercise usually improves the business regardless of whether the assistant is ever built.
Give every entry an owner and a date Content decays quietly. An owner and a last-reviewed date make staleness visible and reviewable, and they make it possible to answer the question a support lead will eventually ask: who approved this wording?
Separate what is public from what is not Internal margins, escalation thresholds and named individuals should not sit in the same collection as customer-facing answers unless access control is enforced at retrieval time. Deciding this at the start is straightforward; untangling it after launch is not.
Test with the awkward questions Evaluate using badly phrased, hostile and out-of-scope questions, not the tidy ones you wrote the content from. The behaviour you are checking for is a clean admission of uncertainty. An assistant that answers everything is not well informed; it is unwilling to stop.
Treat it as a living asset Log what could not be answered and review that list on a schedule. Over a few months the gaps close, and the knowledge base becomes useful to your team on its own terms — often more valuable than the assistant sitting on top of it.
Our generative AI work usually starts with this content step. Contact us and tell us where your answers currently live.
AI Studio · Published 23 Mar 2026
← All insights