RAG explained: giving an AI your own business knowledge
Retrieval-augmented generation (RAG) lets an AI assistant look up your own documents before it answers, instead of relying only on what it learned in training. Here is how it works, what it is good for, and what to check first.
RAG, short for retrieval-augmented generation, is a way of making an AI assistant answer from your own documents. Amazon Web Services defines it as optimizing the output of a large language model so it references an authoritative knowledge base outside its training data before generating a response. In plain terms: instead of the AI guessing from general knowledge, the system first searches your price list, policies, manuals or past quotes, hands the relevant passages to the AI, and the AI writes its answer from those. You do not retrain or build a model to do this.
How RAG works, in four steps
AWS describes the process in four parts. Knowing them helps you ask a vendor sensible questions.
- Prepare your content: documents, databases or other sources are converted into numerical representations and stored in a searchable index (AWS calls it a vector database).
- Retrieve: when someone asks a question, it is converted the same way and matched against the index to find the most relevant pieces of your content.
- Augment: the retrieved passages are added to the question before it goes to the AI, so the AI has your material in front of it when it writes the answer.
- Keep it current: the content has to be updated, either automatically or in periodic batches, or the answers go stale.
Where the idea comes from
The term comes from a 2020 research paper, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks," by Patrick Lewis and colleagues, which combined a language model with a searchable index of Wikipedia so the model could pull in facts while it generated text.
Why a small business would use it
AWS lists several stated benefits. RAG is a more cost-effective way to introduce new information to a model than retraining it. It can bring current information into the answer. And it can show where an answer came from, with citations or references to sources that a person can check. For a Northwestern Ontario business, the practical cases are the ones where the answer already lives in your own files:
- A trades or service company that wants staff to find its own pricing notes, warranty terms or safety procedures without asking the owner.
- A clinic, accounting or insurance office that wants quick, sourced answers from its own policy and procedure documents.
- A lodge or tourism operator that gets the same guest questions about rules, rates and seasonal access, and wants answers pulled from its own current information.
- A remote or seasonal team where the person with the knowledge is rarely reachable by phone.
RAG is only as good as what you feed it. If the retrieval step pulls an outdated price list or the wrong document, the AI will write a confident answer from the wrong material. Check every answer that touches money, safety, health or legal terms against the original document, and use the source references a good tool shows you. Keep the source documents current; AWS lists keeping the data updated as a core part of the process.
What to check before you load documents in
Your documents may contain personal information about customers, patients or employees. The Office of the Privacy Commissioner of Canada has published principles for generative AI that include limiting collection, use and disclosure of personal information to only what is needed for the specified purpose, using anonymized or de-identified data where personal information is not required, and taking reasonable steps to make sure outputs are accurate, especially where they affect decisions about individuals. In practice, start with documents that contain no personal information, such as price lists, procedures and product sheets. Before uploading anything else, ask the vendor where the data is stored, who can see it, and whether it is used to train their models. Our PIPEDA primer covers the basics.
A sensible first test
Pick one folder of low-risk documents, such as your service menu, FAQs and procedures. Load it into a tool that shows its sources, then ask it twenty questions you already know the answers to. Count how many it gets right and whether the cited source is the right one. If it passes on the easy questions, widen the scope slowly. If it fails on the easy ones, the problem is usually messy or contradictory source documents, which are worth fixing regardless of AI.
Amazon Web Services, What is RAG (Retrieval-Augmented Generation) (definition, four-step process, cost, current information and source attribution): https://aws.amazon.com/what-is/retrieval-augmented-generation/ | Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv, submitted May 22, 2020: https://arxiv.org/abs/2005.11401 | Office of the Privacy Commissioner of Canada, Principles for responsible, trustworthy and privacy-protective generative AI technologies (limiting collection, de-identified data, accuracy): https://www.priv.gc.ca/en/privacy-topics/technology/artificial-intelligence/gd_principles_ai/
Frequently asked
Is RAG the same as training an AI on my business?
No. Training changes the model itself. RAG leaves the model alone and looks up your documents at question time, then gives the relevant passages to the model along with the question. AWS describes it as a more cost-effective way to introduce new information than retraining.
Does RAG stop an AI from making things up?
Not entirely. It gives the AI your material to work from and lets a good tool show its sources, but a wrong or outdated document can still produce a wrong answer. Verify anything important against the original.
Is it safe to upload customer information?
Treat it as a privacy question first. Canada's Privacy Commissioner points to limiting use of personal information to what is needed and using de-identified data where possible. Start with documents that contain no personal information and ask the vendor how data is stored and used.
Get the weekly Signal
The AI, funding, government, and tech moves that matter for Northwestern Ontario — one email a week, source-linked, read by a human before it reaches you.
One email a week from Thunder Bay AI. Unsubscribe anytime.