Retrieval Augmented Generation (RAG)

What Retrieval-Augmented Generation is

Retrieval-augmented generation, usually shortened to RAG, is a way of giving an AI tool a set of documents to search at the moment it answers. The reply is then drawn from that material and not from the model’s training alone. It is the mechanism behind a knowledge base, and it is the sense in which this site uses the term.

The three words describe the three steps. The tool retrieves relevant material, the material augments what the model has in front of it, and the model generates an answer from both. Nothing about that requires a person to understand the machinery. It requires knowing what the tool is answering from.

Why a Model needs it

A large language model does not look anything up. It reads a prompt, predicts the most likely next piece of text, and repeats that step until an answer exists. It is completing a pattern and not searching for a fact, and it has no step that checks a claim against a record.

Two limits follow. Training stops at a cut-off, so anything after that point is not in the model. And nothing in the model knows what your business wrote down last week. Ask an ungrounded model about your leave policy and it answers from general patterns about leave policies. The reply may be fluent and reasonable, and it may describe a policy your business does not have.

RAG is the usual answer to both. It connects the tool to a live source, so the answer comes from material that is current and that belongs to you. The large language model and generative AI pages both name it as the technique that closes the gap.

RAG and Grounding

The two terms are close and do different jobs. Grounding is the outcome: an answer tied to specific source material, so that a reviewer can check it. RAG is the usual mechanism that produces it. Grounding says what you want. RAG is one way of getting it.

That is why the two pages link to each other. The grounding page covers how to keep a tool on its source, including naming the source in the prompt, saying what to do when the source is silent and asking the tool to quote the part it used. This page covers what sits underneath and where the courses build it.

The Smallest version

RAG does not have to mean special software. The simplest form is the source part of a prompt. The prompting workshop teaches four essential parts of a prompt, goal, context, source and expectations, and the source is the material the answer should draw on, such as a document you attach. Asking a question of a document you have supplied is the same idea at its smallest scale.

The same workshop teaches the fallback that makes it safe. When the document does not say, the tool is told to reply that the answer is not specified in the document instead of inventing one. Without that instruction the model tends to answer anyway, because answering is what it is built to do.

The larger version: a Knowledge base

At larger scale the same idea becomes a knowledge base, a set of documents an assistant answers from every time. On this site it is one of the three pillars of any chatbot, beside instructions and tools, and the HR course teaches what a knowledge base is as a session of its own.

The agents day is where it is taught under its full name. Its second session builds a Copilot Studio agent, and the outline lists knowledge bases and RAG as a topic, followed by control layers. Unstructured documents cover general questions. Structured FAQs cover the common ones. Topic-based answers cover high-risk policies where the wording must not vary. Each layer is a decision someone made in advance, which is why the finished agent is more predictable than a chat that improvises.

Where it appears across the Catalogue

The idea turns up in every platform the courses use, under different names. The ChatGPT Enterprise course covers asking questions about your own documents and builds an HR assistant as a custom GPT, and teaches why the source behind an answer matters as much as the answer. The Mistral Le Chat workshop has a block on libraries as knowledge bases, covering document questions, policy assistants and company knowledge assistants.

The Copilot Chat day works through what its Work Mode and Web Mode are each grounded in, and the admin course sets a basic agent with a knowledge base beside a chatbot with a knowledge source for a service desk. In every case the pattern is the same. A person asks a question, the assistant answers from the named material, and the answer can be checked against it.

A fixed Knowledge base is not Memory

The distinction that matters most in practice is the one the databases session draws. An agent with a fixed knowledge base answers from what it was given. An agent with a live database it can read and write has memory, which a fixed knowledge base does not.

The first is grounded in a snapshot. The second is grounded in something that changes, and can add to it. That is why the session teaches building the database first, before pointing an agent at it, and why it shows a project and prompt library agent built live, with version history so the library grows. A team choosing between the two is really choosing whether the material should stay as it was or keep changing as people use it.

The same mechanism behind AI Answers

RAG is not only something a business builds. It is also what sits underneath the AI answers that now appear in search. The site’s article on ranking describes query fan-out: before answering, a system generates several variations of the question a person typed, pulls candidate sources for each variation separately, then merges the results with a technique such as Reciprocal Rank Fusion. The article’s description is that fan-out is retrieval-augmented generation run several times per search instead of once.

The consequence for a business is a different target. A page no longer needs to win the single most competitive version of a query. It needs to be the clearest, most specific source for one of the several sub-questions the system retrieves for. The site’s article on AI Overviews adds the link back to this page. Retrieval is also what makes grounding a meaningful check on an AI answer, because a properly grounded answer points to a real source for each claim.

What goes into a Knowledge base is a Decision

Putting documents in front of a tool is a decision about data. A staff handbook or a published regulation is low risk. A folder of client contracts or HR records is not, and the rule about what may be entered into an AI tool applies to what you attach as much as to what you type.

The site’s article on why banning AI at work backfires argues for drawing that line by kind of data and not by tool. It also matters which tool holds the documents. Vendors state that business content typed into their tools is not used to train their models by default, and that is a policy each vendor publishes and a business should read for itself. The agents course adds governance and security essentials, user authentication and controlled topics for high-risk policies.

Mix-ups worth avoiding

Four confusions come up often enough to name. The first is treating RAG as a fix for hallucination, when it narrows the problem and leaves the check in place. The second is treating a knowledge base as memory, when a fixed one is a snapshot and only a live database that the agent can read and write gives it memory.

The third is treating retrieval as verification. A tool that cites a source has shown where it says the answer came from, not that the source supports it. A citation has a recognisable shape, and a model can reproduce that shape with nothing real behind it. The fourth is assuming RAG needs a platform. Attaching one document and naming it as the source is the same idea at its smallest.

Where it goes wrong

An answer can only be as reliable as the source it was grounded in. A tool reading an out-of-date handbook gives a fluent, traceable and wrong answer, and it reads no less confidently for being wrong. The check is what catches it, which is why the courses teach a person in the loop as part of the method.

Two habits keep it honest. Tell the tool what to do when the source is silent, and ask it to quote the part of the source each claim comes from. Then test the assistant against the questions it will actually meet, including the ones it should refuse. The agents day covers testing as part of the build, and the same session covers the limits of the platform, so a team leaves knowing when an agent is the wrong answer.

Where it is taught and where it appears in an Audit

The AI Agents in Action course teaches knowledge bases and RAG by name, and the HR course builds a policy assistant on the same idea. The rest sit in the wider training catalogue.

An audit meets RAG from the other side. It looks at where a business’s workflows depend on its own documents, such as policies, handbooks, regulations and tender material, and at which of those an assistant could answer from with a person checking. Those are the workflows where a knowledge base changes the result most. Knowing what the term means, and what it does not promise, is what lets an audit recommend an assistant built on the right documents and not simply the newest tool.