Skip to content
Courses

RAG (retrieval-augmented generation)

RAG (retrieval-augmented generation) gets an AI to answer from your documents: it first searches them for the passages closest to the question and hands those to the model with the prompt.

Select any words here to ask your Maestro about them.

Your documents hold the 2024 returns policy (30 days) and a 2026 update (14 days). A customer asks how many days they have to return an item. What does the assistant answer?

An online store's assistant that answers from its own documents. Example data.

What it's for: getting an LLM to answer from information that wasn't in its training: your company's manuals, your policies, your reports. Instead of training another model, you find what's relevant and hand it over in the prompt.

How it works: the search compares the question with your documents, split into passages, and takes the closest ones. Those passages go into the model's context window, and it answers from them.

Where it fails: if the search doesn't bring back the right passage, the model answers from what it has. And even with a citation, it can hallucinate: check that the citation says what the answer says.

Practice it in What does an AI do with your file?, where you see which part of your file it read before it gave you a number.

How it's used

How it works, step by step

An assistant that answers from your documents

  1. Split your documents into passages a few paragraphs longDone once, before any question.
  2. Turn each passage into a list of numbers (an embedding)Passages about the same thing end up with similar numbers.
  3. marked: For each question, find the closest passagesThis step decides what the model can know.
  4. Put those passages in the prompt, next to the questionThey take up room in its context window.
  5. The model answers and says which passage it came fromThe citation is what you check.
The model doesn't read all your documents: it reads what the search hands it. When an answer is wrong, check first which passages it got.

When to look closer

It cites its source

"How much is express shipping?"

  1. marked: "Express shipping costs $14.99 [Shipping times]"

The answer comes with its source in brackets. Is that enough to trust the price?

Ask your Maestro about RAG

It answers from the free course page closest to this word, the way it answers while you study.

Example

You: What does "RAG" mean, in plain words?

Your Maestro: I'll explain it with a case of yours: it sticks better than a definition. Which documents at work would you like to ask an AI about?

Asking about: Answers from the page "Which chart should I use? Start with the question"

Only this browser keeps what you ask here. How it works

Learn more

Related terms

  • LLM (large language model)

    An LLM is a program trained on huge amounts of text to continue a piece of text one chunk at a time: it works out which chunk is likeliest to come next and writes it. ChatGPT, Claude and Gemini all run on an LLM.

  • Context window

    Everything a language model has in front of it at a given moment: your instruction, the files it has read and the results of its tools. What isn't in the context, the model can't use.

  • AI hallucination

    A hallucination is an answer from an AI model that sounds confident and well written but is false or comes from no source at all: a number, a fact or a quote nobody ever said.

  • Prompt

    A prompt is what you write to an AI model to ask it for something: the question or instruction, plus everything you give it to work with, like data, examples or the format you want back.

The whole glossary