What is RAG (retrieval-augmented generation)?

Retrieval-augmented generation, or RAG, is how you get a language model to answer from information it was never trained on. Instead of relying purely on what the model memorised during training, a RAG system fetches relevant documents at the moment a question arrives and puts them into the prompt, so the model answers from that supplied text rather than from memory alone.

The problem it solves

A language model’s knowledge is frozen at its training cutoff. Ask it about something that happened afterwards, or about a document it never saw — a company’s internal policy, a product manual, a contract — and it has two options: say it does not know, or guess. Researchers at NIST and Stanford HAI have both flagged this second failure mode, often called hallucination, as a central risk of deploying language models: the model can produce fluent, confident text that is factually wrong, with no visible difference between a grounded answer and an invented one.

RAG addresses this directly by grounding the model’s response in real, current, specific documents rather than whatever it absorbed during training. The model still writes the answer, but it is writing from a source placed in front of it, not pulling from a static, ageing store of learned patterns.

How it works in three steps

RAG systems vary in complexity, but the underlying pattern is consistent:

1. Store your documents. The source material — manuals, articles, internal records, whatever the assistant should know — is processed and stored in a form that can be searched quickly, typically broken into smaller chunks.
2. Retrieve the most relevant pieces. When a question comes in, the system searches that store and pulls out the chunks most likely to contain the answer.
3. Hand those to the model with the question. The retrieved text is inserted into the prompt alongside the user’s question, and the model is asked to answer using that material.

The model never “learns” the documents in the way training does. Each retrieval is temporary, scoped to that one question, and discarded afterwards.

Why it matters

RAG is the mechanism behind most “chat with your documents” tools and company-knowledge assistants, whichever underlying model powers them — ChatGPT is the most recognised name for this kind of interface, though Claude, Gemini and Copilot all support the same pattern. It matters because it reduces hallucination: the model is working from text placed directly in front of it and can, in effect, quote and summarise that text rather than inventing a plausible-sounding fact from training data. Researchers describe this as improving factual grounding rather than eliminating errors outright — the model is still generating language, just from a narrower and more relevant starting point.

This is also why RAG has become the default approach for tasks where currency and specificity matter more than general knowledge: searching a knowledge base, answering questions about a specific document set, or building an assistant that needs to reflect information that changes after any model’s training cutoff.

The catch

RAG is only as good as what it retrieves. If the retrieval step pulls the wrong documents, or misses the one that actually contains the answer, the model does not know that anything went wrong. It answers confidently from whatever it was given, because that is exactly what it was designed to do. A badly matched or outdated document store can therefore produce an answer that sounds well-grounded and is simply wrong. Stanford HAI’s work on evaluating these systems points to retrieval quality, not the model itself, as the most common point of failure in practice.

Around’s own approach — precomputed figures paired with cited sources — is a distant cousin of this idea: ground the answer in something checkable rather than trusting a model’s memory alone.