Specialty Digest

Global English editionLearners’ Study Dashboard — in development
The Knowledge Library for All

Retrieval-Augmented Generation

nounRetrieval and Knowledge Systemsalso RAG, Retrieval Augmented Generation

In one line

Retrieval-augmented generation (RAG) is an AI technique that lets a language model fetch relevant documents before answering, grounding its reply in retrieved text.

Retrieval-augmented generation (RAG) is a technique in artificial intelligence that lets a language model look up relevant documents from an outside source before it writes an answer. The response then draws on that retrieved text instead of relying only on what the model memorized during training. The approach was introduced in a 2020 research paper. It is now a standard way to make AI assistants more current, more specific to a subject and easier to check.

History of retrieval-augmented generation

The idea was formalized in a 2020 paper, “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” by Patrick Lewis and colleagues at Facebook AI Research, University College London and New York University. The authors posted it to the research archive arXiv in May 2020 and presented it at that year’s NeurIPS conference. Their system paired two kinds of memory. The first was parametric memory, the knowledge a pretrained language model stores inside its own numerical weights. The second was non-parametric memory, a searchable index of Wikipedia passages reached through a neural retriever. On open-domain question answering, where a system must answer factual questions without being told which document holds the answer, the authors reported state-of-the-art results at the time.

The motivation was a known weakness. A large language model holds its knowledge in billions of parameters, which makes that knowledge hard to inspect and expensive to update. Retrieval splits the work in two. A search component finds facts, and a language component writes them up in fluent prose. The approach also builds on decades of work in information retrieval, the field behind search engines. After chatbots built on large language models reached the public in late 2022, RAG became a common pattern for organizations that wanted such assistants to answer from their own documents.

How retrieval-augmented generation works

A RAG system runs in two phases: preparing the knowledge, then answering questions.

In the preparation phase, documents such as manuals, policies, web pages or research papers are cleaned and split into smaller passages called chunks. Each chunk is converted into an embedding, a list of numbers that captures the meaning of the text, so that passages on similar topics sit close together mathematically. The embeddings are stored in an index, often a vector database, alongside the original text.

At question time, the system follows four steps:

  1. Encode. The user’s question is converted into an embedding using the same method.
  2. Retrieve. The system searches the index for the passages whose embeddings are closest to the question’s, usually returning a handful.
  3. Augment. The retrieved passages are placed in the prompt next to the question, often with an instruction to answer only from the supplied text.
  4. Generate. The language model writes a response from that prompt. The system can attach the source passages so a reader can check them.

Because the model reads the passages at the moment of the request, changing an answer only requires changing the documents in the index. The model does not have to be retrained.

Types of retrieval-augmented generation

The original paper described two variants. RAG-Sequence uses one set of retrieved documents to produce a whole answer. RAG-Token can draw on different documents for each word piece it generates. Both are mostly of research interest today.

In practice, engineers describe several approaches built on the basic pipeline. Terminology varies between vendors, but these labels are common:

  • Basic (naive) RAG: embed, retrieve the top passages and generate, with no further refinement.
  • Hybrid search RAG: combines keyword matching with vector similarity, so exact terms such as product codes are not missed.
  • Reranked RAG: retrieves a larger candidate set, then uses a second model to put the best passages first.
  • Graph-based RAG: retrieves from a knowledge graph of entities and relationships instead of loose text chunks.
  • Agentic RAG: lets an AI agent decide whether to search, what to search for and whether to search again.

Applications of retrieval-augmented generation

RAG is used wherever an answer must come from a specific, changing body of documents.

  • Customer support: assistants that answer from product manuals, help-center articles and policy pages.
  • Enterprise search: employees ask questions across internal wikis, contracts and meeting notes, with access rules limiting what each person can retrieve.
  • Legal and compliance work: tools that find relevant clauses or regulations and point to the passage they used.
  • Education: tutors that answer from a course’s own readings instead of general internet knowledge.
  • Software development: coding assistants that search a company’s codebase and documentation before suggesting changes.
  • Healthcare and science: literature-search tools that summarize retrieved papers for professionals, who then check the sources.

In high-stakes fields such as medicine and law, the output is meant to support a qualified person, not replace one.

Advantages and disadvantages of RAG

The main advantage is freshness without retraining. Updating a document index is far cheaper and faster than retraining a model, so answers can reflect last week’s policy change. A second advantage is traceability. Because the system knows which passages it used, it can show them, and a reader can confirm the claim. RAG also lets an organization point a general-purpose model at private material without building a new model, and it can respect permissions by retrieving only what a given user may see.

The limits are real. A RAG system is only as good as its retrieval step. If the search returns the wrong passages, or the documents are outdated, badly split or contradictory, the answer will be poor even when the language model is strong. The model can also misread, ignore or over-trust what it retrieves, so RAG lowers the risk of hallucination without removing it. Adding retrieved text makes prompts longer, which raises cost and response time. Retrieval also creates security exposure. A malicious instruction hidden in a retrieved document can try to steer the model, an attack known as prompt injection, and poor access controls can leak confidential material.

RAG vs. fine-tuning

Fine-tuning continues a model’s training on a smaller dataset, changing its weights so it adopts a style, format or specialized skill. RAG leaves the model unchanged and supplies facts at question time. Fine-tuning suits behavior that should stay consistent, such as a house tone or a fixed output format. RAG suits knowledge that changes often or must be cited. Many production systems use both.

FAQs

Does RAG stop AI from making things up? No. It reduces the risk by giving the model source text to work from, but the model can still misstate or misinterpret that text, and the retrieval step can return the wrong material.

Does RAG need a vector database? Not strictly. Vector search is the most common method, but keyword search, hybrid search and knowledge graphs can all serve as the retrieval step.

Who invented RAG? The term and the method were introduced by Patrick Lewis and colleagues in a 2020 paper from Facebook AI Research, University College London and New York University.

More in Retrieval and Knowledge Systems

Key facts

  • RAG was introduced in a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, University College London and New York University.1
  • The original system combined a pretrained language model (parametric memory) with a searchable Wikipedia index (non-parametric memory).1
  • Updating what a RAG system knows means updating its document index, not retraining the model.2
  • RAG can ground answers in an external knowledge base and show sources, which lowers the risk of hallucination without removing it.23

Related concepts

Quick checkWhat are the two kinds of memory combined in the original RAG paper?Show answer

Parametric memory, the knowledge stored in a pretrained language model's weights, and non-parametric memory, a searchable index of Wikipedia passages.

Sources

  1. Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, https://arxiv.org/abs/2005.11401
  2. IBM Research, What is retrieval-augmented generation?, https://research.ibm.com/blog/retrieval-augmented-generation-RAG
  3. Amazon Web Services, What is RAG? Retrieval-Augmented Generation AI Explained, https://aws.amazon.com/what-is/retrieval-augmented-generation/

Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue