What is retrieval-augmented generation?
Retrieval-augmented generation is the practice of fetching relevant documents at request time and placing them in the model''s context so the answer is grounded in specific sources rather than in the model''s parameters alone. A retrieval step — keyword search, vector search, or both — selects passages; the model is then asked to answer using those passages and to cite them. RAG reduces, but does not eliminate, fabricated answers.
Documents are split into chunks, embedded as vectors and indexed. At query time the question is embedded and the nearest chunks are retrieved, often re-ranked by a smaller model, then inserted into the prompt with instructions to answer only from the supplied material. Hybrid systems combine lexical matching, which handles exact identifiers and rare terms well, with vector similarity, which handles paraphrase.
An internal policy assistant retrieves the three most relevant paragraphs of an HR handbook and answers from them, linking each claim to the paragraph it came from, so an employee can verify the answer without trusting the model.
Most enterprise deployments that survive contact with users are retrieval systems with a language model attached, not the other way round. Retrieval quality, not model choice, is usually the limiting factor on answer accuracy.
- That RAG stops hallucination. It reduces it; models still misread or over-generalise retrieved text.
- That a vector database is required. For modest corpora, well-tuned keyword search plus re-ranking is often competitive.
- That chunking is a detail. Chunk size and boundaries change retrieval quality more than most model swaps.
- Published RAG benchmarks use public corpora that rarely resemble messy internal document sets.
- Retrieval quality is corpus-specific; no configuration transfers across organisations without testing.