Retrieval Augmented Generation

A language model does not know your company's documents, your private notes, or yesterday's news. And it may invent answers, instead of admitting that. Retrieval augmented generation, or R A G, solves both problems. It is the most widely used pattern for useful A I apps, and you are ready to understand it.

• The model does not know your documents
• RAG: look it up first, then answer
• The most popular AI app pattern

The RAG pipeline

Here is the pipeline. A user asks a question. We turn the question into an embedding, and search the vector database, for the most relevant chunks. We then build a prompt, that contains the question, and those chunks, with the instruction, answer using only this information. The model writes an answer, grounded in your documents, and can mention its sources. Look it up first, then answer.

1. Questionthe user asks
2. Retrievefind the best chunks
3. Augmentput them in the prompt
4. Generateanswer from the sources

A RAG answer

Here is how it looks to a user. They ask, what is the refund period for online orders? The system finds the matching paragraph in the company policy. The model answers, you can return online orders within thirty days, and it points to the document it used. Without retrieval, the model would have guessed. With retrieval, it reads the real policy.

Why RAG is so popular

Why do so many teams choose R A G? It uses fresh information, because you can update the documents any time, without retraining. It reduces hallucinations, because the answer comes from real sources. It shows its sources, so people can verify. It keeps private data in your control. And it is far cheaper than training a new model. For most business uses, R A G is the first thing to try.

Summary