What Is RAG?
Answering from your own documents.
Ask a language model about Bright Books' return policy, and it'll invent one. It has never seen your policy. It would be like asking a stranger in the street. So what if, each time, before answering, the model got to read the right page of your handbook? That's retrieval augmented generation. It's the most widely used pattern in applied AI.
- The model never saw your policy
- Let it read the right page first
- The most used pattern in applied AI
The RAG pipeline
There are two phases. First, before anyone asks anything, we prepare. We split the documents into chunks, embed them, and store them in a vector database. Then, for each question, we embed the question, find the closest chunks, and add them to the prompt. The model then answers using those facts. Search finds the truth. The model writes it up nicely.
- Chunksplit documents
- Embedmeaning as numbers
- Storevector database
- Retrievefind closest chunks
- Generateanswer from them
Why RAG instead of training?
Why not just train the model on your documents? Because RAG is much cheaper, and it updates instantly. Change a policy file, and answers change right away. You can show sources, so that people can check. Access control stays simple. And it greatly cuts down on invented facts. Fine tuning is for changing how a model behaves. RAG is for what it knows.
Summary
Search finds the facts, and the model writes the answer. Next, we build one. And the very first step turns out to matter the most. How do you cut up a document?
Key takeaways- Retrieve, then generate
- Cheaper than training
- RAG is for knowledge, fine-tuning for behaviour