Making RAG Work

A first R A G system is easy to build, and easy to get wrong. Sometimes it finds the wrong chunks. Sometimes it ignores the right ones. In this lesson, you will learn the most common problems, and the practical fixes professionals use.

• Easy to build, easy to get wrong
• Common failures
• Practical fixes

Common problems and fixes

Here are the classic problems. If the right chunk is never found, improve the chunking, or try a better embedding model. If too many irrelevant chunks arrive, retrieve fewer, or add a re ranking step, that sorts them again. If the model ignores the sources, make the instruction firmer, answer only from the context. And if the answer is not in the documents, tell the model to say, I could not find that.

Better retrieval tricks

Several tricks improve retrieval. Hybrid search, combines keyword matching, with semantic search, so exact names and codes are not missed. Re ranking, uses a second model, to sort the top results. Query rewriting, asks the model to clarify a vague question first. And metadata filters, narrow the search, for example, only documents from this year.

Measuring a RAG system

As always, measure. Check retrieval, did the right chunk appear in the top results? Check faithfulness, is the answer supported by those chunks? And check usefulness, does it really answer the question? Build a test set of real questions, run it after every change, and watch the scores. That is how a prototype becomes a trustworthy product.

Summary