Agentic RAG

Classic RAG searches your documents for every question, whether that's needed or not. Say hello, and it searches. That's wasteful, and it can confuse the model. Agentic RAG is smarter. Search is simply a tool, and the agent decides when it's needed.

• Classic RAG always searches
• Even when you say hello
• Agents decide when to search

Search as a tool

We wrap our search into a tool, with a clear description. We give it two messages. A real policy question, and a simple thank you. Run it. For the helmet, it searched, and it gave the correct rule, that crashed helmets can't be returned. For the thank you, it didn't search at all. It decided for itself. And unlike a fixed pipeline, an agent can search twice, try different words, and combine results.

import numpy as np
from google import genai
from google.genai import types
from sentence_transformers import SentenceTransformer
from kb import POLICIES

embed = SentenceTransformer("all-MiniLM-L6-v2", device="cpu")
vectors = embed.encode(POLICIES, normalize_embeddings=True)

def search_policy(query: str) -> dict:
    """Search Velo Supply's policies. Use this before answering any policy question."""
    scores = vectors @ embed.encode([query], normalize_embeddings=True)[0]
    return {"results": [POLICIES[i] for i in np.argsort(-scores)[:2]]}

client = genai.Client()
for q in ["I crashed in my helmet. Can I send it back?", "Hi, thanks for your help!"]:
    r = client.models.generate_content(
        model="gemini-flash-latest", contents=q,
        config=types.GenerateContentConfig(
            system_instruction="You are Juno. Answer in one sentence.", tools=[search_policy]))
    used = len(r.automatic_function_calling_history or []) > 1
    print(f"Searched: {used} | {r.text}")

Searched: True | Helmets involved in a crash cannot be returned and must be replaced.
Searched: False | Hello! I'm Juno, and I'm happy to help you with whatever you need today.

POLICIES = [
    "Returns are free within thirty days if the item is unused.",
    "Wheelsets have a two-year warranty against manufacturing faults.",
    "Helmets involved in any crash must be replaced and cannot be returned.",
    "Orders over eighty dollars ship free to Europe.",
    "Chains are consumable parts and are not covered by warranty.",
    "Refunds reach the customer's card within five working days.",
]

Classic RAG versus agentic RAG

Which is better? For simple question answering, classic RAG is cheaper and more predictable. An agentic approach pays off when questions are complex, and when you've got several sources. It can check its results, and try again. But it costs more steps. As always, use the simplest one that works.

Answer only from sources

When facts matter, tell the agent to answer only from what it found, and to say so if it found nothing. Ask it to quote the policy that it used. Then a customer gets an answer that can be checked, and you can test whether each answer is grounded. A model that says, I don't know, is worth far more than one that guesses.

Summary