Routing

At a hospital, a triage nurse sees you first. The nurse doesn't treat you. They just decide who should. Heart problem, this way. Broken arm, that way. Routing does this for an agent. It's a small decision, and it makes everything after it better.

• The triage nurse
• Decides who treats you
• A small decision that helps

A model as a router

Each team has its own handler, and each one could have its own prompt, tools, and model. The router has one job. Pick a team from a fixed list. The schema means that it can't invent a new one. And notice the human option. Run it. Each message went to the right place. And the unusual one, from a lawyer, should head for a person. A good router always has a safe exit like that.

from enum import Enum
from pydantic import BaseModel
from llm import ask

class Team(str, Enum):
    tracking = "tracking"
    returns = "returns"
    technical = "technical"
    human = "human"

class Route(BaseModel):
    team: Team

HANDLERS = {
    "tracking": lambda m: "Tracking handler: look up the courier",
    "returns": lambda m: "Returns handler: apply the policy",
    "technical": lambda m: "Technical handler: search the manuals",
    "human": lambda m: "Send to a person",
}

for m in ["Where's my chain?", "This helmet doesn't fit, can I swap it?",
          "How do I index a rear derailleur?", "I'm a lawyer writing about your warranty."]:
    team = ask(f"Choose the team for this message: {m}", schema=Route).team.value
    print(f"{team:<10} {HANDLERS[team](m)}")

tracking   Tracking handler: look up the courier
returns    Returns handler: apply the policy
technical  Technical handler: search the manuals
human      Send to a person

from google import genai
from google.genai import types

client = genai.Client()

def ask(prompt, system=None, schema=None):
    config = types.GenerateContentConfig(temperature=0.2, system_instruction=system)
    if schema:
        config.response_mime_type = "application/json"
        config.response_schema = schema
    r = client.models.generate_content(model="gemini-flash-latest",
                                       config=config, contents=prompt)
    return r.parsed if schema else r.text

Why routing pays off

Routing brings three benefits. Specialised prompts do much better than one giant prompt. You can use cheap models for easy routes, and strong models for hard ones. And it's safer, because each route only gets the tools that it needs. The tracking route simply doesn't have a refund tool.

Handling the edges

Routers need care. Measure them with real messages, because a wrong route spoils everything after it. Give the model an other option, so it doesn't have to force a guess. Start with plain rules for the obvious cases, and keep the model for what's left. Log every decision. You'll find new categories in those logs.

Summary