Choosing the Right Tool

You've learned regular expressions, classic classifiers, embeddings, and large language models. Now the most senior skill of all. Choosing. People tend to reach for the newest tool, but the best engineers reach for the cheapest one that does the job.

• So many tools
• Choosing is the senior skill
• The cheapest that does the job

Costing the options

Say that we handle two thousand messages a day. We estimate a month's cost at example prices, for four approaches. Run it. Keyword rules cost nothing. A small model is around five dollars. A mid model is twenty five. And a large one is a hundred and eighty. That's a large range. Yet the right answer is usually a mix. Rules first, then a small model, and pass only the hard cases to the large one. Check current prices before you decide, as they change quickly.

tickets = 2000 * 30
in_tok, out_tok = 400, 120

price = {"Keyword rules": (0, 0), "Small model": (.10, .40), "Mid model": (.30, 2.50), "Large model": (3.0, 15.0)}
for name, (pi, po) in price.items():
    monthly = tickets * (in_tok * pi + out_tok * po) / 1e6
    print(f"{name:<14} ${monthly:>8,.2f} a month")

Keyword rules  $    0.00 a month
Small model    $    5.28 a month
Mid model      $   25.20 a month
Large model    $  180.00 a month

A decision guide

Here's a simple guide. Fixed formats, use patterns. Clear categories with labels, use a classic classifier. No labels yet, use embeddings. Finding by meaning, use vector search. Open questions and writing, use a language model. And anything high stakes needs a person. Start at the top of this list, and only move down when you have to.

The layered system

The best systems are layered, like a funnel. Rules handle the simple forty percent, instantly and for free. A small classifier handles the next forty percent. A large model handles what's left. And people handle the rest. Each layer only sees what the one before it couldn't do. That's how you serve millions of requests at a sensible cost.

Summary