Designing Great Tools

When an agent behaves badly, people blame the model. They rewrite the prompt, and they try a bigger model. But the usual cause is simpler. The tools are confusing. Imagine being handed a toolbox where every tool is labelled, misc. You'd make mistakes as well.

• People blame the model
• The tools are usually the cause
• A toolbox labelled misc

Bad tool, good tool

Compare these two. The bad one is called do stuff, and it says, handles orders. What does it do? When should you use it? What goes in data? The good one has a clear name, it says when to use it, and it gives an example of an ID. Run it. Two words of guidance, against more than twenty. To a model, that description is the entire documentation. Spend your time here. Teams who've built serious agents say that they spend more time on tools than on prompts.

bad = {
    "name": "do_stuff",
    "description": "Handles orders.",
    "params": ["data"],
}

good = {
    "name": "get_order",
    "description": ("Look up one order by its ID. Use this before answering any "
                    "question about delivery, status, or refunds. IDs look like A17."),
    "params": ["order_id"],
}

for t in (bad, good):
    print(f"{t['name']:<10} {len(t['description'].split()):>2} words of guidance")

do_stuff    2 words of guidance
get_order  22 words of guidance

Keep results small

Remember, everything that a tool returns goes into the context, and gets resent on each step. Here's an order record, with thousands of letters of notes and logs that the model doesn't need. We'll compare returning all of it with returning just three fields. Run it. That's a huge saving, on every single step, and the model has less noise to get confused by. Return only what's useful for the next decision.

import json

ORDER = {"id": "A17", "item": "Carbon wheelset", "status": "delivered",
         "warehouse_notes": "x" * 2000, "audit_log": ["event"] * 300}

def full(order):
    return order

def trimmed(order):
    return {k: order[k] for k in ("id", "item", "status")}

print("Full result:", len(json.dumps(full(ORDER))) // 4, "tokens")
print("Trimmed:", len(json.dumps(trimmed(ORDER))) // 4, "tokens")

Full result: 1200 tokens
Trimmed: 15 tokens

Rules for great tools

Here's the checklist. Give each tool one clear job. Choose names that a new hire would understand. Prefer a few strong tools over dozens of weak ones, because too many tools confuses a model. Make the arguments hard to get wrong. Use an enum instead of free text. And make sure that errors explain how to fix the problem.

Design for tasks, not for APIs

A common mistake is wrapping every API endpoint as a tool. That forces the model to make five calls for one job. Instead, build tools around whole tasks. Rather than get customer, get orders, and get shipments, offer one tool called customer overview. It's fewer steps, fewer tokens, and fewer mistakes.

Summary