Budgets and Stop Rules

An agent doesn't get bored, and it doesn't get embarrassed. If it's confused, it'll try the same thing again and again, all night, spending your money the whole time. So every production agent needs limits, and those limits must live in code.

• Agents never get bored
• Confusion means repeating
• Limits must live in code

A budget object

This small class tracks steps and money. On each model call, we charge it. If it goes over either limit, it raises an error. We then simulate a run in which the context keeps growing. Run it. It stopped itself. Look at how quickly the money adds up, and that's because each step resends a larger history. Whichever limit hits first, the run ends, and a person gets told.

class Budget:
    def __init__(self, steps=8, dollars=0.05):
        self.steps, self.dollars = steps, dollars
        self.used_steps, self.spent = 0, 0.0

    def charge(self, tokens_in, tokens_out):
        self.used_steps += 1
        self.spent += tokens_in / 1e6 * 0.30 + tokens_out / 1e6 * 2.50
        if self.used_steps >= self.steps:
            raise RuntimeError("Step limit reached")
        if self.spent >= self.dollars:
            raise RuntimeError("Budget limit reached")

b = Budget()
try:
    for step in range(100):
        b.charge(tokens_in=4000 + step * 3000, tokens_out=300)
except RuntimeError as e:
    print(f"Stopped after {b.used_steps} steps, ${b.spent:.3f}: {e}")

Stopped after 8 steps, $0.041: Step limit reached

Detecting a stuck agent

A step limit is a blunt tool. We can do better by spotting when an agent is repeating itself. We keep a list of each tool call, along with its arguments. If the same call appears three times, it's stuck. Run it. On the third identical call, we break out. Rather than simply stopping, you can add a message that says, this isn't working, try something else. That often gets it moving again.

from collections import Counter

def is_stuck(calls, limit=3):
    return any(n >= limit for n in Counter(calls).values())

calls = []
for name in ["get_order:A17", "check_stock:Chain", "get_order:A17", "get_order:A17"]:
    calls.append(name)
    print(f"{name:<20} stuck={is_stuck(calls)}")
    if is_stuck(calls):
        print("Breaking the loop, and telling the model to try something else")
        break

get_order:A17        stuck=False
check_stock:Chain    stuck=False
get_order:A17        stuck=False
get_order:A17        stuck=True
Breaking the loop, and telling the model to try something else

Every run needs these limits

Here's your standard set. A maximum number of steps. A maximum cost for each run. A time limit. A cap on identical calls. And a daily budget across all runs, which protects you from a bug that hits every customer. When a limit fires, fail safely, and hand it over to a human.

Summary