Tokens, Context, and Cost

In its first week, a startup's agent got stuck in a loop overnight, and they woke up to a bill of several thousand dollars. Nobody had been watching the tokens. Understanding this one number will save you real money.

• A loop ran overnight
• A bill of thousands
• Nobody watched the tokens

What fills the context window

A model can only see so much at once. That's its context window. And it fills up fast. Your system prompt takes some, each tool description takes more, then comes the conversation, and every tool result. When it's full, something has to go. Managing this space is a core skill. People call it context engineering.

Watch it grow

A handy rule, one token is roughly four letters. Let's simulate twenty turns, where each call resends everything. Run it. It keeps growing with every turn, even though each new message is small. And you pay for the entire pile, every single time. That's the reason why long agent runs get expensive.

def estimate_tokens(text):
    return len(text) // 4

system = "You are Juno... " * 40
tools = "tool description... " * 60
turn = "customer and agent message... " * 8

history = 0
for n in range(1, 21):
    history += estimate_tokens(turn)
    total = estimate_tokens(system) + estimate_tokens(tools) + history
    if n in (1, 5, 10, 20):
        print(f"Turn {n:>2}: {total:>5} tokens sent")

Turn  1:   520 tokens sent
Turn  5:   760 tokens sent
Turn 10:  1060 tokens sent
Turn 20:  1660 tokens sent

Estimating a bill

Let's price it, with example rates, per million tokens. Nine thousand tickets a month. One simple call for each. Then a six step agent. Then a six step agent with a long history. Run it. The agent costs six times more, and a long context costs several times more again. Always do this sum before you build. Check real prices, because they change often.

price_in, price_out = 0.30, 2.50

def cost(tokens_in, tokens_out, tickets):
    one = tokens_in / 1e6 * price_in + tokens_out / 1e6 * price_out
    return one * tickets

print(f"Small prompt: ${cost(1500, 200, 9000):,.2f} a month")
print(f"Six-step agent: ${cost(1500 * 6, 200 * 6, 9000):,.2f} a month")
print(f"Long history: ${cost(9000 * 6, 200 * 6, 9000):,.2f} a month")

Small prompt: $8.55 a month
Six-step agent: $51.30 a month
Long history: $172.80 a month

Summary