Automatic Compaction

Your agent is deep into a long job when suddenly the context window fills up, and it fails. Everything's lost. You've learned how to summarise. Now let's make that happen automatically, before the limit is reached.

• Deep into a long job
• The context fills up
• Summarise before the limit

Compacting automatically

We check the size before each model call. If we're over the limit, then we summarise all but the most recent messages, and replace them with that summary. Run it. The conversation falls to a fraction of its size, the recent messages are still word for word, and the order number survives in the summary. Agents run this quietly, and you never notice it. The important choice is what to keep. Decisions, file names, and open tasks all need to survive.

from llm import ask

LIMIT = 200
messages = [f"Turn {i}: the customer asked about order A17 and the wheel fault, and we checked it." for i in range(1, 15)]

def size(msgs):
    return sum(len(m) for m in msgs) // 4

def compact(msgs, keep=3):
    old, recent = msgs[:-keep], msgs[-keep:]
    note = ask("Summarise in one sentence, keeping order IDs:\n" + "\n".join(old))
    return [f"Summary: {note}"] + recent

print("Before:", size(messages), "tokens")
if size(messages) > LIMIT:
    messages = compact(messages)
print("After:", size(messages), "tokens")
print(messages[0][:90])

Before: 288 tokens
After: 85 tokens
Summary: The customer repeatedly asked about order A17 and the wheel fault, and we checked

from google import genai
from google.genai import types

client = genai.Client()

def ask(prompt, system=None, schema=None):
    config = types.GenerateContentConfig(temperature=0.2, system_instruction=system)
    if schema:
        config.response_mime_type = "application/json"
        config.response_schema = schema
    r = client.models.generate_content(model="gemini-flash-latest",
                                       config=config, contents=prompt)
    return r.parsed if schema else r.text

Summary