Knowing When to Stop

Ask a junior employee to tidy the stockroom, and they'll come back after ten minutes. Ask, is it done? They'll say yes. But what does done mean? Agents have the same problem. They stop too early, or they never stop. The fix is a clear definition of done.

• Tidy the stockroom
• What does done mean?
• Agents stop early, or never

A definition of done

We write down what success means, as a checklist. Status confirmed, next step stated, and under fifty words. Then a small check runs before the agent can finish. Here are two replies. Run it. The vague reply fails, and the checker says what's missing. The real one passes. That missing field can go straight back into the loop, as feedback. So the agent keeps working until the job's truly done, or until a limit is hit.

from pydantic import BaseModel
from llm import ask

class Done(BaseModel):
    met: bool
    missing: str

criteria = ["Order status is confirmed", "Next step is stated", "Reply is under 50 words"]

for reply in ["We're looking into it.",
              "Order B42 has shipped. It should arrive by Friday, and I'll email you the tracking link."]:
    check = ask(f"Criteria: {criteria}\nReply: {reply}\nAre all criteria met?", schema=Done)
    print(f"met={check.met} | {reply[:42]} | missing: {check.missing[:34]}")

met=False | We're looking into it. | missing: Order status is confirmed
met=True | Order B42 has shipped. It should arrive by | missing: None

from google import genai
from google.genai import types

client = genai.Client()

def ask(prompt, system=None, schema=None):
    config = types.GenerateContentConfig(temperature=0.2, system_instruction=system)
    if schema:
        config.response_mime_type = "application/json"
        config.response_schema = schema
    r = client.models.generate_content(model="gemini-flash-latest",
                                       config=config, contents=prompt)
    return r.parsed if schema else r.text

The ways an agent can stop

An agent has several ways out, and you should plan for each. Goal met, so reply. Stuck, so try another approach, once. Out of budget, so hand it over, with a summary of what's been tried. Not allowed, so ask for approval. Or not sure, so ask the customer a question. Agents that can ask for help are far more useful than ones that guess.

Summary