Reflection

You send an email, and the moment you press send, you spot the mistake. If only you'd read it once. Agents can do just that. Before they finish, they stop and check their own work. It's cheap, and it catches a surprising number of errors.

• You spot the mistake after sending
• Read it once first
• Agents can check their work

Checking an answer

Here's an answer with a hidden mistake. It says that seventeen times twenty eight is four hundred and seventy six, which is right. Ten percent off that should be four hundred and twenty eight point four. But it says four hundred and thirty. We ask the model to check each step. Run it. It spotted the error. And we've printed the true total, to confirm. The same model that makes a mistake can often catch it, when it's asked to review rather than to answer.

from llm import ask

task = "How much do 17 chains cost at $28 each, with 10% off orders over $400?"
answer = "17 x 28 = 476. Over $400, so 10% off gives $430."

critique = ask(f"Task: {task}\nAnswer: {answer}\n"
               "Check the arithmetic step by step. If it's wrong, give the correct "
               "total in one short sentence. If it's right, say OK.")
print(critique)
print("True total:", round(17 * 28 * 0.9, 2))

- 17 × $28 = $476 (Correct)
- Since $476 is over $400, the 10% discount applies (Correct)
- 10% of $476 is $47.60, and $476 − $47.60 = $428.40, not $430 (Incorrect)

The correct total is $428.40.
True total: 428.4

from google import genai
from google.genai import types

client = genai.Client()

def ask(prompt, system=None, schema=None):
    config = types.GenerateContentConfig(temperature=0.2, system_instruction=system)
    if schema:
        config.response_mime_type = "application/json"
        config.response_schema = schema
    r = client.models.generate_content(model="gemini-flash-latest",
                                       config=config, contents=prompt)
    return r.parsed if schema else r.text

Making reflection work

Reflection works best when it's specific. Don't ask, is this good? Ask, did you use the tool results? Is every number backed by a source? Does this meet the customer's request? Giving the reviewer a fresh context helps too, so it isn't biased by its own reasoning. A separate reviewer, or even a separate model, is stronger still.

The full thinking toolkit

Let's put it together. A model thinks before acting. For long jobs, it writes a plan. After each step, it looks at the result, and updates that plan. And before finishing, it reflects. Those four habits are what make the difference between an agent which wanders, and one that gets the job done.

Summary