A Coding Agent

Coding agents are the biggest success story in this field. Developers now hand them whole features. Why does this work so well? Because code has something that most jobs don't. An honest judge. A test either passes, or it fails. The agent can't talk its way out of that.

• Coding agents are the biggest success
• Why does it work so well?
• Tests are an honest judge

The coding loop

Here's the loop that powers every coding agent. Read the code. Make a change. Run the tests. Read the errors. And go around again. The tests provide truthful feedback, on every loop. That's why tasks with good tests work so well with agents.

1. Readunderstand the code
2. Changeedit a file
3. Run teststhe honest judge
4. Read errorsreal feedback
5. Repeatuntil green

An agent that fixes its own code

Here's a tiny coding agent. We've got three tests, including an awkward one, with accented letters. The model writes a function. We run it along with the tests, in a separate process. If it fails, we send the error back, and we try again. Run it. It passed on the first attempt this time. On harder tasks, it won't, and that's when the loop earns its keep. The error goes straight back to the model, and it keeps going until the tests pass. That's the whole idea behind coding agents, in twenty lines.

import re, subprocess, sys
from llm import ask

TESTS = """
assert slug("Carbon Wheelset 700c") == "carbon-wheelset-700c"
assert slug("  Hello,   World!! ") == "hello-world"
assert slug("Crème brûlée") == "creme-brulee"
print("All tests passed")
"""

task = "Write a Python function slug(text) that makes URL slugs. Return only code."
for attempt in range(1, 5):
    code = ask(task)
    code = re.sub(r"^```\w*\n|```\s*$", "", code.strip(), flags=re.M)
    r = subprocess.run([sys.executable, "-c", code + TESTS],
                       capture_output=True, text=True, timeout=10)
    if r.returncode == 0:
        print(f"Attempt {attempt}: {r.stdout.strip()}")
        break
    error = r.stderr.strip().splitlines()[-1]
    print(f"Attempt {attempt}: {error[:60]}")
    task += f"\nYour last code failed with: {error}\nCode:\n{code}"

Attempt 1: All tests passed

from google import genai
from google.genai import types

client = genai.Client()

def ask(prompt, system=None, schema=None):
    config = types.GenerateContentConfig(temperature=0.2, system_instruction=system)
    if schema:
        config.response_mime_type = "application/json"
        config.response_schema = schema
    r = client.models.generate_content(model="gemini-flash-latest",
                                       config=config, contents=prompt)
    return r.parsed if schema else r.text

Editing, not rewriting

Real agents don't rewrite whole files. That'd be slow, and risky. They use exact find and replace, which is much safer. Here's an edit tool. The text must match exactly once, or it refuses. That stops it changing the wrong thing. Run it. The word prices appears twice, so it was refused, with a clear error. The right edit worked. And a line that doesn't exist was refused. A narrow tool like this prevents a whole class of mistakes.

import pathlib

file = pathlib.Path("cart.py")
file.write_text("def total(prices):\n    return sum(prices) * 1.2\n")

def edit(old, new):
    text = file.read_text()
    if text.count(old) != 1:
        return f"Error: found {text.count(old)} matches, need exactly 1"
    file.write_text(text.replace(old, new))
    return "Edited"

print(edit("prices", "items"))
print(edit("1.2", "1.25"))
print(edit("banana", "pear"))
print(file.read_text())

Error: found 2 matches, need exactly 1
Edited
Error: found 0 matches, need exactly 1
def total(prices):
    return sum(prices) * 1.25

Summary