Orchestrator and Workers

Some jobs can't be split up in advance. If you ask whether Velo Supply should sell e-bike batteries, nobody knows the right sub questions until they've read the request. So one model plans the work, others do it, and a final step puts it all together. That's how a manager works.

• Some jobs can't be split in advance
• One model plans
• Others do the work

How it works

The orchestrator reads the task, and decides on the subtasks. Each worker gets one small job, and a clean context. They all report back. And the results get combined into one answer. The big difference from parallelism is that the model chooses the subtasks at run time, rather than you.

Plan, work, combine

The planner returns a list of questions, using a schema, so that our code can loop over it. Each worker answers one question. They all run at the same time. Then a final call combines the notes. Run it. There are the workers, on subtasks that the model chose. And there's one final recommendation. The number of workers can change with the task. That flexibility is why this pattern powers research agents.

import asyncio
from pydantic import BaseModel
from llm import ask

class Plan(BaseModel):
    questions: list[str]

request = "Should Velo Supply start selling e-bike batteries?"

plan = ask(f"Split this into exactly three short research questions:\n{request}", schema=Plan)

async def worker(q):
    answer = await asyncio.to_thread(ask, f"Answer in one sentence: {q}")
    print("Worker:", q[:56])
    return f"{q} {answer}"

async def main():
    notes = await asyncio.gather(*[worker(q) for q in plan.questions])
    print("---")
    print(ask("Write a two-sentence recommendation from these notes:\n" + "\n".join(notes)))

asyncio.run(main())

Worker: What are the projected profit margins and costs of addin
Worker: What is the market demand for e-bike batteries among Vel
Worker: What are the regulatory, shipping, and storage requireme
---
Velo Supply should proceed with adding e-bike batteries to its catalog to capitalize on rapidly growing customer demand and attractive retail profit margins of 25% to 40%. However, this expansion must be paired with dedicated investment into compliance and facility upgrades to safely handle the strict Class 9 hazardous shipping, UN 38.3 certification, and fire-resistant storage requirements.

from google import genai
from google.genai import types

client = genai.Client()

def ask(prompt, system=None, schema=None):
    config = types.GenerateContentConfig(temperature=0.2, system_instruction=system)
    if schema:
        config.response_mime_type = "application/json"
        config.response_schema = schema
    r = client.models.generate_content(model="gemini-flash-latest",
                                       config=config, contents=prompt)
    return r.parsed if schema else r.text

Getting orchestration right

Three tips. Give each worker a clear brief, with its goal and limits, because workers only know what you tell them. Cap the number of workers, because a planner can get carried away. And keep what each worker returns short, so the final step doesn't drown in text.

Summary