Parallel Tools and Limits

A customer asks, do you have the chain, the helmet, and the wheelset? Checking them one after another means three slow steps, with the model waiting in between. But those checks don't depend on each other. So why not do them all at once?

• Three checks, one question
• They don't depend on each other
• Do them all at once

One at a time, or together

Here, each lookup takes point four of a second. First, we run three, one after the other. Then, we use async i o to run all three at once. Run it. Three times faster, for the same work. In an agent, the waiting is nearly all network time, so running tools at once saves real seconds. Remember this only works if the tools really are independent.

import asyncio, time

async def lookup(name, seconds):
    await asyncio.sleep(seconds)
    return f"{name} done"

async def main():
    start = time.time()
    for n in ["orders", "stock", "courier"]:
        await lookup(n, 0.4)
    print(f"One at a time: {time.time() - start:.1f}s")

    start = time.time()
    results = await asyncio.gather(*[lookup(n, 0.4) for n in ["orders", "stock", "courier"]])
    print(f"All together:  {time.time() - start:.1f}s", results)

asyncio.run(main())

One at a time: 1.2s
All together:  0.4s ['orders done', 'stock done', 'courier done']

Models can ask for several

Modern models know about this. We ask about three items, and we only have one tool. Run it. Look at how many calls came back in one reply. The model requested three checks at once, rather than one per step. Your loop just needs to run them all, and return all of the results together. That's why our loop had a for statement around the calls.

from google import genai
from google.genai import types
from shop import check_stock

client = genai.Client()
config = types.GenerateContentConfig(
    tools=[check_stock],
    automatic_function_calling=types.AutomaticFunctionCallingConfig(disable=True))

r = client.models.generate_content(
    model="gemini-flash-latest", config=config,
    contents="Do you have Chain, Helmet, and Carbon wheelset in stock?")

print("Calls in one reply:", len(r.function_calls))
for c in r.function_calls:
    print(" ", c.name, dict(c.args))

Calls in one reply: 3
  check_stock {'item': 'Chain'}
  check_stock {'item': 'Helmet'}
  check_stock {'item': 'Carbon wheelset'}

ORDERS = {
    "A17": {"item": "Carbon wheelset", "price": 640, "status": "delivered", "days_ago": 12},
    "B42": {"item": "Chain", "price": 28, "status": "shipped", "days_ago": 2},
    "C09": {"item": "Helmet", "price": 95, "status": "delivered", "days_ago": 45},
}
STOCK = {"Carbon wheelset": 3, "Chain": 14, "Helmet": 0}
REFUNDS = []

def get_order(order_id: str) -> dict:
    """Look up one order by its ID, such as A17."""
    return ORDERS.get(order_id, {"error": f"No order {order_id}"})

def check_stock(item: str) -> dict:
    """Return how many of an item are in stock."""
    return {"item": item, "in_stock": STOCK.get(item, 0)}

def refund(order_id: str, amount: float) -> dict:
    """Refund money to the customer for an order."""
    REFUNDS.append((order_id, amount))
    return {"refunded": amount, "order_id": order_id}

class Scripted:
    """A stand-in model that replays a fixed list of replies."""
    def __init__(self, replies):
        self.replies = list(replies)
        self.calls = 0

    def ask(self, messages):
        self.calls += 1
        return self.replies.pop(0)

When not to go parallel

Be careful, though. Never run actions in parallel if the order matters. Reading is usually safe. Writing is not. If two tools change the same record at the same moment, one could overwrite the other. A good default is, reads in parallel, and writes one at a time.

Summary