Long-Term Memory

A great barista remembers your order after a few visits. They don't remember every conversation, just the bits that matter. That's what long term memory is. You don't store everything. You choose the facts that will still be useful later.

• A great barista remembers
• Not every conversation
• Just the bits that matter

Extracting lasting facts

At the end of a conversation, a small model call pulls out what's worth keeping. Here, the customer tells us three things. They ride carbon rims. They want email only. And they ask about closing time. Which of those will matter next month? Run it. It kept the two lasting facts, and it skipped the one off question. That's the skill. Good memory is mostly about what you leave out.

import json
from pydantic import BaseModel
from llm import ask

class Facts(BaseModel):
    facts: list[str]

chat = """Customer: I only ride carbon rims, so I need the right brake pads.
Customer: Also, please never call me. Email only.
Customer: Anyway, what time do you close today?"""

out = ask("List lasting facts about this customer. Skip one-off questions.\n" + chat, schema=Facts)

profile = {"priya@example.com": out.facts}
print(json.dumps(profile, indent=2))

{
  "priya@example.com": [
    "Only rides carbon rims",
    "Prefers communication by email only and should not be called"
  ]
}

from google import genai
from google.genai import types

client = genai.Client()

def ask(prompt, system=None, schema=None):
    config = types.GenerateContentConfig(temperature=0.2, system_instruction=system)
    if schema:
        config.response_mime_type = "application/json"
        config.response_schema = schema
    r = client.models.generate_content(model="gemini-flash-latest",
                                       config=config, contents=prompt)
    return r.parsed if schema else r.text

A simple memory store

The store itself can be simple. Remember saves a fact for a customer, and it skips any duplicates. Recall gets them back. Here it's a JSON file, and in production, it would be a database. Run it. Two facts, and the duplicate was ignored. At the start of each conversation, those facts get loaded into the system prompt. Now Priya gets remembered.

import json, pathlib, time

FILE = pathlib.Path("memory.json")

def remember(customer, fact):
    data = json.loads(FILE.read_text()) if FILE.exists() else {}
    items = data.setdefault(customer, [])
    if fact not in [i["fact"] for i in items]:
        items.append({"fact": fact, "saved": round(time.time())})
    FILE.write_text(json.dumps(data))

def recall(customer):
    data = json.loads(FILE.read_text()) if FILE.exists() else {}
    return [i["fact"] for i in data.get(customer, [])]

remember("priya", "Rides carbon rims")
remember("priya", "Email only, never call")
remember("priya", "Rides carbon rims")
print(recall("priya"))

['Rides carbon rims', 'Email only, never call']

Memory rules

Long term memory needs care. Customers must be able to see and delete what you hold, and in many countries, that's the law. Facts go stale, so store dates, and prefer recent ones. Old facts can conflict with new ones, so update rather than just adding. And never store secrets, such as passwords or card numbers.

Summary