Conversation Memory

Priya writes to Juno on Monday about her wheelset. On Tuesday, she writes again, and Juno asks her to explain it all from the start. She's furious. Nothing frustrates a customer more than repeating themselves. So memory isn't a nice extra. It's the difference between a tool and an assistant.

• Priya explains on Monday
• Juno forgets by Tuesday
• Memory isn't a nice extra

Four kinds of memory

Agents use four layers of memory. Working memory is the current conversation. A summary keeps older turns in a compressed form. A long term store holds facts about a customer, across sessions. And a knowledge base holds documents to be searched when needed. Only the top layer sits in context all the time. The rest gets fetched when it's needed.

A sliding window

Let's begin with the simplest fix. Keep everything in storage, but send only the last few messages. Here we keep ten messages, and send four. Run it. All ten are stored, and only four go to the model. It's cheap, and it never overflows. The problem? Priya's name was in message one. It's gone now. A sliding window has no sense of what's important.

class Memory:
    def __init__(self, limit=4):
        self.messages, self.limit = [], limit

    def add(self, role, text):
        self.messages.append(f"{role}: {text}")

    def window(self):
        return self.messages[-self.limit:]

m = Memory(limit=4)
for i in range(1, 6):
    m.add("customer", f"message {i}")
    m.add("juno", f"reply {i}")

print("Stored:", len(m.messages))
print("Sent to the model:", m.window())

Stored: 10
Sent to the model: ['customer: message 4', 'juno: reply 4', 'customer: message 5', 'juno: reply 5']

Summary