Workshop: A Document Q and A Bot

Now you will build something real, a bot that answers questions from your own documents. We will go step by step, from loading text, to chunking, to retrieval, to a grounded answer. You will see exactly where each idea from the course appears.

• Load and split documents
• Find the best chunks
• Answer from the sources

Step one: load and chunk

Step one. We load our document, and split it into chunks. Here, we split on blank lines, so each paragraph becomes one chunk. We also keep the chunk number, so that we can cite the source later. For real projects, you would use a tool that splits by size, with a little overlap, but the idea is the same.

text = open("policy.txt").read()

chunks = [c.strip() for c in text.split("\n\n") if c.strip()]
for i, c in enumerate(chunks):
    print(i, c[:60])

0 Returns: you can return online orders within 30 days of deli
1 Shipping: standard delivery takes 3 to 5 working days.
2 Gift cards: gift cards never expire and cannot be exchanged

Returns: you can return online orders within 30 days of delivery.

Shipping: standard delivery takes 3 to 5 working days.

Gift cards: gift cards never expire and cannot be exchanged for cash.

Step two: embed and search

Step two. We turn each chunk into an embedding, and search. To keep this lesson simple, and runnable, we use a very small embedding, a count of the words. It is not as clever as a real model, but it shows the idea exactly. We score each chunk against the question, with cosine similarity, and pick the best one. In a real project, only the embedding function changes.

import math
from collections import Counter

chunks = ["Returns: you can return online orders within 30 days of delivery.",
          "Shipping: standard delivery takes 3 to 5 working days.",
          "Gift cards: gift cards never expire."]

def embed(t):
    return Counter(t.lower().replace(":", "").replace(".", "").split())

def cosine(a, b):
    dot = sum(a[w] * b[w] for w in a)
    return dot / (math.sqrt(sum(v*v for v in a.values())) * math.sqrt(sum(v*v for v in b.values())))

q = embed("how many days do I have to return online orders")
scores = [(round(cosine(q, embed(c)), 2), c[:30]) for c in chunks]
print(sorted(scores, reverse=True)[0])

(0.38, 'Returns: you can return online')

Step three: build the prompt

Step three, the augmented prompt. We take the best chunk, and place it inside the prompt, with a firm instruction. Answer only using the context below. If the answer is not there, say, I could not find that. We also add the question. This single prompt is what we would send to the model, through the A P I.

best = "Returns: you can return online orders within 30 days of delivery."
question = "How long do I have to return an order?"

prompt = f"""Answer only using the context below.
If the answer is not in the context, say: I could not find that.

Context:
{best}

Question: {question}"""
print(prompt)

Answer only using the context below.
If the answer is not in the context, say: I could not find that.

Context:
Returns: you can return online orders within 30 days of delivery.

Question: How long do I have to return an order?

Step four and beyond

Step four, send that prompt to the model, and show the answer, with the source. Then improve. Swap in a real embedding model, and a vector database. Retrieve the top three chunks, instead of one. Add a re ranking step. Build a test set of questions, and measure. You have just built the core of almost every document assistant, in use today.

Summary