Follow-Up Questions

Making sense of it.

⏱ 2 min read▶ Video at 2:22:54🎓 Free

Watch this lesson in the NLP courseWatch on YouTube ↗Full NLP course

A customer asks about their order for Dune. Then they ask, can I still change it? If you search your documents for, can I still change it, you'll find nothing useful. What's it? People talk in shorthand. Search engines don't understand that.

A customer asks about Dune
'Can I still change it?'
What's 'it'?

Rewriting the question

Before searching, we ask a language model to rewrite the question so that it stands alone, using the conversation so far. Run it. Can I still change it, becomes, can I still change my order for Dune? Now search can find the right policy. It's one small and cheap call, and it fixes a huge share of failures in chat based RAG systems. Use a small, fast model for this, as it runs on each turn.

rewrite.pypython

from llm import ask

history = "Customer: I ordered Dune last week.\nAgent: Yes, order AB1234.\n"
question = "can I still change it?"

search = ask(f"Rewrite the last question so that it makes sense alone. Return only the question.\n{history}Customer: {question}")
print("Before:", question)
print("After: ", search)

Output

Before: can I still change it?
After:  Can I still change order AB1234 for Dune?

Also uses tickets.py

TICKETS = [
    ("my order arrived late and the box was crushed", "delivery"),
    ("the parcel never came, where is my delivery", "delivery"),
    ("courier left the package in the rain", "delivery"),
    ("tracking has not moved for a week", "delivery"),
    ("shipping took three weeks, far too slow", "delivery"),
    ("delivery driver was rude and threw the parcel", "delivery"),
    ("package is lost, nobody knows where it went", "delivery"),
    ("the courier says delivered but nothing arrived", "delivery"),
    ("i want my money back for this book", "refund"),
    ("please refund the second copy you charged me for", "refund"),
    ("you took payment twice, return my money", "refund"),
    ("how long does a refund take to reach my card", "refund"),
    ("cancel my order and give me a refund", "refund"),
    ("i returned the book but got no money back", "refund"),
    ("charged the wrong price, want the difference back", "refund"),
    ("refund has not appeared on my statement", "refund"),
    ("the book has missing pages in chapter four", "quality"),
    ("cover is torn and the spine is broken", "quality"),
    ("pages are printed upside down in the middle", "quality"),
    ("ink is smudged on every page", "quality"),
    ("the binding fell apart after one read", "quality"),
    ("this copy has a coffee stain on the cover", "quality"),
    ("wrong edition, and the print quality is awful", "quality"),
    ("several pages are stuck together badly", "quality"),
]
QUERIES = [
    ("where is my parcel, it has not arrived", "delivery"),
    ("i need my money back please", "refund"),
    ("the pages are torn and the cover is bent", "quality"),
    ("courier lost my package", "delivery"),
    ("refund the payment to my card", "refund"),
    ("print is smudged and hard to read", "quality"),
]

Also uses llm.py

from google import genai
from google.genai import types

client = genai.Client()

def ask(prompt, system=None, schema=None, temperature=0.1):
    config = types.GenerateContentConfig(temperature=temperature, system_instruction=system)
    if schema:
        config.response_mime_type = "application/json"
        config.response_schema = schema
    r = client.models.generate_content(model="gemini-flash-latest", config=config, contents=prompt)
    return r.parsed if schema else r.text

Summary

Rewrite before you search. Next, how to get the very best result to the top.

Key takeaways

Follow-ups depend on context
Rewrite them to stand alone
Use a small, fast model

Next up: Reranking — the best result goes first

❮ An FAQ Bot Reranking ❯