Vector Search
Finding the closest meaning.
You've turned a million chunks into vectors. A customer asks a question. How do you find the five closest vectors in a few milliseconds? Comparing against each one would be far too slow. That's the job of a vector index.
- A million vectors
- Five closest in milliseconds
- Comparing all would be slow
FAISS in a few lines
FAISS is a fast vector search library, from Meta. We embed our four small policy files, and add them to an index. Then we search for two questions. Run it. How long until I get my money back, finds the returns file, and there's no shared wording. Are you open at the weekend finds stores. That's semantic retrieval. Real systems use approximate methods, which trade a tiny bit of accuracy for huge speed gains, and databases like pgvector, and Qdrant add filtering and storage.
docs = {
"returns.md": "Books can be returned within 30 days if unread. Refunds reach your card in 5 working days. Gift cards cannot be refunded.",
"shipping.md": "Standard delivery takes 3 to 5 days. Express takes 1 day and costs $6. We ship to Canada and the UK, but not to Australia.",
"loyalty.md": "Bright Plus members pay $20 a year, get free express delivery, and earn one point per dollar. 100 points give a $5 voucher.",
"stores.md": "Our Oslo shop opens at 9 and closes at 6, Monday to Saturday. We are closed on Sundays.",
}
import faiss
import numpy as np
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2", device="cpu")
names = list(docs)
vectors = model.encode(list(docs.values()), normalize_embeddings=True)
index = faiss.IndexFlatIP(vectors.shape[1])
index.add(vectors)
for q in ["how long until I get my money back?", "are you open at the weekend?"]:
v = model.encode([q], normalize_embeddings=True)
score, ids = index.search(v, 2)
print(q)
for s, i in zip(score[0], ids[0]):
print(f" {s:.2f} {names[i]}")A complete RAG system
Here's the whole thing, working, with a real language model. Answer embeds the question, finds the top two files, and builds a prompt with those facts. We tell the model to use only these documents, and to cite its source. We ask two questions. Run it. Sydney isn't mentioned by name, but the shipping file says that there's no delivery to Australia, and the model joined those dots, with a citation. And for coffee, it says that it doesn't know, instead of making something up. This is Scribe's biggest upgrade. It's now answering from Bright Books' real documents.
docs = {
"returns.md": "Books can be returned within 30 days if unread. Refunds reach your card in 5 working days. Gift cards cannot be refunded.",
"shipping.md": "Standard delivery takes 3 to 5 days. Express takes 1 day and costs $6. We ship to Canada and the UK, but not to Australia.",
"loyalty.md": "Bright Plus members pay $20 a year, get free express delivery, and earn one point per dollar. 100 points give a $5 voucher.",
"stores.md": "Our Oslo shop opens at 9 and closes at 6, Monday to Saturday. We are closed on Sundays.",
}
import faiss
from google import genai
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2", device="cpu")
names, texts = list(docs), list(docs.values())
vectors = model.encode(texts, normalize_embeddings=True)
index = faiss.IndexFlatIP(vectors.shape[1])
index.add(vectors)
client = genai.Client()
def answer(question, k=2):
v = model.encode([question], normalize_embeddings=True)
_, ids = index.search(v, k)
context = "\n".join(f"[{names[i]}] {texts[i]}" for i in ids[0])
prompt = (f"Answer using only these documents. Cite the file in brackets. "
f"If the answer is not there, say you do not know.\n\n"
f"{context}\n\nQuestion: {question}")
return client.models.generate_content(model="gemini-flash-latest",
contents=prompt).text.strip()
print(answer("Can you deliver to Sydney?"))
print(answer("Do you sell coffee?"))Summary
Embed, search, and answer, with citations. You've just built what many companies are paying for. But simple RAG fails in real use. It misses exact codes and names. Next, how to make retrieval strong.
Key takeaways- FAISS finds the closest vectors
- Prompt with the retrieved facts
- Cite sources, allow I don't know