Vector Databases and Chunking

Searching a few documents is easy. Searching millions, in a fraction of a second, needs special tools. And before searching, long documents must be cut into pieces. In this lesson, you will learn both, vector databases, and chunking.

• Fast search over millions of items
• Special databases for embeddings
• Cutting documents into pieces

What a vector database does

A vector database stores embeddings, and finds the nearest ones very quickly. Instead of comparing your question with every single item, it uses clever indexes, that jump straight to the right neighbourhood. Each record holds three things. The embedding, the original text, and some extra details, called metadata, like the title, or the date. Well known examples are Pinecone, Chroma, Qdrant, Weaviate, and the pgvector extension for Postgres.

1. Embeddingthe numbers that capture meaning
2. Original textwhat to show or send to the model
3. Metadatatitle, source, date, author
4. Fast indexjumps to the nearest neighbours

Why we chunk documents

A long document has many topics. If you embed the whole thing as one point, its meaning becomes a blur, and search gets worse. So we cut documents into chunks, small pieces of a few hundred words each. Each chunk gets its own embedding, so a question can find the exact paragraph that answers it. Chunks that are too big, are vague. Chunks that are too small, lose their context.

Chunking tips

A few tips help. Aim for chunks of about two hundred to five hundred words. Split at natural boundaries, like paragraphs, and headings, instead of in the middle of a sentence. Let neighbouring chunks overlap a little, so no idea is cut in half. And store the source with every chunk, so you can show where an answer came from.

Summary