Embeddings Everywhere
Meaning as a number list.
One of the most useful things a neural network gives you isn't its final answer. It's the second-last layer. A list of numbers that captures what something means. Those lists now power search engines, recommendation systems, and the memory of A I assistants.
- The second-last layer
- A list of numbers for meaning
- Powers modern search
Search by meaning
We load a small pretrained language model, and we turn sentences into lists of numbers. Then we compare a question with three documents, using cosine similarity. Run it. Look at the top match. It's about changing login details, and it doesn't contain the words reset, or password. Keyword search would've missed it completely. The network matched the meaning. This is called semantic search, and it's the core of R A G systems.
from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer("all-MiniLM-L6-v2", device="cpu")
query = "How do I reset my password?"
docs = ["Steps to change your login details",
"Our office opens at nine",
"The best pizza in Naples"]
scores = util.cos_sim(model.encode(query), model.encode(docs))[0]
for doc, s in sorted(zip(docs, scores.tolist()), key=lambda p: -p[1]):
print(f"{s:.3f} {doc}")0.520 Steps to change your login details 0.066 Our office opens at nine 0.038 The best pizza in Naples
Summary
Embeddings turn meaning into numbers that you can compare. Build one good embedding, and you can reuse it for search, clustering, and recommendations. Next, a quick tour of some other data types that deep learning handles, such as graphs and sound.
Key takeaways- Embeddings capture meaning
- Cosine similarity compares them
- Semantic search beats keywords