AI Hints
Home›Statistics & Math for ML Tutorial›Length, Similarity and Matrices
Statistics & Math for ML · Lesson 80 of 138

Cosine Similarity

The angle between.

⏱ 2 min read▶ Video at 1:34:25🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

Two documents about football should be similar, even if one is much longer than the other. Distance would say they're far apart, just because of length. Cosine similarity ignores length, and compares only direction.

Cosine Similarity
From the video: Cosine Similarity
  • Two football articles
  • Different lengths
  • Compare direction only

Comparing word counts

Each document is a vector of word counts, for the words goal, team, recipe, and oven. Cosine similarity is the dot product, divided by both lengths. Run it. The two football documents score nearly one, even though one is three times longer. The recipe scores close to zero.

cosine.pypython
import numpy as np

def cosine(a, b):
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))

#                 goal team recipe oven
football = np.array([4,   3,   0,     0])
long_match = np.array([12, 10,  1,     0])
cake = np.array([0,   1,   5,     4])

print("football vs long match:", round(cosine(football, long_match), 3))
print("football vs cake:      ", round(cosine(football, cake), 3))
Output
football vs long match: 0.997
football vs cake:       0.093

Everywhere in AI

Cosine similarity runs search engines, recommendations, and modern AI. Language models turn sentences into vectors called embeddings, and similar meanings end up pointing in similar directions. When you search, the system finds the vectors with the highest cosine similarity to your question.

keyvalue
Formulaa · b ÷ (‖a‖ ‖b‖)
1same direction
0unrelated
Used insearch, embeddings, RAG

Summary

Cosine similarity compares direction, ignoring length. It's how AI finds similar text, images, and products. Next, matrices.

Key takeaways
  • a · b / (‖a‖‖b‖)
  • Ignores length
  • Powers embeddings
Next up: Matrices — tables of numbers