AI Hints
Home›Statistics & Math for ML Tutorial›Length, Similarity and Matrices
Statistics & Math for ML · Lesson 79 of 138

Length and Distance

Norms.

⏱ 2 min read▶ Video at 1:33:14🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

How long is a vector? And how far apart are two data points? These questions power recommendation systems, clustering, and nearest neighbour models. The answer is the norm.

Length and Distance
From the video: Length and Distance
  • How long is a vector?
  • How far apart are two points?
  • The norm

Pythagoras in any dimension

The length of a vector is the square root of the sum of its squared values. For the vector three, four, that's five, just like Pythagoras. The distance between two points is the length of their difference. Run it. Lin alg dot norm does it all.

norm.pypython
import numpy as np

v = np.array([3, 4])
print("length:", np.sqrt((v ** 2).sum()), np.linalg.norm(v))

a = np.array([1, 2, 3])
b = np.array([4, 6, 3])
print("distance:", np.linalg.norm(a - b))
Output
length: 5.0 5.0
distance: 5.0

Two common norms

The usual length is called the L two norm, or Euclidean distance, the straight line. The L one norm adds up the absolute values instead, like walking along city blocks, so it's called Manhattan distance. Both appear in machine learning, especially in regularization, which we'll meet later.

L2: Euclidean

  • √(Σ x²)straight linenp.linalg.norm(v)

L1: Manhattan

  • Σ |x|city blocksnp.linalg.norm(v, 1)

Summary

A vector's length is its norm, and distance is the norm of the difference. L two is the straight line, and L one is city blocks. Next, cosine similarity.

Key takeaways
  • np.linalg.norm
  • distance = ‖a − b‖
  • L1 and L2
Next up: Cosine Similarity — the angle between