AI Hints
Home›Statistics & Math for ML Tutorial›Solving and Eigenvectors
Statistics & Math for ML · Lesson 87 of 138

Eigenvectors

Directions that don't turn.

⏱ 2 min read▶ Video at 1:42:27🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

When a matrix transforms space, most vectors get turned to point somewhere new. But a few special directions don't turn at all. They only get stretched or shrunk. Those are eigenvectors, and the stretch amount is the eigenvalue.

Eigenvectors
From the video: Eigenvectors
  • Most vectors turn
  • Some only stretch
  • Eigenvectors and eigenvalues

Finding them

Lin alg dot EIGH is for symmetric matrices. It returns the eigenvalues from smallest to largest, and the eigenvectors as columns. We take the last eigenvector, V, and check that A times V equals the eigenvalue times V. Run it. Multiplying by A just stretched V by three. Its direction didn't change.

eig.pypython
import numpy as np

A = np.array([[2.0, 1.0],
              [1.0, 2.0]])

values, vectors = np.linalg.eigh(A)
print("eigenvalues:", values)

v = vectors[:, -1]
print("A @ v:", (A @ v).round(3))
print("3 * v:", (3 * v).round(3))
Output
eigenvalues: [1. 3.]
A @ v: [2.121 2.121]
3 * v: [2.121 2.121]

Why eigenvectors matter

For a covariance matrix, the eigenvectors point along the directions where the data spreads out most, and the eigenvalues say how much it spreads. That's the key to PCA. Eigenvectors also appear in Google's original PageRank algorithm, and in how we analyse networks.

keyvalue
A v = λ vonly stretched
λthe eigenvalue
Covariance matrixdirections of spread
Used inPCA, PageRank

Summary

Eigenvectors are directions a matrix only stretches, and eigenvalues are the stretch amounts. Next, using them to simplify data, with PCA.

Key takeaways
  • np.linalg.eigh
  • A v = λ v
  • Directions of spread
Next up: PCA — the main directions