PCA in Practice
Fewer features.
Scikit learn comes with a dataset of handwritten digits. Each image is eight by eight pixels, so sixty four features. Can we squash sixty four numbers down to just a few, and still keep most of the information?

- Handwritten digits
- 64 pixel features
- Squash them down?
Explained variance
We fit PCA from scikit learn on the digits, keeping ten components. Explained variance ratio says what share of the total variance each component captures. Run it. Just ten components keep about three quarters of the information in all sixty four pixels.
import numpy as np
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
X, y = load_digits(return_X_y=True)
pca = PCA(n_components=10).fit(X)
print("features:", X.shape[1])
print("first 3:", pca.explained_variance_ratio_[:3].round(3))
print("10 components keep:", pca.explained_variance_ratio_.sum().round(3))features: 64 first 3: [0.149 0.136 0.118] 10 components keep: 0.738
Seeing sixty four dimensions
With just two components, we can draw every image as a dot, coloured by which digit it is. Run it. Even in two dimensions, the digits form clear groups. PCA is a great first look at any high dimensional data.
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
X, y = load_digits(return_X_y=True)
Z = PCA(n_components=2).fit_transform(X)
plt.scatter(Z[:, 0], Z[:, 1], c=y, cmap="tab10", s=8)
plt.colorbar(label="digit"); plt.title("Digits in 2 dimensions")
plt.savefig("digits_pca.png")Summary
PCA compresses many features into a few components that keep most of the variance. It's great for visualizing and speeding up models. Next, a linear algebra quiz.
Key takeaways- PCA(n_components=k)
- explained_variance_ratio_
- Plot in 2D