Seeing High Dimensions

We can't see in sixty four dimensions. But what if we could draw a flat map, where points that are close in the real space stay close on the map? Two clever algorithms do exactly this. They have become some of the most popular tools for exploring data.

• We can't see 64 dimensions
• Draw a faithful flat map
• Neighbours stay neighbours

Two maps of the digits

We take a thousand digit images, and draw each as one dot, in two ways. P C A on the left, and T S N E on the right. Colour shows the true digit, but neither method saw the labels. Run it. P C A gives a messy overlap, because two straight directions cannot hold it all. But look at the map on the right. The digits form clear, separate islands. Nobody told it what digits are. It found that structure by itself.

import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE

X, y = load_digits(return_X_y=True)
X, y = X[:1000], y[:1000]

pca2 = PCA(n_components=2).fit_transform(X)
tsne = TSNE(n_components=2, init="pca", random_state=0).fit_transform(X)

fig, ax = plt.subplots(1, 2, figsize=(10, 4.5))
ax[0].scatter(*pca2.T, c=y, cmap="tab10", s=8)
ax[0].set_title("PCA")
ax[1].scatter(*tsne.T, c=y, cmap="tab10", s=8)
ax[1].set_title("t-SNE")
plt.tight_layout()
plt.savefig("maps.png", dpi=120)
print("Each image: 64 numbers. Each dot: 2 numbers.")

Each image: 64 numbers. Each dot: 2 numbers.

How to read these maps

These maps are powerful, but they lie in specific ways. Cluster sizes mean nothing, and neither do distances between far-apart clusters. Only the neighbours can be trusted. Different settings can give very different pictures. So use them to explore and to get ideas, never as proof. U M A P is a newer alternative, which is often faster, and keeps more of the global structure.

Summary