Matrices as Transformations
Stretch and rotate.
Here's a beautiful way to think about matrices. Multiplying a vector by a matrix moves it. A matrix can stretch space, squash it, rotate it, or flip it. Every layer of a neural network is a transformation like this.

- A matrix moves vectors
- Stretch, squash, rotate
- Neural network layers
Transforming a square
We take the corners of a square, and multiply them by two matrices. One stretches X by two. The other rotates by forty five degrees. Run it. The pink square becomes a wide rectangle, and a tilted diamond.
import numpy as np
import matplotlib.pyplot as plt
sq = np.array([[0, 0], [1, 0], [1, 1], [0, 1], [0, 0]]).T
stretch = np.array([[2, 0], [0, 1]])
c = np.cos(np.pi / 4)
rotate = np.array([[c, -c], [c, c]])
for M, col in [(np.eye(2), "#e879f9"), (stretch, "#fbbf24"), (rotate, "#22d3ee")]:
plt.plot(*(M @ sq), color=col)
plt.gca().set_aspect("equal"); plt.title("Square, stretched, rotated")
plt.savefig("transform.png")Why this view helps
Thinking of matrices as transformations explains a lot. A neural network layer transforms data into a new space, where the classes are easier to separate. PCA, which we'll see soon, finds a rotation that lines data up with its most important directions. And the inverse of a matrix undoes its transformation.
| key | value |
|---|---|
| Stretch | scale an axis |
| Rotate | turn space |
| Neural layer | transform, then bend |
| Inverse | undo it |
Summary
A matrix transforms space, stretching, rotating, or flipping vectors. Next, the identity and the inverse.
Key takeaways- M @ v moves v
- Stretch and rotate
- Layers transform data