Vectors and Matrices

You've built real models, and you've done it with almost no maths. That was on purpose. Now that you've seen what the ideas do, the maths will feel like a friend, and not like a wall. You need much less than people say. Three short lessons will do it.

• You've got this far already
• Now maths will feel like a friend
• Three short lessons

Vectors and matrices

A vector is just a list of numbers, such as one house, described by its features. A matrix is a table of numbers, with one row per example. That's all your data is. The most important operation is the matrix product. It multiplies each row by the weights, and adds up the results. So a single line of maths can make predictions for a million houses at once. Most machine learning is built on that one operation.

Predictions as matrix maths

Here's our little rooms dataset, written as a matrix. The first column is all ones, which makes room for the start value. W holds the start and the slope. The at sign is the matrix product. In one symbol, we get all five predictions. Now the last part is magic. For linear regression, there's an exact formula, so we can get the best weights with no gradient descent at all. Run it. It gives the same answer that took us five thousand steps, earlier in the course.

import numpy as np

X = np.array([[1, 3.0], [1, 4.0], [1, 5.0], [1, 6.0], [1, 7.0]])
y = np.array([1.5, 1.9, 2.4, 2.8, 3.3])
w = np.array([0.13, 0.45])

print("X shape:", X.shape, "  w shape:", w.shape)
print("Predictions:", (X @ w).round(2))

best = np.linalg.inv(X.T @ X) @ X.T @ y
print("Exact answer:", best.round(3))

X shape: (5, 2)   w shape: (2,)
Predictions: [1.48 1.93 2.38 2.83 3.28]
Exact answer: [0.13 0.45]

Summary