Measuring Mistakes

You cannot improve what you cannot measure. A model needs one single number that says how wrong it is. That number is called the loss, and choosing it is one of the most important decisions in machine learning. Every model you will ever train is simply trying to push this one number down.

• One number says how wrong
• It is called the loss
• All training pushes it down

Why we square the errors

Here is how we build it. Take each error, the guess minus the truth. We cannot just add them, because positive and negative errors would cancel out, and a terrible model could score zero. So we square each error. Now they are all positive, and something useful happens. Big mistakes get punished far more than small ones. An error of two counts as four, and an error of ten counts as one hundred. Then we take the average. This is the mean squared error, or M S E.

Scoring three lines

Here is M S E in code. Mse takes a slope and a start, makes the guesses, squares the errors, and takes the mean. Now we score three different lines. Run it. The first line has the highest error. The second is much better. And the third is nearly perfect. We now have a way to compare any two lines. So what we really want is a method that finds the lowest possible error, automatically.

import numpy as np

price = np.array([1.5, 1.9, 2.4, 2.8, 3.3])
rooms = np.array([3, 4, 5, 6, 7])

def mse(slope, start):
    guess = slope * rooms + start
    return ((guess - price) ** 2).mean()

for slope, start in [(0.3, 0.5), (0.4, 0.2), (0.45, 0.13)]:
    print(f"slope {slope:.2f} start {start:.2f} -> error {mse(slope, start):.4f}")

slope 0.30 start 0.50 -> error 0.1900
slope 0.40 start 0.20 -> error 0.0380
slope 0.45 start 0.13 -> error 0.0006

Summary