The Simplest Model

Welcome to your first real algorithm. Linear regression is more than two hundred years old, and it is still used every day at the biggest companies in the world. It is also the perfect teacher, because almost every idea in machine learning, including the giant neural networks behind A I chatbots, appears here in a tiny, understandable form.

• Two hundred years old
• Still used everywhere
• The same ideas power neural networks

A line is a model

A straight line has just two numbers. The slope, which says how much the price rises for each extra room. And the start, which is where the line begins. That is the whole model, two numbers. These numbers have a name. We call them parameters. Prediction is simple. Take the rooms, multiply by the slope, and add the start. Training is the search for the best two numbers.

A prediction by hand

Let's write it ourselves, no libraries. Predict is one line, slope times x, plus start. Here are five homes, with their rooms and prices. We try a slope of point three, and a start of point five, purely as a guess. We print our guesses, the true prices, and the errors between them. Our guess is not terrible, but every error is negative. Our line sits too low. How do we measure just how bad it is, with a single number?

import numpy as np

def predict(x, slope, start):
    return slope * x + start

rooms = np.array([3, 4, 5, 6, 7])
price = np.array([1.5, 1.9, 2.4, 2.8, 3.3])

guess = predict(rooms, slope=0.3, start=0.5)
print("Guess: ", guess.round(2))
print("Actual:", price)
print("Errors:", (guess - price).round(2))

Guess:  [1.4 1.7 2.  2.3 2.6]
Actual: [1.5 1.9 2.4 2.8 3.3]
Errors: [-0.1 -0.2 -0.4 -0.5 -0.7]

Summary