Least Squares
Minimizing squared errors.
What makes one line better than another? For each point, the line makes an error, the gap between the real score and the predicted one. The best line is the one that makes the total of the squared errors as small as possible. That's called least squares.

- Every line makes errors
- Square them, add them up
- The smallest total wins
Comparing lines
We write a function that computes the sum of squared errors for any slope and intercept. Then we try a few guesses, and the least squares line. Run it. The polyfit line has the smallest error. No other line can beat it.
import numpy as np
from study import hours, scores
def sse(m, b):
return ((scores - (m * hours + b)) ** 2).sum()
print("guess 1:", sse(5, 40))
print("guess 2:", sse(6, 35))
m, b = np.polyfit(hours, scores, 1)
print("best: ", sse(m, b).round(1))guess 1: 45 guess 2: 29 best: 22.7
Also uses study.py
import numpy as np
hours = np.array([1, 2, 3, 4, 5, 6, 7, 8])
scores = np.array([42, 50, 53, 61, 64, 72, 74, 85])The formula
For one feature, there's a neat formula. The slope is the covariance of X and Y, divided by the variance of X. And the intercept makes the line pass through the point of the two means. Later in the course, we'll find the same line with gradient descent, the way neural networks learn.
| key | value |
|---|---|
| Error | actual − predicted |
| Least squares | minimize Σ error² |
| Slope | cov(x, y) ÷ var(x) |
| Intercept | ȳ − slope × x̄ |
Summary
Least squares picks the line with the smallest sum of squared errors. The slope is covariance over variance. Next, measuring how good the line is.
Key takeaways- Minimize Σ error²
- slope = cov / var
- Line passes through the means