AI Hints
Statistics & Math for ML · Lesson 70 of 138

Simple Linear Regression

The best line.

⏱ 2 min read▶ Video at 1:22:20🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

Correlation says hours and scores move together. Regression goes further. It draws the best straight line through the points, so we can predict a score from the hours studied. It's the simplest machine learning model of all.

Simple Linear Regression
From the video: Simple Linear Regression
  • Correlation says they're linked
  • Regression draws the line
  • The simplest ML model

Fitting a line

A line is Y equals MX plus B, where M is the slope, and B is the intercept. NP dot polyfit with degree one finds the best M and B. Run it. Each extra hour of study adds almost six points. We draw the points, and the line.

fit_line.pypython
import numpy as np
import matplotlib.pyplot as plt
from study import hours, scores

m, b = np.polyfit(hours, scores, 1)
plt.scatter(hours, scores, s=80)
plt.plot(hours, m * hours + b, color="#22d3ee")
plt.title(f"score = {m:.2f} x hours + {b:.1f}")
plt.savefig("line.png")
Also uses study.py
import numpy as np

hours = np.array([1, 2, 3, 4, 5, 6, 7, 8])
scores = np.array([42, 50, 53, 61, 64, 72, 74, 85])

Making predictions

Now we can predict. For a student who studies four and a half hours, plug it into the line. Run it. About sixty two points. But be careful predicting twenty hours, far outside our data. The line would promise over a hundred and fifty points, which is impossible.

predict.pypython
import numpy as np
from study import hours, scores

m, b = np.polyfit(hours, scores, 1)
for h in [4.5, 20]:
    print(f"{h} hours -> {m * h + b:.1f} points")
Output
4.5 hours -> 62.6 points
20 hours -> 151.4 points
Also uses study.py
import numpy as np

hours = np.array([1, 2, 3, 4, 5, 6, 7, 8])
scores = np.array([42, 50, 53, 61, 64, 72, 74, 85])

Summary

Linear regression finds the best line, Y equals MX plus B, and uses it to predict. Don't trust it far outside your data. Next, what best actually means.

Key takeaways
  • y = mx + b
  • np.polyfit(x, y, 1)
  • Don't extrapolate far
Next up: Least Squares — minimizing squared errors