AI Hints
Home›Statistics & Math for ML Tutorial›Residuals and Many Features
Statistics & Math for ML · Lesson 74 of 138

Multiple Regression

Many features.

⏱ 2 min read▶ Video at 1:27:05🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

House prices don't depend on just one thing. Size, number of rooms, and age all matter. Multiple regression uses many features at once. Each feature gets its own weight, called a coefficient.

Multiple Regression
From the video: Multiple Regression
  • Size, rooms, age
  • Many features at once
  • A weight for each

LinearRegression

We simulate five hundred houses, with prices in thousands. The true formula is two per square metre, plus ten per room, minus one and a half per year of age. Scikit learn's linear regression learns the weights from the data. Run it. It recovers the true weights closely.

multiple.pypython
import numpy as np
from sklearn.linear_model import LinearRegression
from houses import X, price

model = LinearRegression().fit(X, price)

for name, w in zip(["size", "rooms", "age"], model.coef_):
    print(f"{name:6s} {w:6.2f}")
print("intercept:", round(model.intercept_, 1))
Output
size     2.03
rooms    9.51
age     -1.52
intercept: 47.0
Also uses houses.py
import numpy as np

rng = np.random.default_rng(0)
n = 500
size = rng.uniform(50, 250, n)
rooms = np.round(size / 30 + rng.normal(0, 1, n)).clip(1)
age = rng.uniform(0, 60, n)
price = 2.0 * size + 10 * rooms - 1.5 * age + 50 + rng.normal(0, 25, n)
X = np.column_stack([size, rooms, age])

Reading coefficients

Each coefficient is the change in price for one more unit of that feature, holding all the others fixed. One more room, with the same size and age, adds about ten thousand. That holding fixed part is important. It's what lets multiple regression separate the effects of features that are linked, like size and rooms.

keyvalue
Coefficienteffect of +1 unit
Holding others fixedthe key idea
Scale firstto compare weights
sklearnLinearRegression().fit

Summary

Multiple regression gives every feature its own coefficient, the effect of that feature with the others held fixed. Next, a quiz on regression.

Key takeaways
  • LinearRegression
  • coef_ and intercept_
  • Others held fixed
Next up: Regression Quiz — test yourself