AI Hints
Statistics & Math for ML · Lesson 22 of 138

Standardizing Features

Mean 0, std 1.

⏱ 2 min read▶ Video at 24:16🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

A model sees two features. Age, from twenty to sixty, and income, from twenty thousand to two hundred thousand. Income's numbers are thousands of times bigger, so many models will think it's far more important. We need to put features on the same scale.

Standardizing Features
From the video: Standardizing Features
  • Age: 20 to 60
  • Income: 20k to 200k
  • Different scales confuse models

Turning every value into a z score

Standardizing replaces every value with its z score. Subtract the column's mean, and divide by its standard deviation. Axis zero means, do it for each column. Run it. Both columns now have a mean of zero and a standard deviation of one.

standardize.pypython
import numpy as np

X = np.array([[25, 30_000], [35, 52_000], [45, 80_000],
              [52, 120_000], [60, 190_000]], dtype=float)

Z = (X - X.mean(axis=0)) / X.std(axis=0)

print(Z.round(2))
print("means:", Z.mean(axis=0).round(2) + 0)
print("stds: ", Z.std(axis=0).round(2))
Output
[[-1.49 -1.14]
 [-0.68 -0.75]
 [ 0.13 -0.25]
 [ 0.7   0.45]
 [ 1.35  1.69]]
means: [0. 0.]
stds:  [1. 1.]

With scikit-learn

In practice, you'd use scikit learn's standard scaler. Fit learns the mean and standard deviation, and transform applies them. Fit transform does both at once. Run it. The same answer. Remember to fit only on your training data, and reuse that scaler on new data.

scaler.pypython
import numpy as np
from sklearn.preprocessing import StandardScaler

X = np.array([[25, 30_000], [35, 52_000], [45, 80_000],
              [52, 120_000], [60, 190_000]], dtype=float)

scaler = StandardScaler()
print(scaler.fit_transform(X).round(2))
print("learned means:", scaler.mean_.tolist())
Output
[[-1.49 -1.14]
 [-0.68 -0.75]
 [ 0.13 -0.25]
 [ 0.7   0.45]
 [ 1.35  1.69]]
learned means: [43.4, 94400.0]

Summary

Standardizing turns each feature into z scores, with mean zero and standard deviation one. Scikit learn's standard scaler does it for you. Next, another way to scale, min max scaling.

Key takeaways
  • (x − mean) / std
  • StandardScaler
  • Fit on training data only
Next up: Min-Max Scaling — squeeze into 0 to 1