AI Hints
Statistics & Math for ML · Lesson 122 of 138

Correlation Matrices

Many features at once.

⏱ 2 min read▶ Video at 2:25:56🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

A dataset with ten features has forty five pairs of features. Checking each correlation one by one would be slow. A correlation matrix shows them all at once, and a heatmap makes the patterns jump out.

Correlation Matrices
From the video: Correlation Matrices
  • Ten features, 45 pairs
  • All correlations at once
  • A heatmap

A heatmap

We load the diabetes dataset that comes with scikit learn, as a pandas data frame. Dot corr computes every pairwise correlation. Then we draw it as a coloured grid. Run it. Bright squares off the diagonal show strongly linked features, like the blood measures S one and S two.

corr_matrix.pypython
import matplotlib.pyplot as plt
from sklearn.datasets import load_diabetes

df = load_diabetes(as_frame=True).frame.drop(columns="target")
corr = df.corr()

plt.imshow(corr, cmap="coolwarm", vmin=-1, vmax=1)
plt.xticks(range(len(corr)), corr.columns, rotation=90)
plt.yticks(range(len(corr)), corr.columns)
plt.colorbar(); plt.title("Feature correlations")
plt.tight_layout(); plt.savefig("corr.png")

Why it matters for models

Features that are very strongly correlated carry almost the same information. That's called multicollinearity. It can make linear regression coefficients unstable and hard to interpret. Common fixes are dropping one of the pair, combining them, using PCA, or adding ridge regularization.

keyvalue
df.corr()all pairs at once
Diagonalalways 1
Multicollinearityfeatures that overlap
Fixesdrop, combine, PCA, ridge

Summary

A correlation matrix shows every pair of features at once. Strongly linked features can confuse linear models. Next, a paradox that fools almost everyone.

Key takeaways
  • df.corr()
  • Heatmap with imshow
  • Watch for overlap
Next up: Simpson's Paradox — when groups flip the trend