AI Hints
Home›Statistics & Math for ML Tutorial›The Central Limit Theorem
Statistics & Math for ML · Lesson 46 of 138

Is It Normal?

Q-Q plots.

⏱ 2 min read▶ Video at 52:24🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

Many statistical tools assume your data is roughly normal. A histogram gives a hint, but it's hard to judge by eye. AQQ plot makes it much easier.

Is It Normal?
From the video: Is It Normal?
  • Many tools assume normal
  • Histograms are hard to judge
  • AQ-Q plot

Points on a line

AQQ plot compares your data's quantiles with the quantiles a normal distribution would have. If the data is normal, the points fall on a straight line. Sci Pi's probplot draws it. On the left is normal data, and on the right, skewed incomes. Run it. The left points hug the line. The right ones curve away, a sign of skew.

qq.pypython
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import probplot

rng = np.random.default_rng(0)
fig, ax = plt.subplots(1, 2, figsize=(9, 4.5))
probplot(rng.normal(0, 1, 500), plot=ax[0]); ax[0].set_title("Normal data")
probplot(rng.lognormal(0, 0.8, 500), plot=ax[1]); ax[1].set_title("Skewed data")
for a in ax: a.get_lines()[0].set_color("#e879f9"); a.get_lines()[1].set_color("#22d3ee")
plt.tight_layout(); plt.savefig("qq.png")

How normal is normal enough?

Real data is never perfectly normal. A little wobble at the ends is fine. What matters is strong skew, heavy tails, or two peaks. If your data isn't normal, you can transform it, with a log, or use methods that don't assume normality, like the bootstrap, which we'll see soon.

keyvalue
On the lineroughly normal
Curvedskewed
S-shape at endsheavy tails
Not normal?log, or the bootstrap

Summary

AQQ plot shows whether data is roughly normal. Points on a line mean yes. Next, the theorem that explains why normal is everywhere.

Key takeaways
  • scipy.stats.probplot
  • Straight line = normal
  • Curves mean skew
Next up: The Central Limit Theorem — averages become normal