AI Hints
Home›Statistics & Math for ML Tutorial›The Central Limit Theorem
Statistics & Math for ML · Lesson 47 of 138

The Central Limit Theorem

Averages become normal.

⏱ 2 min read▶ Video at 53:39🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

Here's one of the most surprising facts in all of statistics. Take any distribution, even a very skewed one. Draw a sample, and compute its mean. Do that again and again. The means will form a normal distribution. That's the central limit theorem.

The Central Limit Theorem
From the video: The Central Limit Theorem
  • Start with any distribution
  • Take many sample means
  • They form a bell curve

Seeing it happen

Wait times often follow an exponential distribution, which is very skewed. On the left are ten thousand single wait times. On the right, we take ten thousand samples of thirty waits each, and plot their means. Run it. The raw data is skewed, but the sample means make a bell curve.

clt.pypython
import numpy as np
import matplotlib.pyplot as plt

rng = np.random.default_rng(0)
waits = rng.exponential(5, 10_000)
means = rng.exponential(5, size=(10_000, 30)).mean(axis=1)

fig, ax = plt.subplots(1, 2, figsize=(9, 4.5))
ax[0].hist(waits, bins=50); ax[0].set_title("Single waits")
ax[1].hist(means, bins=50); ax[1].set_title("Means of 30 waits")
plt.tight_layout(); plt.savefig("clt.png")

Why it matters so much

The central limit theorem is why we can use the normal distribution for averages, even when the data itself isn't normal. It's the foundation of confidence intervals and hypothesis tests. It needs a decent sample size, about thirty is a common rule of thumb, and independent values.

keyvalue
Any distributionmeans become normal
Sample sizeabout 30 or more
Spread of meansshrinks as n grows
Powersintervals and tests

Summary

The central limit theorem says sample means are normally distributed, whatever the original data looks like. Next, a quiz on distributions.

Key takeaways
  • Any shape in
  • Means come out normal
  • n ≈ 30 is often enough
Next up: Distributions Quiz — test yourself