AI Hints
Statistics & Math for ML · Lesson 27 of 138

The Law of Large Numbers

More trials, better answers.

⏱ 2 min read▶ Video at 30:25🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

Flip a coin ten times, and you might get seven heads. Flip it ten thousand times, and you'll get very close to half. That's the law of large numbers. The more trials, the closer the average gets to the true probability.

The Law of Large Numbers
From the video: The Law of Large Numbers
  • 10 flips: maybe 7 heads
  • 10,000 flips: close to half
  • More trials, closer to the truth

Watching it settle

We flip a coin five thousand times, and after each flip, compute the share of heads so far. Cumulative sum keeps a running total. Run it. At the start, the line jumps all over the place. But it settles down, closer and closer to a half.

lln.pypython
import numpy as np
import matplotlib.pyplot as plt

rng = np.random.default_rng(5)
flips = rng.integers(0, 2, 5000)
share = np.cumsum(flips) / np.arange(1, 5001)

plt.plot(share)
plt.axhline(0.5, color="#22d3ee", linestyle="--")
plt.title("Share of heads"); plt.xlabel("flips")
plt.savefig("lln.png")

Why it matters

This is why casinos always win in the long run, even if some players get lucky. And it's why machine learning needs lots of data. With a small sample, results bounce around by chance. With a big sample, they settle near the truth. Be wary of conclusions from tiny datasets.

keyvalue
Few trialsnoisy, can mislead
Many trialsclose to the truth
Casinoswin in the long run
MLmore data, steadier results

Summary

The law of large numbers says averages settle near the truth as trials grow. Small samples are noisy. Next, combining events with and, or, and not.

Key takeaways
  • np.cumsum running totals
  • Noisy at first
  • Settles with more data
Next up: And, Or, Not — combining events