AI Hints
Statistics & Math for ML · Lesson 51 of 138

Confidence Intervals

A range for the truth.

⏱ 2 min read▶ Video at 58:50🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

Instead of saying, the average height is one seventy point four, it's more honest to give a range. Something like, between one sixty eight and one seventy two. A confidence interval is a range that's likely to contain the true value.

Confidence Intervals
From the video: Confidence Intervals
  • Not one number
  • A range
  • Likely to contain the truth

Mean plus or minus two standard errors

Thanks to the central limit theorem, sample means are normal. So about ninety five percent of the time, the sample mean is within one point nine six standard errors of the truth. The interval is the mean, plus or minus one point nine six standard errors. Run it.

ci.pypython
import numpy as np

rng = np.random.default_rng(2)
sample = rng.normal(170, 8, 100)

mean = sample.mean()
se = sample.std(ddof=1) / np.sqrt(len(sample))
print(f"95% CI: {mean - 1.96 * se:.1f} to {mean + 1.96 * se:.1f}")
Output
95% CI: 168.4 to 171.4

What 95 percent really means

Here's the right way to think about it. If we repeated our study a thousand times, about ninety five percent of the intervals would contain the true mean. Let's do exactly that. Run it. About ninety five percent of the intervals caught the true value.

coverage.pypython
import numpy as np

rng = np.random.default_rng(3)
hits = 0
for _ in range(1000):
    s = rng.normal(170, 8, 100)
    se = s.std(ddof=1) / 10
    hits += abs(s.mean() - 170) < 1.96 * se
print("intervals containing 170:", hits / 1000)
Output
intervals containing 170: 0.95

Summary

A ninety five percent confidence interval is the estimate plus or minus about two standard errors. Ninety five percent of such intervals catch the truth. Next, intervals in code.

Key takeaways
  • mean ± 1.96 × SE
  • 95% of intervals catch the truth
  • Report ranges, not points
Next up: Intervals in Code — scipy t interval