AI Hints
Home›Statistics & Math for ML Tutorial›Intervals and the Bootstrap
Statistics & Math for ML · Lesson 53 of 138

The Bootstrap

Resampling.

⏱ 2 min read▶ Video at 1:01:18🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

What if you want a confidence interval for something without a neat formula, like the median? Or what if your data isn't normal? There's a brilliant trick called the bootstrap. Your sample pretends to be the population.

The Bootstrap
From the video: The Bootstrap
  • No formula?
  • Not normal?
  • The bootstrap

How it works

Draw a new sample from your own data, the same size, with replacement. So some values appear twice, and some not at all. Compute your statistic on it. Repeat that thousands of times. The spread of those results shows how much your statistic would vary from sample to sample.

1. Resamplesame size, with replacement
2. Computethe statistic
3. Repeatthousands of times
4. Readthe middle 95%

A bootstrap interval for the mean

We resample our twelve wait times ten thousand times, and take each mean. The middle ninety five percent of those means, from the two point five to the ninety seven point five percentile, is our interval. Run it. Close to the t interval, and a little narrower, without any formula.

bootstrap.pypython
import numpy as np

waits = np.array([4.1, 3.5, 5.2, 4.8, 3.9, 4.4, 6.0, 3.2, 4.7, 5.1, 3.8, 4.3])
rng = np.random.default_rng(0)

boots = [rng.choice(waits, len(waits)).mean() for _ in range(10_000)]
low, high = np.percentile(boots, [2.5, 97.5])
print(f"bootstrap 95% CI: {low:.2f} to {high:.2f}")
Output
bootstrap 95% CI: 4.00 to 4.86

Summary

The bootstrap resamples your data with replacement, thousands of times, and reads the interval from the results. Next, bootstrapping anything you like.

Key takeaways
  • Resample with replacement
  • Repeat 10,000 times
  • Middle 95%
Next up: Bootstrap Anything — median interval