AI Hints
Statistics & Math for ML · Lesson 49 of 138

Sampling Error

Samples vary.

⏱ 2 min read▶ Video at 56:37🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

Ask fifty random people their height, and compute the mean. Ask a different fifty, and you'll get a slightly different mean. Neither is wrong. Samples just vary by chance. That variation is called sampling error.

Sampling Error
From the video: Sampling Error
  • Fifty people: one mean
  • Another fifty: another mean
  • Samples vary by chance

Five samples

We create a population of a million heights, with a true mean of one seventy. Then we draw five random samples of fifty people, and print each sample's mean. Run it. Every sample mean is a little off, some above and some below the true value.

samples.pypython
import numpy as np

rng = np.random.default_rng(0)
population = rng.normal(170, 8, 1_000_000)
print("true mean:", population.mean().round(2))

for i in range(5):
    sample = rng.choice(population, 50)
    print("sample mean:", sample.mean().round(2))
Output
true mean: 170.01
sample mean: 169.32
sample mean: 171.49
sample mean: 169.56
sample mean: 170.03
sample mean: 170.88

Error is not a mistake

Sampling error isn't a mistake in your work. It's the natural wobble from looking at only part of the population. The whole point of inference is to measure that wobble. Then we can say, the true mean is probably within this range, rather than pretending our sample is perfect.

keyvalue
Sampling errornatural wobble
Not a mistakejust chance
Bigger samplessmaller wobble
Inferencemeasures the wobble

Summary

Sample statistics vary from sample to sample, just by chance. That's sampling error. Next, measuring how big it is, with the standard error.

Key takeaways
  • Samples vary
  • Not a mistake
  • Measure the wobble
Next up: Standard Error — how much averages vary