Confidence Intervals

Saying that average spend is seven dollars and forty cents sounds exact. But it came from a sample. A more honest answer is, it's somewhere between six ninety and seven ninety. That range is called a confidence interval, and it changes how people make decisions.

• $7.40 sounds exact
• It came from a sample
• A range is more honest

Two ways to get a range

The classic formula takes the mean, plus or minus about two standard errors. The standard error shrinks with the sample size. The bootstrap gets the same result by brute force. Resample your data many times, recompute the mean, and look at the spread. Run it. The two agree closely. The bootstrap needs no formula at all, so it works on medians, ratios, and almost anything else. It's one of the most useful tricks in the subject.

import numpy as np
import pandas as pd

df = pd.read_csv("customers.csv").drop_duplicates()
sample = df["avg_spend"].sample(200, random_state=0)

mean, se = sample.mean(), sample.std() / np.sqrt(len(sample))
print(f"Estimate: ${mean:.2f}")
print(f"95% range: ${mean - 1.96 * se:.2f} to ${mean + 1.96 * se:.2f}")

rng = np.random.default_rng(0)
boot = [rng.choice(sample, len(sample)).mean() for _ in range(5000)]
print("Bootstrap:", np.percentile(boot, [2.5, 97.5]).round(2))

Estimate: $10.77
95% range: $10.26 to $11.28
Bootstrap: [10.28 11.28]

Reading an interval properly

A ninety five percent interval means that if we repeated this many times, about ninety five percent of the ranges would contain the truth. For a decision, ask whether the entire range leads to the same action. If it does, you can act with confidence. If it doesn't, you might need more data. A wide range is honest information, not a failure.

Summary