AI Hints
Statistics & Math for ML · Lesson 32 of 138

Bayes: The Medical Test

Why positives mislead.

⏱ 2 min read▶ Video at 36:02🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

A disease affects one percent of people. A test catches ninety nine percent of sick people. But it also wrongly says positive for five percent of healthy people. You test positive. What's the chance you're sick? Pause and guess.

Bayes: The Medical Test
From the video: Bayes: The Medical Test
  • 1% have the disease
  • Test catches 99%
  • 5% false positives

A million people

Instead of a formula, let's simulate a million people. One percent are sick. Sick people test positive ninety nine percent of the time, and healthy people five percent of the time. Then we look only at the people who tested positive. Run it. Only about seventeen percent of them are sick.

medical.pypython
import numpy as np

rng = np.random.default_rng(0)
n = 1_000_000
sick = rng.random(n) < 0.01
positive = np.where(sick, rng.random(n) < 0.99, rng.random(n) < 0.05)

print("positives:", positive.sum())
print("P(sick | positive):", sick[positive].mean().round(3))
Output
positives: 59176
P(sick | positive): 0.167

Why so low?

Because healthy people vastly outnumber sick people. Five percent of nearly a million healthy people is about fifty thousand false alarms. That swamps the roughly ten thousand true positives. When something is rare, even a good test gives mostly false positives. In machine learning, the same thing happens with rare classes, like fraud.

keyvalue
True positivesabout 10,000
False positivesabout 50,000
Somost positives are healthy
Lessonrare events need care

Summary

When a condition is rare, most positive results are false alarms. Always consider the base rate, the prior. Next, using Bayes to catch spam.

Key takeaways
  • Simulate to check
  • Base rates matter
  • Rare means many false alarms
Next up: Bayes for Spam — the naive Bayes idea