AI Hints
Statistics & Math for ML · Lesson 31 of 138

Bayes' Theorem

Updating beliefs.

⏱ 2 min read▶ Video at 34:54🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

The chance that a sick person tests positive is not the same as the chance that a positive person is sick. People mix these up all the time, even doctors. Bayes' theorem shows how to go from one to the other.

Bayes' Theorem
From the video: Bayes' Theorem
  • P(positive | sick)
  • is not P(sick | positive)
  • Bayes flips it around

The formula

Bayes' theorem says the probability of A given B equals the probability of B given A, times the probability of A, divided by the probability of B. The probability of A is called the prior, what you believed before. The result is called the posterior, what you believe after seeing the evidence.

keyvalue
FormulaP(A|B) = P(B|A) · P(A) ÷ P(B)
PriorP(A), belief before
LikelihoodP(B|A), fit of the evidence
PosteriorP(A|B), belief after

Bayes as a function

Here it is as a function. P of B is the total chance of the evidence, from both the A cases and the not A cases. Let's try it. A rare disease has a prior of one percent. Run it. We'll work out what this means in the next lesson.

bayes.pypython
def bayes(prior, p_evidence_if_true, p_evidence_if_false):
    p_evidence = (p_evidence_if_true * prior
                  + p_evidence_if_false * (1 - prior))
    return p_evidence_if_true * prior / p_evidence

print(round(bayes(prior=0.01, p_evidence_if_true=0.99,
                  p_evidence_if_false=0.05), 3))
Output
0.167

Summary

Bayes' theorem turns the probability of evidence given a cause into the probability of the cause given the evidence. It updates a prior into a posterior. Next, the medical test that surprises everyone.

Key takeaways
  • Prior × likelihood
  • Divide by the evidence
  • Posterior
Next up: Bayes: The Medical Test — why positives mislead