AI Hints
Statistics & Math for ML · Lesson 55 of 138

Hypothesis Testing

Innocent until proven.

⏱ 2 min read▶ Video at 1:03:32🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

A company changes its website, and sales go up a little. Did the change work, or was it just luck? Hypothesis testing is a careful way to answer, is this effect real, or could chance alone explain it?

Hypothesis Testing
From the video: Hypothesis Testing
  • Sales went up a little
  • Real, or just luck?
  • Hypothesis testing

Like a court case

A court assumes the defendant is innocent, unless the evidence is strong. Hypothesis testing works the same way. The null hypothesis says nothing interesting is happening. No effect, no difference. We only reject it if the data would be very surprising under the null.

Null hypothesisno effect, just chance
Alternativethere is an effect
Evidencehow surprising is the data?
Verdictreject the null, or don't

The steps

Every test follows the same steps. State the null hypothesis. Choose a significance level, usually five percent. Compute a test statistic from your data. Find the p value. And if the p value is below the significance level, reject the null.

  1. Statethe null
  2. Chooseα = 0.05
  3. Computea test statistic
  4. Findthe p-value
  5. Decidep < α? reject
The steps
From the video: The steps

Summary

A hypothesis test assumes no effect, and asks whether the data is surprising enough to reject that. Next, the p value.

Key takeaways
  • Null: no effect
  • Evidence must be strong
  • Reject, or don't
Next up: The p-value — how surprising?