AI Hints
Home›Statistics & Math for ML Tutorial›Testing in Practice
Statistics & Math for ML · Lesson 59 of 138

Chi-Square Test

Categories.

⏱ 2 min read▶ Video at 1:08:32🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

Is there a link between someone's age group and their favourite social app? Both are categories, so we can't use means. Instead, we count, and compare the counts with what we'd expect if there were no link. That's the chi square test.

Chi-Square Test
From the video: Chi-Square Test
  • Age group and favourite app
  • Both are categories
  • Compare counts

chi2_contingency

We put the counts in a table. Rows are age groups, and columns are apps. Chi two contingency works out the counts we'd expect if there were no link, and measures how far the real counts are from them. Run it. The p value is tiny, so age group and favourite app are clearly linked.

chi2.pypython
import numpy as np
from scipy.stats import chi2_contingency

#                 App A  App B  App C
counts = np.array([[90,    40,    20],    # under 25
                   [50,    60,    40],    # 25 to 45
                   [20,    40,    90]])   # over 45

chi2, p, dof, expected = chi2_contingency(counts)
print(f"chi2 = {chi2:.1f}, p = {p:.2e}")
print(expected.round(1))
Output
chi2 = 104.0, p = 1.41e-21
[[53.3 46.7 50. ]
 [53.3 46.7 50. ]
 [53.3 46.7 50. ]]

Reading the result

The expected table assumes every age group likes the apps in the same proportions. Under twenty fives were expected to choose app A fifty three times, but chose it ninety times. Big gaps like that make chi square large, and the p value small. Chi square needs counts, not percentages, and expected counts of at least five.

keyvalue
Rows × columnsa contingency table
Expectedif there's no link
Big gapsbig chi-square
Needscounts ≥ 5 expected

Summary

The chi square test compares observed counts with the counts you'd expect if two categories weren't linked. Next, a test that only uses shuffling.

Key takeaways
  • chi2_contingency
  • Observed vs expected
  • Use counts
Next up: Permutation Tests — shuffle the labels