AI Hints
Statistics & Math for ML · Lesson 57 of 138

The t-Test

Comparing two groups.

⏱ 2 min read▶ Video at 1:06:06🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

Do students who use a new study app score higher than those who don't? We have two groups, and their scores. The t test checks whether the difference in their means is bigger than chance would explain.

The t-Test
From the video: The t-Test
  • App users vs non-users
  • Two groups of scores
  • Is the difference real?

ttest_ind

Here are the scores for both groups. T test IND compares two independent groups. Run it. The app group is about five points higher, and the p value is below five percent. So we reject the null. The difference is statistically significant.

ttest.pypython
import numpy as np
from scipy.stats import ttest_ind

rng = np.random.default_rng(14)
app = rng.normal(75, 10, 40)
no_app = rng.normal(70, 10, 40)

result = ttest_ind(app, no_app)
print(f"means: {app.mean():.1f} vs {no_app.mean():.1f}")
print(f"t = {result.statistic:.2f}, p = {result.pvalue:.4f}")
Output
means: 75.3 vs 70.1
t = 2.32, p = 0.0232

Which t-test?

There are a few versions. Two independent groups use T test IND. If the same people were measured twice, before and after, use the paired test, T test REL. And to compare one group to a fixed number, use T test one sample. If the groups have very different spreads, pass equal var equals false.

keyvalue
ttest_indtwo separate groups
ttest_relsame people, before and after
ttest_1sampone group vs a number
equal_var=Falsedifferent spreads

Summary

The t test checks whether two group means differ by more than chance. Use the independent, paired, or one sample version to fit your data. Next, A/B testing.

Key takeaways
  • ttest_ind
  • ttest_rel for pairs
  • p < 0.05: significant
Next up: A/B Testing — which version wins?