AI Hints
Statistics & Math for ML · Lesson 67 of 138

Covariance

Moving together.

⏱ 2 min read▶ Video at 1:18:57🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

Do students who study more get higher scores? When one variable goes up, does the other tend to go up too? Covariance is the first way to measure that.

Covariance
From the video: Covariance
  • Study more, score more?
  • Do they move together?
  • Covariance

Multiply the deviations

For each student, take hours minus its mean, and score minus its mean, and multiply them. When both are above average, or both below, the product is positive. Covariance is the average of those products. Run it. It's positive, so hours and scores move together.

covariance.pypython
import numpy as np
from study import hours, scores

dh = hours - hours.mean()
ds = scores - scores.mean()
print("covariance:", round((dh * ds).sum() / (len(hours) - 1), 2))
print("np.cov:    ", round(np.cov(hours, scores)[0, 1], 2))
Output
covariance: 34.36
np.cov:     34.36
Also uses study.py
import numpy as np

hours = np.array([1, 2, 3, 4, 5, 6, 7, 8])
scores = np.array([42, 50, 53, 61, 64, 72, 74, 85])

The problem with covariance

The sign of the covariance is useful. Positive means they rise together, and negative means one falls as the other rises. But the size is hard to read, because it depends on the units. Measure time in minutes instead of hours, and the covariance becomes sixty times bigger. We need a version without units.

keyvalue
Positiverise together
Negativeone up, one down
Near zerono straight-line link
Problemsize depends on units

Summary

Covariance measures whether two variables move together, but its size depends on the units. Next, correlation, which fixes that.

Key takeaways
  • Multiply deviations
  • Sign tells direction
  • Size depends on units
Next up: Correlation — from -1 to 1