Matrix Rank
How much information.
A data matrix might have ten columns, but if some columns are just combinations of others, it really carries less information. The rank of a matrix counts how many truly independent columns it has.

- Ten columns
- Some are copies or combinations
- Rank counts the real ones
np.linalg.matrix_rank
Here, the third column is just the first plus the second, and the fourth is double the first. So only two columns are independent. Run it. The matrix has four columns, but its rank is two.
import numpy as np
rng = np.random.default_rng(0)
a, b = rng.normal(size=100), rng.normal(size=100)
X = np.column_stack([a, b, a + b, 2 * a])
print("columns:", X.shape[1])
print("rank: ", np.linalg.matrix_rank(X))columns: 4 rank: 2
Why rank matters
If the rank is lower than the number of columns, the matrix X transpose X is singular, so the normal equation can't be solved exactly. Linear regression can't tell the duplicate features apart. Dropping redundant features, or using ridge regression, fixes it. Low rank is also why PCA can compress data so well.
| key | value |
|---|---|
| Rank | independent columns |
| Full rank | no redundant columns |
| Low rank | XᵀX is singular |
| Fixes | drop features, ridge |
Summary
The rank counts how many independent columns a matrix has. Redundant features lower it, and break the normal equation. Next, second derivatives, and curvature.
Key takeaways- np.linalg.matrix_rank
- Duplicates lower rank
- Ridge helps