Stratified Splits
Keep the class balance.
If only five percent of your data is the rare class, a random train test split might put too few of them in the test set, or too many. Stratified splitting keeps the same class balance in every part.

- 5% rare class
- Random splits can wobble
- Keep the balance
stratify=y
We make labels with exactly five percent positives. Then we split three ways. Plain random splits with different seeds, and a stratified split, passing stratify equals Y. Run it. The random test sets get two and a half percent, and seven percent. The stratified one is exactly five percent.
import numpy as np
from sklearn.model_selection import train_test_split
y = np.array([1] * 50 + [0] * 950)
X = np.arange(1000).reshape(-1, 1)
for seed in [2, 9]:
_, _, _, yte = train_test_split(X, y, test_size=0.2, random_state=seed)
print("random test share: ", yte.mean())
_, _, _, yte = train_test_split(X, y, test_size=0.2, stratify=y, random_state=0)
print("stratified test share:", yte.mean())random test share: 0.025 random test share: 0.07 stratified test share: 0.05
Use it by default
For classification, stratifying is almost always a good idea. Cross validation in scikit learn does it automatically for classifiers, using stratified K fold. For time series, though, never shuffle. Train on the past, and test on the future, or your score will be far too optimistic.
| key | value |
|---|---|
| train_test_split | stratify=y |
| Cross-validation | StratifiedKFold |
| Rare classes | most important |
| Time series | split by time, no shuffle |
Summary
Stratified splits keep each class's share the same in training and test data. Use them for classification. Next, what to do when one class is rare.
Key takeaways- stratify=y
- StratifiedKFold
- Time series: no shuffle