Cross-Validation
Many splits.
One train test split can be lucky or unlucky. Maybe the easy examples all landed in the test set. Cross validation repeats the split several times, and averages the results, for a much more reliable score.

- One split can be lucky
- Repeat the split
- Average the scores
K-fold
In five fold cross validation, we cut the data into five equal parts, called folds. We train on four folds, and test on the fifth. Then we rotate, so every fold gets a turn as the test set. That gives five scores, and we report their mean and standard deviation.
cross_val_score
Scikit learn does it in one line. We score a logistic regression with five fold cross validation on the breast cancer dataset that comes with scikit learn. Run it. Five scores, each a little different. The mean is our estimate, and the standard deviation shows how much it wobbles.
from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = load_breast_cancer(return_X_y=True)
model = make_pipeline(StandardScaler(), LogisticRegression())
scores = cross_val_score(model, X, y, cv=5)
print("scores:", scores.round(3))
print(f"mean {scores.mean():.3f} ± {scores.std():.3f}")scores: [0.982 0.982 0.974 0.974 0.991] mean 0.981 ± 0.007
Summary
Cross validation trains and tests on several splits, and averages them. It gives a more reliable score, with a sense of its spread. Next, interview questions on statistics.
Key takeaways- k folds
- cross_val_score
- Report mean ± std