The Model That Broke on Friday

On Monday, your model is a hero. Ninety percent accurate, everyone is clapping. By Friday, it is quietly wrong, and nobody knows. No error message. No crash. Just bad decisions, made at scale. This is the problem that M L Ops exists to solve, and in this course, you will master it.

A model that quietly breaks

This is not a drawing, it is real code, running right now. We train a model to predict which customers will leave. On Monday's data, it is evaluated, and we print its accuracy. Then, on Friday, the customers have changed a little. They pay higher prices, call support more, and sign different contracts. We score the same model on the new data, and print the accuracy again. Watch the two numbers. Nothing crashed, no error appeared, but the model got much worse.

from churn import make_data
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

train = make_data(3000, seed=1)
X, y = train.drop(columns="churn"), train["churn"]
model = LogisticRegression(max_iter=500).fit(X, y)

monday = make_data(1000, seed=2)
pred = model.predict(monday.drop(columns="churn"))
print("Monday accuracy:", round(accuracy_score(monday["churn"], pred), 2))

friday = make_data(1000, seed=3, shifted=True)
pred = model.predict(friday.drop(columns="churn"))
print("Friday accuracy:", round(accuracy_score(friday["churn"], pred), 2))

Monday accuracy: 0.88
Friday accuracy: 0.62

import numpy as np, pandas as pd

def make_data(n, seed, shifted=False):
    rng = np.random.default_rng(seed)
    tenure = rng.integers(1, 72, n)
    charges = rng.normal(65 + (28 if shifted else 0), 18, n).clip(20, 140).round(2)
    calls = rng.poisson(1.5 + (1.5 if shifted else 0), n)
    contract = rng.choice([0, 1, 2], n, p=[.55, .25, .20] if not shifted else [.30, .30, .40])
    logit = -1.0 - 0.04 * tenure + 0.025 * (charges - 65) + 0.45 * calls - 0.8 * contract
    if shifted:
        logit = -0.2 - 0.01 * tenure - 0.01 * (charges - 65) + 0.2 * calls - 0.2 * contract
    churn = (rng.random(n) < 1 / (1 + np.exp(-logit))).astype(int)
    return pd.DataFrame({"tenure": tenure, "charges": charges, "calls": calls, "contract": contract, "churn": churn})

What really happened

Nothing in the code was wrong. The model did exactly what it learned. The world changed, and the model did not. Data in production drifts away from the data used in training. Customers change, seasons change, competitors change, and even a new version of an app can change what the data means. Without monitoring, you find out from an angry customer, weeks later. With M L Ops, you find out within hours, and the system can even react.

1. Trainlearn from past data
2. Deploypredict on new data
3. World changesnew patterns appear
4. Accuracy dropssilently, without errors

What you will master

In this course, you will learn how to keep a model healthy, from the first experiment to years in production. You will version data and code, track experiments, and build repeatable pipelines. You will package models, and serve them, test them, and deploy them automatically. You will monitor them, detect drift, and retrain safely. And you will do it with real tools, real code, and real terminals, so that anyone can follow.

Summary