Case Study: Recommendations
Millions of choices, in milliseconds.
When you open a video app, or an online shop, a recommendation system decides what you see. It runs for millions of users, answers in milliseconds, and learns from every click. It is a perfect case study, because it needs almost everything in this course. Let's see how its operations fit together.
- Millions of users
- Answers in milliseconds
- Learns from every click
A two-stage recommender
Recommenders usually work in two stages, because scoring millions of items for every user is too slow. First, candidate generation. A fast method, often based on embeddings, picks a few hundred items, that might interest the user. Second, ranking. A richer model scores those few hundred, using many features, and sorts them. Then rules adjust the list, to add variety, hide items already seen, and respect business needs. Each stage is its own model, with its own data, serving, and monitoring.
- Candidate generationmillions down to hundreds
- Ranking modelscores with rich features
- Business rulesvariety, freshness, safety
- Show the listand log what happened
The operational challenges
Recommenders raise classic operational challenges. Freshness. New items, and new interests, appear constantly, so features and indexes must update within minutes. Scale. Feature lookups must happen in a few milliseconds, which is why online feature stores matter. Feedback loops. The system only learns from what it shows, so exploration is essential. Evaluation. Offline scores are poor predictors of real engagement, so A B tests are the final judge. And monitoring. Watch click through rates, diversity, and the share of new items shown.
Mapping to what you learned
Let's map the case to the course. Data pipelines collect clicks, and views, with strict point in time joins. A feature store serves user and item features. Experiment tracking and a registry manage candidate and ranking models. Pipelines retrain them daily, or hourly. Serving uses batching, and tight latency budgets. Canary releases, and A B tests, decide every change. Monitoring watches drift, quality, and business metrics. Nothing in this system is exotic. It is the same loop, run with discipline, at scale.
| Course topic | In a recommender |
|---|---|
| Data pipelines | clicks and views, point-in-time joins |
| Feature store | user and item features, online and offline |
| Registry and pipelines | two models, retrained often |
| Serving | batching, tight latency |
| Experiments | canaries and A/B tests |
| Monitoring | drift, diversity, engagement |
Summary
Good. A recommender has two stages, candidate generation and ranking, and needs fresh features, millisecond serving, exploration, and A B tests. It is the whole M L Ops loop at scale. Next, a different challenge, fraud detection, where the interesting cases are extremely rare.
Key takeaways- Two stages: candidates, ranking
- Fresh, fast features
- A/B tests decide