Quiz Time: Production

Here are six more questions, this time about running models for real. Serving, drift, monitoring, releases, security, and language model apps. Same rules as before. Listen, pause, think, and then check your answer.

• Six more questions
• Production topics
• Think before the answer

Serving

Question 1. Where should a model service load the model? A, On every request. B, Once, when the service starts. C, Only on Mondays. D, In the browser. Pause the video, and think. The answer is B. Loading once, at start up, keeps requests fast, and avoids repeated work.

Drift

Question 2. Which test asks whether two samples come from the same distribution? A, The K S test. B, The unit test. C, The load test. D, The smoke test. Pause the video, and think. The answer is A. The Kolmogorov Smirnov test compares distributions, and a small p value signals drift.

Monitoring

Question 3. Which layer of monitoring is most often neglected? A, Infrastructure. B, Data and model quality. C, Network cables. D, Screen brightness. Pause the video, and think. The answer is B. Teams watch servers well, but silent failures live in the data, and the model quality.

Release

Question 4. What does a canary release do? A, Sends a small share of traffic to the new model. B, Deletes the old model first. C, Retrains every minute. D, Hides the model. Pause the video, and think. The answer is A. A small share of real traffic tests the new version, with the option to roll back quickly.

Security

Question 5. Why is loading an unknown pickle file risky? A, It is very slow. B, It can execute code on your machine. C, It uses too much disk. D, It changes the labels. Pause the video, and think. The answer is B. Pickle can run arbitrary code when loaded, so prefer safe formats, and verify sources.

LLMOps

Question 6. What sits at the centre of the LLM application loop? A, The GPU. B, Evaluation. C, The logo. D, The font. Pause the video, and think. The answer is B. Evaluation against a golden set, after every change, is what keeps language model apps trustworthy.

Summary