Quiz Time: The Foundations

Let's check what you have learned. In this quiz, there are six questions about how generative A I works. For each one, I will read the question, and four answers. Then you will have a few seconds to think, before I give the answer. Ready? Let's go.

• Six quick questions
• Pause, think, then check
• No pressure, just practice

A token is...

Question 1. What is a token? A, A whole book. B, A small piece of text. C, A kind of image. D, A password. Pause the video, and think. The answer is B. Text is cut into small pieces called tokens, and each piece becomes a number.

Language model basics

Question 2. What is the core job of a language model? A, Searching the web. B, Predicting the next token. C, Storing documents. D, Drawing pictures. Pause the video, and think. The answer is B. Everything else is built on predicting the next token, again and again.

Embeddings

Question 3. What does an embedding capture? A, The font of a word. B, The meaning of a token as numbers. C, The price of a model. D, The speed of a chip. Pause the video, and think. The answer is B. Similar meanings land close together, as positions on a map of meaning.

Attention

Question 4. What does attention let a model do? A, Decide which words matter for each word. B, Make the screen brighter. C, Delete old data. D, Skip training. Pause the video, and think. The answer is A. Each word looks at the other words, and decides how much each one matters.

Training

Question 5. What does gradient descent do? A, Counts the tokens. B, Nudges weights to reduce the error. C, Sorts the dataset. D, Translates the answer. Pause the video, and think. The answer is B. Like a ball rolling downhill, it takes small steps that lower the loss.

Temperature

Question 6. What does a high temperature do? A, Makes answers more random and creative. B, Makes the model faster. C, Deletes the context. D, Makes answers shorter. Pause the video, and think. The answer is A. It flattens the chances, so less likely words get a real opportunity.

Summary