Quiz: Advanced

This round covers attention, transformers, generative models, and practical skills. These are the kinds of questions you'd get in a real interview. As before, commit to your answer.

• Attention and transformers
• Generative models
• Interview level

Question 1

Question 1. In attention, what do the softmax weights decide? A, How much of each value to blend in. B, The learning rate. C, Batch size. D, The image size. Pause the video and choose. The answer is A. Each word becomes a weighted blend of the others.

Question 2

Question 2. Why do transformers need position embeddings? A, To save memory. B, Attention by itself ignores order. C, To speed up the GPU. D, To remove the mask. Pause the video and choose. The answer is B. Attention sees a set, so order must be added.

Question 3

Question 3. What's the purpose of the mask in a GPT? A, Hide future words. B, Make images smaller. C, Remove noise. D, Improve colour. Pause the video and choose. The answer is A. Each position can only look at itself and earlier positions.

Question 4

Question 4. What is the target when training a language model? A, The same text, shifted by one. B, Random noise. C, The first word only. D, An image. Pause the video and choose. The answer is A. Every position predicts the next token.

Question 5

Question 5. Low temperature does what to text generation? A, Makes it safer and more repetitive. B, Makes it random. C, Stops generation. D, Adds new words. Pause the video and choose. The answer is A. The top choice dominates at low temperatures.

Question 6

Question 6. In a GAN, what does the discriminator do? A, Generates images. B, Tells real from fake. C, Adds noise. D, Compresses data. Pause the video and choose. The answer is B. It's the detective, and it pushes the forger to improve.

Question 7

Question 7. During training, what does a diffusion model predict? A, The class label. B, The noise that was added. C, The next word. D, The file size. Pause the video and choose. The answer is B. It learns to estimate the noise, so that it can remove it.

Question 8

Question 8. You've got two hundred images for a new task. What should you try first? A, Train a huge network from scratch. B, Transfer learning. C, A GAN. D, Delete half of them. Pause the video and choose. The answer is B. A pretrained network gives you years of visual experience for free.

Question 9

Question 9. Why quantise a model? A, To make it larger. B, To reduce size and speed it up. C, To add accuracy. D, To train faster on data. Pause the video and choose. The answer is B. Fewer bits per weight, usually with only a small loss of accuracy.

Question 10

Question 10. Model works in training but fails on test, with dropout. What did you likely forget? A, model.eval(). B, Buying more data. C, Saving the file. D, Importing NumPy. Pause the video and choose. The answer is A. Dropout and batch norm behave differently at test time.

Summary