Quiz: Foundations

Time to test yourself. Testing is how memory forms. Here are ten questions on the foundations. Pause on each one, and commit to an answer before you hear mine. Keep score, and we'll see where you stand.

• Testing builds memory
• Ten questions
• Keep score

Question 1

Question 1. What does a neuron compute? A, A weighted sum, then an activation. B, A random number. C, A file on disk. D, A decision tree. Pause the video and choose. The answer is A. A neuron multiplies inputs by weights, adds them, and applies a bend.

Question 2

Question 2. Without activation functions, a deep network is equivalent to what? A, A random forest. B, A single linear layer. C, A convolution. D, A transformer. Pause the video and choose. The answer is B. Stacked linear layers collapse into one linear layer.

Question 3

Question 3. What does loss dot backward do? A, Updates the weights. B, Computes the gradients. C, Loads data. D, Saves the model. Pause the video and choose. The answer is B. It computes the gradients. The optimiser step is what changes the weights.

Question 4

Question 4. Why call zero grad each step? A, To speed up the GPU. B, Because gradients add up by default. C, To reset the weights. D, To shuffle data. Pause the video and choose. The answer is B. Otherwise, each new gradient would be added to the old ones.

Question 5

Question 5. Your ten class model's first loss is about two point three. What does that suggest? A, It is broken. B, It is about right for random weights. C, It has finished training. D, The data is missing. Pause the video and choose. The answer is B. The log of ten is about two point three, so it's what we expect.

Question 6

Question 6. Which learning rate problem makes loss explode? A, Too small. B, Too large. C, Exactly right. D, None. Pause the video and choose. The answer is B. Steps that are too large jump across the valley, and climb out.

Question 7

Question 7. What does dropout do during training? A, Removes data. B, Randomly switches off neurons. C, Doubles the weights. D, Stops training. Pause the video and choose. The answer is B. It stops neurons from depending on each other.

Question 8

Question 8. Why use a convolution instead of a linear layer on images? A, It uses fewer shared weights. B, It needs more memory. C, It is always slower. D, It removes colour. Pause the video and choose. The answer is A. One small detector is shared across the whole image.

Question 9

Question 9. What problem did ResNet's skip connections solve? A, Slow data loading. B, Deep networks failing to train. C, Colour errors. D, File size. Pause the video and choose. The answer is B. They give gradients a highway, so layers only learn the change.

Question 10

Question 10. Before tuning anything, what's the great sanity test? A, Buy a bigger GPU. B, Overfit one small batch. C, Add more layers. D, Change the optimiser. Pause the video and choose. The answer is B. If you can't memorise one batch, there's a bug.

Summary