Your Cheat Sheet
The whole course on one page.
Here's the entire course, in as few words as possible. Pause any screen, and take a screenshot. Keep these close when you start your next project.
- The whole course, compressed
- Pause and screenshot
- Keep it close
Which architecture?
Start here. For tables, don't use deep learning at all, use boosting. For images, use a pretrained ConvNeXt or vision transformer. For text, use a pretrained transformer. For sound, use a spectrogram and a pretrained model. For generating images, use diffusion. For search, use embeddings.
| Data or task | Use |
|---|---|
| Tables | Gradient boosting |
| Images | Pretrained CNN or ViT |
| Text | Pretrained transformer |
| Audio | Spectrogram + pretrained |
| Generate images | Diffusion |
| Search | Embeddings |
Good starting settings
Here are safe defaults. AdamW, with a learning rate of about three times ten to the minus four. A warm up with cosine decay. Batch size, whatever fits. Dropout, between zero and point two. Weight decay, of point zero one. And gradient clipping, at one.
| Setting | Good default |
|---|---|
| Optimiser | AdamW |
| Learning rate | 3e-4 |
| Schedule | Warm-up + cosine |
| Dropout | 0 to 0.2 |
| Weight decay | 0.01 |
| Gradient clipping | 1.0 |
The recipe
And here's the recipe for each project. Look at your data. Check the first loss. Overfit one batch. Add the full data, and get a baseline. Add regularisation. Tune the learning rate. Check your errors. Then export and serve. If you're stuck, climb back down the ladder.
Summary
That's your whole course, on three screens. Next, let's clear up some myths, and talk about where you go from here. It's time for our final lesson.
Key takeaways- Pick by data type
- Start with safe defaults
- Follow the recipe