Famous Architectures

In nineteen ninety eight, Yann LeCun's network was reading the numbers on bank cheques across America. It took fourteen more years for the world to catch up. Each famous network since then added one key idea. Learn those ideas, and you'll understand how any modern vision model works.

• 1998: reading bank cheques
• Fourteen years to catch up
• Each network added one idea

The family tree of vision

LeNet, in nineteen ninety eight, proved that convolutions work. AlexNet, in twenty twelve, added ReLU, dropout, and G P Us, and started the revolution. V G G, in twenty fourteen, showed that many small filters work better than a few large ones. ResNet, in twenty fifteen, added skip connections, and suddenly networks could have over a hundred layers. And in twenty twenty, Vision Transformers showed that attention can see too. ConvNeXt then proved that modern C N Ns can compete.

• 1998LeNet
• 2012AlexNet
• 2014VGG
• 2015ResNet
• 2020Vision Transformer

The key ideas

Here's the short version. Small filters, stacked deep, see more for less cost. Batch normalisation keeps training stable. Skip connections let very deep networks learn at all. And global average pooling replaces the huge final layers, cutting millions of weights. Those four ideas are behind almost every vision network you'll meet.

Summary