Why Images Need Convolutions

Aria needs eyes. So far, we've treated images as one long list of pixels, which throws away something vital. If a cat moves one inch to the right, every pixel changes, and our flat network sees something totally new. Your own eyes don't work like that. In the nineteen fifties, two scientists found out how they do work.

• Aria needs eyes
• Flat networks lose the layout
• How does real vision work?

Two big problems with flat networks

Flat networks have two big problems. First, size. A small colour photo has over a hundred thousand inputs, and a fully connected layer multiplies that into a hundred million weights or more. Second, position. A feature learned in the top left corner has to be learned again for the bottom right. Convolutions solve both, with one lovely idea. Slide one small detector across the whole image.

Counting the weights

Let's count. A fully connected layer on a small photo, with a thousand outputs. Compared to a convolution layer, with sixty four detectors. Run it. A hundred and fifty million weights against about two thousand. That's a saving of tens of thousands of times, and the convolutional version often sees better. Sharing weights isn't a compromise. It's a better fit for how images work.

from torch import nn

flat = nn.Linear(224 * 224 * 3, 1000)
conv = nn.Conv2d(3, 64, kernel_size=3)

print("Fully connected:", sum(p.numel() for p in flat.parameters()))
print("One conv layer: ", sum(p.numel() for p in conv.parameters()))

Fully connected: 150529000
One conv layer:  1792

Summary