Receptive Fields

You've got a network that keeps missing large defects, though it's great at small ones. Adding more data doesn't help. The reason is geometric. A neuron can only respond to the part of the image that it's connected to. If that area's too small, it literally can't see the whole object.

• Misses large defects
• More data doesn't help
• A neuron's view is limited

Working out the field

Each layer widens the view. A layer adds its kernel size, multiplied by the combined stride so far. So pooling makes all of the later layers grow faster. We walk through a small network. Run it. Three, five, six, ten, fourteen, sixteen, and then twenty four. After seven layers, each output sees only twenty four pixels. If your defects are a hundred pixels wide, then no neuron can ever see all of one. That's the reason. Now you know what to do. Add more layers, use more pooling, or shrink the image.

layers = [("conv 3", 3, 1), ("conv 3", 3, 1), ("pool 2", 2, 2),
          ("conv 3", 3, 1), ("conv 3", 3, 1), ("pool 2", 2, 2), ("conv 3", 3, 1)]

field, jump = 1, 1
for name, k, s in layers:
    field += (k - 1) * jump
    jump *= s
    print(f"{name}: each output sees {field} pixels")

conv 3: each output sees 3 pixels
conv 3: each output sees 5 pixels
pool 2: each output sees 6 pixels
conv 3: each output sees 10 pixels
conv 3: each output sees 14 pixels
pool 2: each output sees 16 pixels
conv 3: each output sees 24 pixels

Using this knowledge

Before you train, work out your receptive field, and compare it with your objects. Dilated convolutions widen the view without extra layers. Transformers can look across the whole image from the first layer. Detection networks use several scales, so that small layers catch small objects, and deep layers catch large ones. A little arithmetic can prevent a wasted week.

Summary