Resizing Properly

A team trained a model using one image library, and deployed it using another. Accuracy fell by several points. The training data was fine, and so was the model. The cause was one line, the way that images were being resized. Nearly everybody ignores this, and it matters.

• Train with one library, deploy with another
• Accuracy fell
• One line: resizing

Aliasing

Here's a pattern of fine stripes, and we shrink it to a quarter of its size, in two ways. Nearest neighbour simply picks every fourth pixel. Area averages the pixels that are being combined. Run it. The true image averages out to a smooth grey, so a shrunken version ought to be close to flat. Area gives low contrast, and that's right. But nearest neighbour has a contrast of ninety. It's invented bold stripes that aren't there at all. That's called aliasing, the same thing that makes wagon wheels spin backwards in old films. When shrinking images, always use area, or a filter that's designed to prevent it.

import cv2
import numpy as np

x = np.arange(600)
stripes = (127 + 127 * np.sin(x * 2 * np.pi / 3.3)).astype(np.uint8)
img = np.tile(stripes, (100, 1))

nearest = cv2.resize(img, (150, 25), interpolation=cv2.INTER_NEAREST)
area = cv2.resize(img, (150, 25), interpolation=cv2.INTER_AREA)
print(f"True average: {img.mean():.0f}")
print(f"Nearest neighbour contrast: {nearest.std():.0f}")
print(f"Area average contrast: {area.std():.0f}")

True average: 127
Nearest neighbour contrast: 90
Area average contrast: 17

Choosing an interpolation

When shrinking, use area. When enlarging, use bicubic or Lanczos for quality, and bilinear for speed. For masks and label images, always use nearest, because averaging labels would invent classes that don't exist. Most importantly, use the exact same method in training and in production.

Summary