JPEG and Your Model
What compression does.
Your training photos came straight from a good camera. But in production, the images get sent across the network, heavily compressed to save bandwidth. Everything looks the same to you. Does it look the same to a model?
- Training photos were clean
- Production images are compressed
- Does a model notice?
Quality against size
We save the same image at four quality levels, then measure the file size, and how far the result is from the original. Run it. Going from ninety five to fifty cuts the file to less than a fifth, with hardly any error. That's the sweet spot. Below that, the savings shrink, and the damage grows quickly. At quality five, the error's ten times worse, and you've saved almost nothing more. So what should you do? Find the point just before the drop, and include compression in your training augmentation, at the level that you'll meet in production.
import cv2
from belt import belt
img, _, _ = belt()
for q in [95, 50, 15, 5]:
ok, data = cv2.imencode(".jpg", img, [cv2.IMWRITE_JPEG_QUALITY, q])
back = cv2.imdecode(data, 1)
print(f"Quality {q:>2}: {len(data) / 1000:5.1f} KB, error {cv2.absdiff(img, back).mean():.2f}")Quality 95: 42.3 KB, error 1.10 Quality 50: 7.6 KB, error 1.80 Quality 15: 6.5 KB, error 4.71 Quality 5: 6.0 KB, error 10.77
Also uses belt.py
import numpy as np
import cv2
def belt(n=7, seed=1, size=(480, 640), bruises=2, light=1.0):
"""Draw a simulated conveyor belt of apples. Returns the image and the true boxes."""
rng = np.random.default_rng(seed)
h, w = size
img = np.full((h, w, 3), (70, 75, 80), np.uint8)
img = cv2.add(img, rng.normal(0, 6, img.shape).clip(0, 30).astype(np.uint8))
boxes, spots, taken = [], [], []
while len(boxes) < n:
r = int(rng.integers(32, 48))
x, y = int(rng.integers(r + 5, w - r - 5)), int(rng.integers(r + 5, h - r - 5))
if any((x - a) ** 2 + (y - b) ** 2 < (r + c + 8) ** 2 for a, b, c in taken):
continue
taken.append((x, y, r))
red = int(rng.integers(150, 225))
cv2.circle(img, (x, y), r, (35, 40, red), -1, cv2.LINE_AA)
cv2.circle(img, (x - r // 3, y - r // 3), r // 4, (90, 90, min(255, red + 30)), -1, cv2.LINE_AA)
boxes.append((x - r, y - r, x + r, y + r))
for x, y, r in taken[:bruises]:
cv2.ellipse(img, (x + r // 3, y + r // 4), (r // 3, r // 4), 20, 0, 360, (25, 45, 70), -1, cv2.LINE_AA)
spots.append((x, y, r))
img = cv2.GaussianBlur(img, (3, 3), 0)
return np.clip(img * light, 0, 255).astype(np.uint8), boxes, spotsOther hidden changes
Your image pipeline can change things in other ways. Phones apply sharpening and beauty filters. Video streams use their own heavy compression. Colour order can flip, as OpenCV uses blue, green, red, and everyone else uses red, green, blue. Image orientation might be stored as a tag and ignored. Save a sample of real production images, and compare them with your training data.
Summary
Train on images like the ones that you'll really see. Next, what does each neuron actually look at?
Key takeaways- JPEG 50 is often a sweet spot
- Add compression to training
- Watch BGR versus RGB