White Balance
When red isn't red.
At Orchard Vision, ripeness is judged by colour. But in the morning the shed is lit by cold daylight, and in the evening by warm lamps. The same apple looks different at different hours. Our brains correct for this automatically, so we never notice. A camera doesn't.
- Ripeness is judged by colour
- Morning and evening light differ
- Our brains correct, cameras don't
The grey world trick
We add a colour cast to our image, as a warm lamp would. Then we apply a neat idea called the grey world assumption. On average, a normal scene ought to be about neutral grey. So we scale each channel, so that their averages match. Run it. The tinted channels were sixty eight, eighty three, and seventy two, which is quite unbalanced. After the fix, they all sit at seventy four. It's two lines of code. In practice, it's even better to put a small grey card in the camera's view, and to calibrate against that.
import numpy as np
from belt import belt
img, _, _ = belt()
tinted = np.clip(img * np.array([1.0, 1.15, 0.75]), 0, 255).astype(np.uint8)
means = tinted.reshape(-1, 3).mean(axis=0)
fixed = np.clip(tinted * (means.mean() / means), 0, 255).astype(np.uint8)
print("Tinted channel means:", means.round(0).tolist())
print("Fixed channel means: ", fixed.reshape(-1, 3).mean(axis=0).round(0).tolist())Tinted channel means: [68.0, 83.0, 72.0] Fixed channel means: [74.0, 74.0, 74.0]
Also uses belt.py
import numpy as np
import cv2
def belt(n=7, seed=1, size=(480, 640), bruises=2, light=1.0):
"""Draw a simulated conveyor belt of apples. Returns the image and the true boxes."""
rng = np.random.default_rng(seed)
h, w = size
img = np.full((h, w, 3), (70, 75, 80), np.uint8)
img = cv2.add(img, rng.normal(0, 6, img.shape).clip(0, 30).astype(np.uint8))
boxes, spots, taken = [], [], []
while len(boxes) < n:
r = int(rng.integers(32, 48))
x, y = int(rng.integers(r + 5, w - r - 5)), int(rng.integers(r + 5, h - r - 5))
if any((x - a) ** 2 + (y - b) ** 2 < (r + c + 8) ** 2 for a, b, c in taken):
continue
taken.append((x, y, r))
red = int(rng.integers(150, 225))
cv2.circle(img, (x, y), r, (35, 40, red), -1, cv2.LINE_AA)
cv2.circle(img, (x - r // 3, y - r // 3), r // 4, (90, 90, min(255, red + 30)), -1, cv2.LINE_AA)
boxes.append((x - r, y - r, x + r, y + r))
for x, y, r in taken[:bruises]:
cv2.ellipse(img, (x + r // 3, y + r // 4), (r // 3, r // 4), 20, 0, 360, (25, 45, 70), -1, cv2.LINE_AA)
spots.append((x, y, r))
img = cv2.GaussianBlur(img, (3, 3), 0)
return np.clip(img * light, 0, 255).astype(np.uint8), boxes, spotsSummary
Balance your colours before you trust them. Next, a problem that gets worse when the light gets low. Noise.
Key takeaways- Light colour changes pixel colour
- Grey world balances channels
- A grey card works best