Separating Moving Things
Background subtraction.
If your camera doesn't move, then there's a wonderful shortcut. The background never changes. So anything which is different from it must be something new. That one idea powers security cameras and traffic counters around the world, and it needs almost no compute.
- A still camera
- The background never changes
- Anything different is new
Learning the background
Mog two builds a statistical model of what each pixel normally looks like, and it updates slowly over time. For twenty frames, we show it an empty belt. Then an apple appears, and starts to move. Run it. On frame twenty, nothing is moving. By frame thirty, around four thousand eight hundred pixels have been flagged, which is about the size of our apple. It didn't need any training data, and it took no labels. It just watched for a moment. That's about the cheapest detector that you can build.
import cv2
import numpy as np
from belt import belt
bg = cv2.createBackgroundSubtractorMOG2(history=50, detectShadows=False)
empty, _, _ = belt(n=1, seed=5, bruises=0)
empty[:] = np.median(empty.reshape(-1, 3), axis=0)
for frame in range(40):
image = empty.copy()
x = 40 + frame * 14
if frame > 20:
cv2.circle(image, (x, 240), 40, (40, 40, 200), -1)
mask = bg.apply(image)
if frame in (20, 30, 39):
print(f"Frame {frame}: moving pixels {(mask > 0).sum()}")Frame 20: moving pixels 0 Frame 30: moving pixels 4768 Frame 39: moving pixels 4768
Also uses belt.py
import numpy as np
import cv2
def belt(n=7, seed=1, size=(480, 640), bruises=2, light=1.0):
"""Draw a simulated conveyor belt of apples. Returns the image and the true boxes."""
rng = np.random.default_rng(seed)
h, w = size
img = np.full((h, w, 3), (70, 75, 80), np.uint8)
img = cv2.add(img, rng.normal(0, 6, img.shape).clip(0, 30).astype(np.uint8))
boxes, spots, taken = [], [], []
while len(boxes) < n:
r = int(rng.integers(32, 48))
x, y = int(rng.integers(r + 5, w - r - 5)), int(rng.integers(r + 5, h - r - 5))
if any((x - a) ** 2 + (y - b) ** 2 < (r + c + 8) ** 2 for a, b, c in taken):
continue
taken.append((x, y, r))
red = int(rng.integers(150, 225))
cv2.circle(img, (x, y), r, (35, 40, red), -1, cv2.LINE_AA)
cv2.circle(img, (x - r // 3, y - r // 3), r // 4, (90, 90, min(255, red + 30)), -1, cv2.LINE_AA)
boxes.append((x - r, y - r, x + r, y + r))
for x, y, r in taken[:bruises]:
cv2.ellipse(img, (x + r // 3, y + r // 4), (r // 3, r // 4), 20, 0, 360, (25, 45, 70), -1, cv2.LINE_AA)
spots.append((x, y, r))
img = cv2.GaussianBlur(img, (3, 3), 0)
return np.clip(img * light, 0, 255).astype(np.uint8), boxes, spotsWhere it breaks
Background subtraction has clear limits. Sudden light changes flag the whole frame. Shadows look like objects. Something that stops moving gradually melts into the background. And it's useless on a moving camera. Still, for a fixed camera and a clear scene, it's a fantastic first filter. Run your costly model only on frames where something's actually moving.
Summary
Model the background, and whatever's left is what matters. Next, how fast is the belt going?
Key takeaways- Learn what's normal per pixel
- Needs a still camera
- Use it to trigger bigger models