Measuring Speed
Optical flow.
Air jets knock bad apples off the belt, and timing is everything. Fire too soon, or too late, and you'll hit the wrong fruit. The belt's speed drifts during the day. So we need to measure it, from the camera, in every frame.
- Air jets knock out bad apples
- Timing is everything
- Measure speed from the camera
Tracking points
Here are two frames, where the belt has moved by nine pixels. Good features to track finds small corners that are easy to follow. Then Lucas Kanade finds where each of those moved to. We take the median, so that a few bad tracks can't spoil it. Run it. Two hundred points tracked. The speed comes out at exactly nine pixels per frame, and sideways drift is zero. Multiply that by your frame rate and calibration, and you've got millimetres per second.
import cv2
import numpy as np
from belt import belt
a, _, _ = belt(n=6, seed=4)
shift = np.float32([[1, 0, 9], [0, 1, 0]])
b = cv2.warpAffine(a, shift, (640, 480), borderMode=cv2.BORDER_REPLICATE)
g1, g2 = cv2.cvtColor(a, cv2.COLOR_BGR2GRAY), cv2.cvtColor(b, cv2.COLOR_BGR2GRAY)
points = cv2.goodFeaturesToTrack(g1, 200, 0.05, 10)
moved, ok, _ = cv2.calcOpticalFlowPyrLK(g1, g2, points, None)
flow = (moved - points)[ok.ravel() == 1].reshape(-1, 2)
print("Points tracked:", len(flow))
print(f"Belt speed: {np.median(flow[:, 0]):.1f} pixels per frame")
print(f"Sideways drift: {np.median(flow[:, 1]):.1f}")Points tracked: 200 Belt speed: 9.0 pixels per frame Sideways drift: 0.0
Also uses belt.py
import numpy as np
import cv2
def belt(n=7, seed=1, size=(480, 640), bruises=2, light=1.0):
"""Draw a simulated conveyor belt of apples. Returns the image and the true boxes."""
rng = np.random.default_rng(seed)
h, w = size
img = np.full((h, w, 3), (70, 75, 80), np.uint8)
img = cv2.add(img, rng.normal(0, 6, img.shape).clip(0, 30).astype(np.uint8))
boxes, spots, taken = [], [], []
while len(boxes) < n:
r = int(rng.integers(32, 48))
x, y = int(rng.integers(r + 5, w - r - 5)), int(rng.integers(r + 5, h - r - 5))
if any((x - a) ** 2 + (y - b) ** 2 < (r + c + 8) ** 2 for a, b, c in taken):
continue
taken.append((x, y, r))
red = int(rng.integers(150, 225))
cv2.circle(img, (x, y), r, (35, 40, red), -1, cv2.LINE_AA)
cv2.circle(img, (x - r // 3, y - r // 3), r // 4, (90, 90, min(255, red + 30)), -1, cv2.LINE_AA)
boxes.append((x - r, y - r, x + r, y + r))
for x, y, r in taken[:bruises]:
cv2.ellipse(img, (x + r // 3, y + r // 4), (r // 3, r // 4), 20, 0, 360, (25, 45, 70), -1, cv2.LINE_AA)
spots.append((x, y, r))
img = cv2.GaussianBlur(img, (3, 3), 0)
return np.clip(img * light, 0, 255).astype(np.uint8), boxes, spotsSummary
Follow small corners, and take the median. Next, let's use tracking to solve the owner's question properly.
Key takeaways- Track corners between frames
- The median resists bad tracks
- Convert with calibration