Images as Waves
The frequency domain.
A new strip light goes up over the belt, and suddenly fine horizontal stripes appear across each photo. It's interference between the lamp and the camera. No normal filter can remove it, without blurring everything. But there's a different way of looking at an image, and from there, the stripes are trivial to remove.
- A new strip light
- Stripes across every photo
- A different way of looking
Every image is made of waves
Here's a beautiful idea. Any image can be built by adding together simple waves, of different frequencies. Smooth areas are low frequencies. Edges and fine detail are high frequencies. The Fourier transform breaks an image into those waves. And a regular pattern, such as stripes, turns into a single bright spot.
Deleting stripes
We add stripes to a photo, with a period of eight rows. The FFT converts the image to frequencies, and shift moves the centre to the middle. Our stripes appear as two bright spots, above and below the centre. So we simply set those small regions to zero. Then the inverse transform brings us back to a normal image. Run it. The stripe strength falls from twenty one to about one. And the rest of the image is almost untouched, as we only removed two tiny spots. Try doing that with a blur.
import cv2
import numpy as np
from belt import belt
img, _, _ = belt()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY).astype(float)
gray += 30 * np.sin(np.arange(gray.shape[0]) * 2 * np.pi / 8)[:, None]
f = np.fft.fftshift(np.fft.fft2(gray))
h, w = gray.shape
y = h // 2 + round(h / 8)
f[y - 3:y + 4, w // 2 - 3:w // 2 + 4] = 0
y = h // 2 - round(h / 8)
f[y - 3:y + 4, w // 2 - 3:w // 2 + 4] = 0
fixed = np.fft.ifft2(np.fft.ifftshift(f)).real
row_noise = lambda a: a.mean(axis=1).std()
print(f"Stripes before: {row_noise(gray):.1f}, after: {row_noise(fixed):.1f}")
cv2.imwrite("fft.png", np.hstack([gray, fixed]).clip(0, 255).astype(np.uint8))Stripes before: 21.3, after: 1.2
Also uses belt.py
import numpy as np
import cv2
def belt(n=7, seed=1, size=(480, 640), bruises=2, light=1.0):
"""Draw a simulated conveyor belt of apples. Returns the image and the true boxes."""
rng = np.random.default_rng(seed)
h, w = size
img = np.full((h, w, 3), (70, 75, 80), np.uint8)
img = cv2.add(img, rng.normal(0, 6, img.shape).clip(0, 30).astype(np.uint8))
boxes, spots, taken = [], [], []
while len(boxes) < n:
r = int(rng.integers(32, 48))
x, y = int(rng.integers(r + 5, w - r - 5)), int(rng.integers(r + 5, h - r - 5))
if any((x - a) ** 2 + (y - b) ** 2 < (r + c + 8) ** 2 for a, b, c in taken):
continue
taken.append((x, y, r))
red = int(rng.integers(150, 225))
cv2.circle(img, (x, y), r, (35, 40, red), -1, cv2.LINE_AA)
cv2.circle(img, (x - r // 3, y - r // 3), r // 4, (90, 90, min(255, red + 30)), -1, cv2.LINE_AA)
boxes.append((x - r, y - r, x + r, y + r))
for x, y, r in taken[:bruises]:
cv2.ellipse(img, (x + r // 3, y + r // 4), (r // 3, r // 4), 20, 0, 360, (25, 45, 70), -1, cv2.LINE_AA)
spots.append((x, y, r))
img = cv2.GaussianBlur(img, (3, 3), 0)
return np.clip(img * light, 0, 255).astype(np.uint8), boxes, spotsSummary
Some problems are easy in the world of frequencies. Next, before more software, let's talk about the hardware that decides everything.
Key takeaways- Images are sums of waves
- Patterns become bright spots
- Delete spots, delete the pattern