Depth from Two Views

Hold one finger in front of your face, and close each eye in turn. Your finger jumps sideways. Nearby things jump a lot, and far things barely move. That jump is called disparity, and your brain uses it to see depth. We can do the same, with two cameras.

• Close each eye in turn
• Your finger jumps
• That jump gives depth

Depth from disparity

The formula is simple. Depth equals focal length, times the distance between the cameras, divided by disparity. Run it. A large disparity means something's close. A small one means it's far. And notice how a one pixel error matters more and more as things get further away. That's why stereo cameras are excellent up close, and poor at long range.

focal, baseline = 800.0, 0.06

for disparity in [80, 40, 16, 8, 4]:
    depth = focal * baseline / disparity
    print(f"disparity {disparity:>3} pixels  ->  depth {depth:5.2f} m")

disparity  80 pixels  ->  depth  0.60 m
disparity  40 pixels  ->  depth  1.20 m
disparity  16 pixels  ->  depth  3.00 m
disparity   8 pixels  ->  depth  6.00 m
disparity   4 pixels  ->  depth 12.00 m

A depth map

Stereo matching finds disparity for every pixel, by comparing small blocks between the left and right images. We simulate a right view, with the central region shifted more, as though it's closer. Run it. The result is a depth map, with bright areas that are near. For AppleEye, that means measuring the true size of each apple. Today, neural networks can even estimate depth from a single photo, using what they've learned about the world.

import cv2
import numpy as np
import matplotlib.pyplot as plt
from skimage import data

left = cv2.cvtColor(data.astronaut(), cv2.COLOR_RGB2GRAY)
h, w = left.shape
right = np.zeros_like(left)
yy, xx = np.mgrid[0:h, 0:w]
shift = (((xx - 256) ** 2 + (yy - 200) ** 2) < 12000) * 16 + 8
for y in range(h):
    for x in range(w):
        if x - shift[y, x] >= 0:
            right[y, x - shift[y, x]] = left[y, x]

depth = cv2.StereoBM_create(numDisparities=32, blockSize=15).compute(left, right)
plt.figure(figsize=(5, 4)); plt.imshow(depth, cmap="magma"); plt.axis("off")
plt.tight_layout(); plt.savefig("depth.png", dpi=110)
print("Depth map shape:", depth.shape)

Depth map shape: (512, 512)

Summary