An Image Is Numbers

When you look at an apple, you see an apple. When a computer looks, it sees a grid of numbers. There's no apple. There's no colour. There's only a table. Everything in computer vision comes down to finding meaning in that table.

• You see an apple
• A computer sees a table
• Find meaning in numbers

A grid of numbers

Every image is a grid of tiny squares called pixels. Each pixel has three numbers, for how much red, green, and blue it holds, from zero to two hundred and fifty five. Watch as we move along a row. The numbers change as we pass through the fruit. A million pixel photo is three million numbers. All that vision models do is turn these numbers into meaning.

Looking at the numbers

Let's look. This photo is a NumPy array. Shape says five hundred and twelve rows, five hundred and twelve columns, and three colour channels. The top left pixel is just three numbers. Values run from zero to two hundred and fifty five. Run it. Over seven hundred thousand numbers, in a single small photo. Reading shape is the first skill in vision, just as in all deep learning.

from skimage import data

image = data.astronaut()

print("Shape:", image.shape)
print("Top-left pixel:", image[0, 0])
print("Smallest and largest:", image.min(), image.max())
print("Total numbers:", image.size)

Shape: (512, 512, 3)
Top-left pixel: [154 147 151]
Smallest and largest: 0 255
Total numbers: 786432

Channels and grayscale

We can pull the three colours apart. Each channel is a gray image. Bright means there's a lot of that colour. Grayscale mixes them, using weights that match the human eye, which is most sensitive to green. Run it. Look at the four images. Each channel shows something different. Many classic algorithms use grayscale, since it's simpler, and shape often matters more than colour.

import numpy as np
import matplotlib.pyplot as plt
from skimage import data

image = data.astronaut()
red, green, blue = image[..., 0], image[..., 1], image[..., 2]
gray = (0.299 * red + 0.587 * green + 0.114 * blue).astype(np.uint8)

fig, ax = plt.subplots(1, 4, figsize=(10, 3))
for a, part, name in zip(ax, [red, green, blue, gray], ["Red", "Green", "Blue", "Gray"]):
    a.imshow(part, cmap="gray"); a.set_title(name); a.axis("off")
plt.tight_layout(); plt.savefig("channels.png", dpi=110)
print("Colour shape:", image.shape, " Gray shape:", gray.shape)

Colour shape: (512, 512, 3)  Gray shape: (512, 512)

Summary