The Week Everything Changed

September, twenty twelve. Every year, the best computer vision teams in the world compete to label a million photographs. Progress had been slow, a percent or so each year. Then a small team from Toronto entered, with an old idea that most experts had given up on. They didn't just win. They cut the error by nearly a half. That week, the field changed forever. Their idea was deep learning.

• 2012: the ImageNet contest
• A small team, an old idea
• They nearly halved the error

Layers of understanding

Why did it work so well? In traditional machine learning, people had to design the features by hand. To spot a cat, engineers tried to describe ears and whiskers. A deep network learns these things by itself. The first layers learn tiny edges. The next combine edges into shapes. Then shapes become parts, like eyes and wheels. And finally, parts become whole objects. Each layer builds on the last. That's the deep in deep learning, and it's the most powerful idea in modern A I.

Your first deep network

Let's not wait. Here's a complete deep learning program. We load seventy thousand images of handwritten digits. Our model has a layer of a hundred and twenty eight neurons. Then come the five lines that you'll see for the rest of your career. Predict, measure the loss, clear the old gradients, compute new ones, and take a step. We go through the data three times. Run it. About ninety seven percent, on digits that it has never seen. Twenty five lines. By the end of this course, you'll understand every single word of it.

import torch
from torch import nn
from torchvision import datasets, transforms

root = "data"
tf = transforms.ToTensor()
train = datasets.MNIST(root, train=True, transform=tf)
test = datasets.MNIST(root, train=False, transform=tf)
loader = torch.utils.data.DataLoader(train, batch_size=128, shuffle=True)

model = nn.Sequential(nn.Flatten(), nn.Linear(784, 128), nn.ReLU(),
                      nn.Linear(128, 10))
optimiser = torch.optim.Adam(model.parameters())
loss_fn = nn.CrossEntropyLoss()

for epoch in range(3):
    for images, labels in loader:
        loss = loss_fn(model(images), labels)
        optimiser.zero_grad()
        loss.backward()
        optimiser.step()

X = test.data.float().unsqueeze(1) / 255
accuracy = (model(X).argmax(1) == test.targets).float().mean()
print(f"Accuracy on unseen digits: {accuracy:.3f}")

Accuracy on unseen digits: 0.963

Summary