Backpropagation Idea
The chain rule in networks.
A neural network can have millions of weights. To train it, we need the gradient of the error with respect to every one of them. Backpropagation computes them all efficiently, using the chain rule, from the output backwards.

- Millions of weights
- A gradient for each
- Backpropagation
One tiny neuron
Here's a neuron with one weight. It computes W times X, passes it through a sigmoid, and compares the output with the target. Backpropagation multiplies the local derivatives, from the loss back to W. Then we check it with a tiny nudge. Run it. The chain rule and the nudge agree.
import numpy as np
sigmoid = lambda z: 1 / (1 + np.exp(-z))
x, target, w = 2.0, 1.0, 0.3
def loss(w):
return (sigmoid(w * x) - target) ** 2
out = sigmoid(w * x)
grad = 2 * (out - target) * out * (1 - out) * x
print("backprop:", round(grad, 5))
print("nudge: ", round((loss(w + 1e-6) - loss(w)) / 1e-6, 5))backprop: -0.32427 nudge: -0.32427
Forward, then backward
Training a network repeats four steps. The forward pass computes the prediction, layer by layer. The loss measures the error. The backward pass sends gradients back through each layer, with the chain rule. And the optimizer updates every weight a little, using gradient descent.
- Forwardcompute the prediction
- Lossmeasure the error
- Backwardchain rule, layer by layer
- Updategradient descent step

Summary
Backpropagation is the chain rule, applied from the output back through every layer. Libraries like PyTorch do it automatically. Next, a quiz on calculus.
Key takeaways- Chain rule backwards
- Check with a nudge
- Forward, loss, backward, update