Loss Functions
Measuring mistakes.
Before a model can improve, it needs a single number that says how wrong it is. That's the loss function. Training means changing the weights to make the loss as small as possible. The choice of loss changes what the model cares about.

- One number: how wrong?
- Training shrinks it
- The choice matters
MSE and MAE
Mean squared error squares each error, so big mistakes count much more. Mean absolute error just takes the size of each error. Here, one prediction is badly wrong. Run it. The one big error pushes the squared error up to twenty point four, but the absolute error is only two point four.
import numpy as np
actual = np.array([10, 12, 11, 13, 12])
predicted = np.array([11, 12, 10, 13, 22])
err = predicted - actual
print("errors:", err)
print("MSE:", (err ** 2).mean())
print("MAE:", np.abs(err).mean())errors: [ 1 0 -1 0 10] MSE: 20.4 MAE: 2.4
Common losses
For predicting numbers, mean squared error is the usual choice. Mean absolute error is better when you don't want outliers to dominate. Huber loss is a mix of the two. And for classification, we use cross entropy, which we'll meet in a few lessons.
| Loss | Task | Note |
|---|---|---|
| MSE | regression | punishes big errors |
| MAE | regression | robust to outliers |
| Huber | regression | a mix of both |
| Cross-entropy | classification | for probabilities |
Summary
A loss function turns mistakes into one number to minimize. Squared error punishes big mistakes, and absolute error is more forgiving. Next, the sigmoid.
Key takeaways- MSE squares errors
- MAE is robust
- Training minimizes loss