Second Derivatives
Curvature and minimums.
The first derivative tells you the slope. The second derivative tells you how the slope is changing. That's curvature. Is the curve bending upwards like a bowl, or downwards like a hill?

- First derivative: slope
- Second derivative: change in slope
- Bowl or hill?
Bowl or hill?
At a flat point, the first derivative is zero. The second derivative tells us what kind of flat point it is. We use Sim Pi on X cubed minus three X, which has two flat points. Run it. At minus one, the second derivative is negative, so it's a hilltop. At one, it's positive, so it's the bottom of a valley.
import sympy as sp
x = sp.symbols("x")
f = x**3 - 3 * x
flat = sp.solve(sp.diff(f, x), x)
for p in flat:
curve = sp.diff(f, x, 2).subs(x, p)
print(f"x = {p}: second derivative = {curve}")x = -1: second derivative = -6 x = 1: second derivative = 6
Curvature in machine learning
A loss that curves sharply needs small steps, or gradient descent overshoots. A gently curving loss can take bigger steps. Some optimizers use second derivative information, called the Hessian, to choose better steps. And a positive second derivative everywhere means the loss is convex, a single bowl with one bottom.
| key | value |
|---|---|
| f''(x) > 0 | bowl, a minimum |
| f''(x) < 0 | hill, a maximum |
| Hessian | second derivatives in many dimensions |
| Convex | one bowl, one bottom |
Summary
The second derivative measures curvature. Positive is a bowl, negative is a hill. Next, the difference between the lowest point, and just a low point.
Key takeaways- f'' > 0: minimum
- f'' < 0: maximum
- Convex: one bowl