Derivative Rules
Power, sum, product.
We don't need to shrink intervals every time. Mathematicians worked out simple rules that give derivatives directly. A handful of them covers almost everything in machine learning.

- No need to shrink every time
- Simple rules
- A handful covers ML
The main rules
The power rule says the derivative of X to the N is N times X to the N minus one. So X squared becomes two X. A constant has a derivative of zero. A constant multiplier stays in front. And the derivative of a sum is the sum of the derivatives.
| Function | Derivative |
|---|---|
| xⁿ | n·xⁿ⁻¹ |
| x² | 2x |
| constant c | 0 |
| c·f(x) | c·f'(x) |
| f(x) + g(x) | f'(x) + g'(x) |
| eˣ | eˣ |
Letting SymPy do it
Sim Pi is a Python library for symbolic maths. It applies the rules for us. We define X as a symbol, write a function, and call diff. Run it. Three X squared, plus two X, minus five, has the derivative six X plus two. At X equals one, the slope is eight.
import sympy as sp
x = sp.symbols("x")
f = 3 * x**2 + 2 * x - 5
df = sp.diff(f, x)
print("f'(x) =", df)
print("slope at x = 1:", df.subs(x, 1))f'(x) = 6*x + 2 slope at x = 1: 8
Summary
The power, constant, and sum rules give derivatives directly, and Sim Pi can apply them for you. Next, the chain rule, the most important rule for neural networks.
Key takeaways- Power rule
- Sum rule
- sp.diff