AI Hints
Statistics & Math for ML · Lesson 93 of 138

Derivative Rules

Power, sum, product.

⏱ 2 min read▶ Video at 1:50:23🎓 Free
Watch this lesson in the Statistics & Math for ML courseWatch on YouTube ↗Full Statistics & Math for ML course

We don't need to shrink intervals every time. Mathematicians worked out simple rules that give derivatives directly. A handful of them covers almost everything in machine learning.

Derivative Rules
From the video: Derivative Rules
  • No need to shrink every time
  • Simple rules
  • A handful covers ML

The main rules

The power rule says the derivative of X to the N is N times X to the N minus one. So X squared becomes two X. A constant has a derivative of zero. A constant multiplier stays in front. And the derivative of a sum is the sum of the derivatives.

FunctionDerivative
xⁿn·xⁿ⁻¹
x²2x
constant c0
c·f(x)c·f'(x)
f(x) + g(x)f'(x) + g'(x)
eˣeˣ

Letting SymPy do it

Sim Pi is a Python library for symbolic maths. It applies the rules for us. We define X as a symbol, write a function, and call diff. Run it. Three X squared, plus two X, minus five, has the derivative six X plus two. At X equals one, the slope is eight.

rules.pypython
import sympy as sp

x = sp.symbols("x")
f = 3 * x**2 + 2 * x - 5

df = sp.diff(f, x)
print("f'(x) =", df)
print("slope at x = 1:", df.subs(x, 1))
Output
f'(x) = 6*x + 2
slope at x = 1: 8

Summary

The power, constant, and sum rules give derivatives directly, and Sim Pi can apply them for you. Next, the chain rule, the most important rule for neural networks.

Key takeaways
  • Power rule
  • Sum rule
  • sp.diff
Next up: The Chain Rule — functions inside functions