The Sigmoid
Squash into 0 to 1.
A line can output any number, like minus five, or a hundred. But a probability must be between zero and one. The sigmoid function squashes any number into that range. It's the bridge from a line to a probability.

- Lines output any number
- Probabilities must be 0 to 1
- The sigmoid squashes
The S curve
The sigmoid is one, divided by one plus E to the minus Z. Big positive inputs give nearly one. Big negative inputs give nearly zero. And zero gives exactly a half. Run it. It makes a smooth S shaped curve.
import numpy as np
import matplotlib.pyplot as plt
def sigmoid(z):
return 1 / (1 + np.exp(-z))
z = np.linspace(-8, 8, 200)
plt.plot(z, sigmoid(z))
plt.axhline(0.5, color="#22d3ee", ls="--"); plt.axvline(0, color="#22d3ee", ls="--")
plt.title("sigmoid(z) = 1 / (1 + e^-z)")
plt.savefig("sigmoid.png")A handy derivative
The sigmoid has a beautifully simple derivative. Sigmoid of Z, times one minus sigmoid of Z. We used it in the backpropagation lesson. Its slope is biggest at zero, and nearly flat at the ends. When outputs get stuck near zero or one, the gradients get tiny, which can slow learning.
| key | value |
|---|---|
| σ(z) | 1 ÷ (1 + e⁻ᶻ) |
| σ(0) | 0.5 |
| σ'(z) | σ(z) × (1 − σ(z)) |
| Flat ends | tiny gradients |
Summary
The sigmoid squashes any number into a probability between zero and one. Next, using it to build a classifier, logistic regression.
Key takeaways- 1 / (1 + e^−z)
- 0.5 at zero
- S-shaped