Logistic Regression
Probabilities from a line.
Will a student pass, given the hours they studied? Logistic regression takes a line, W times hours plus B, and passes it through the sigmoid. The result is the probability of passing.

- Pass or fail?
- A line, then a sigmoid
- A probability
Fitting and predicting
Here are sixteen students, with their hours, and whether they passed. Scikit learn's logistic regression finds the best W and B. Predict proba gives probabilities for fail and pass. Run it. One hour of study gives a low chance of passing. Eight hours gives a very high chance.
from sklearn.linear_model import LogisticRegression
from exam import hours, passed
model = LogisticRegression().fit(hours.reshape(-1, 1), passed)
for h in [1, 4, 8]:
p = model.predict_proba([[h]])[0, 1]
print(f"{h} hours: P(pass) = {p:.2f}")1 hours: P(pass) = 0.05 4 hours: P(pass) = 0.44 8 hours: P(pass) = 0.97
Also uses exam.py
import numpy as np
hours = np.array([0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 8, 9])
passed = np.array([0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1])How it learns
Logistic regression is trained with gradient descent, using the cross entropy loss, not squared error. The output is a probability, and we choose a threshold, usually a half, to turn it into a yes or no. It's simple, fast, and easy to explain, so it's often the first classifier people try.
| key | value |
|---|---|
| Model | σ(w · x + b) |
| Output | P(class = 1) |
| Threshold | usually 0.5 |
| Loss | cross-entropy |
Summary
Logistic regression is a line passed through a sigmoid. It outputs probabilities, and it's a great first classifier. Next, softmax, for more than two classes.
Key takeaways- σ(w·x + b)
- predict_proba
- Threshold at 0.5