Practice: A Model from Scratch
Logistic regression by hand.
For the last project, let's build logistic regression ourselves, using only NumPy. Dot products, the sigmoid, the cross entropy gradient, and gradient descent. Every piece comes from this course.

- Logistic regression by hand
- Only NumPy
- Every piece from this course
Training loop
We make a dataset with four features, and standardize it. The model is the sigmoid of X times W, plus B. The gradient of the cross entropy loss is X transpose, times the errors, divided by N. We take five hundred gradient descent steps. Run it. Our hand built model matches scikit learn's accuracy.
import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
X, y = make_classification(n_samples=1000, n_features=4, random_state=1)
X = (X - X.mean(0)) / X.std(0)
w, b = np.zeros(4), 0.0
for _ in range(500):
p = 1 / (1 + np.exp(-(X @ w + b)))
w -= 0.5 * X.T @ (p - y) / len(y)
b -= 0.5 * (p - y).mean()
print("ours: ", ((X @ w + b > 0) == y).mean())
print("sklearn:", LogisticRegression().fit(X, y).score(X, y))ours: 0.848 sklearn: 0.85
What you just used
That short loop used almost every big idea in the course. Standardizing from the spread section. Dot products and matrix multiplication from linear algebra. The sigmoid and cross entropy from machine learning maths. And gradient descent from calculus. Neural networks are this same loop, with more layers.
| key | value |
|---|---|
| Standardize | z-scores |
| X @ w | matrix multiplication |
| Sigmoid + cross-entropy | probabilities and loss |
| Update | gradient descent |
Summary
You built a classifier from scratch, with nothing but the maths from this course. Next, a bonus section that goes a little deeper, starting with log transforms.
Key takeaways- Standardize
- Sigmoid of X @ w
- Gradient descent