Maximum Likelihood
The most likely parameters.
A coin lands heads seven times in ten flips. What's your best guess for its chance of heads? Most people say zero point seven. Maximum likelihood explains why that's the right answer, and it's how most models choose their parameters.

- 7 heads in 10 flips
- Best guess for P(heads)?
- Maximum likelihood
The likelihood curve
For every possible value of P, we compute the likelihood, the probability of seeing exactly seven heads in ten flips if P were true. Run it. The curve peaks at exactly zero point seven. That's the value of P that makes our data most likely.
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import binom
p = np.linspace(0, 1, 201)
likelihood = binom.pmf(7, 10, p)
plt.plot(p, likelihood)
plt.axvline(p[likelihood.argmax()], color="#22d3ee", ls="--")
plt.title(f"Best p = {p[likelihood.argmax()]:.2f}"); plt.xlabel("p")
plt.savefig("likelihood.png")Likelihood in machine learning
Training a model is usually maximum likelihood in disguise. Minimizing cross entropy is the same as maximizing the likelihood of the labels. And minimizing squared error is the same as maximum likelihood, if the errors are normal. In practice, we maximize the log likelihood, because adding logs is easier than multiplying tiny probabilities.
| key | value |
|---|---|
| Likelihood | P(data | parameters) |
| MLE | parameters that maximize it |
| Cross-entropy | = negative log likelihood |
| Logs | sums instead of products |
Summary
Maximum likelihood picks the parameters that make the observed data most probable. Most training is maximum likelihood. Next, bias and variance.
Key takeaways- Likelihood = P(data | θ)
- Pick the peak
- Loss = −log likelihood