Bayes for Spam
The naive Bayes idea.
Early spam filters used Bayes' theorem, and it's still a great first model for text. The idea is simple. Some words, like free, appear much more often in spam. So seeing them should raise our belief that an email is spam.

- Early spam filters used Bayes
- Some words appear in spam more
- Evidence raises belief
One word of evidence
Say forty percent of emails are spam. The word free appears in sixty percent of spam, but only five percent of normal emails. Run it. After seeing free, the chance of spam jumps from forty percent to about eighty nine percent.
prior_spam = 0.40
p_free_if_spam = 0.60
p_free_if_ham = 0.05
p_free = p_free_if_spam * prior_spam + p_free_if_ham * (1 - prior_spam)
posterior = p_free_if_spam * prior_spam / p_free
print("before:", prior_spam)
print("after seeing 'free':", round(posterior, 3))before: 0.4 after seeing 'free': 0.889
Why naive?
A real email has many words. Naive Bayes multiplies the evidence from every word, as if each word were independent of the others. That's not really true, which is why it's called naive. But it works surprisingly well, it's very fast, and scikit learn has it built in, as multinomial NB.
| key | value |
|---|---|
| Each word | adds evidence |
| Naive | assumes words are independent |
| Fast | great first text model |
| sklearn | MultinomialNB |
Summary
Naive Bayes updates the chance of spam word by word. It's simple, fast, and a classic first model for text. Next, counting, and permutations.
Key takeaways- Prior × word evidence
- Naive: independent words
- MultinomialNB