Naive Bayes in Code
MultinomialNB.
Earlier, we updated the chance of spam from one word, by hand. Now let's train a real naive Bayes spam filter on a few messages, with scikit learn, and see it classify new ones.

- Bayes by hand, earlier
- Now a real filter
- Train and classify
A tiny spam filter
Count vectorizer turns each message into word counts. Multinomial NB learns how often each word appears in spam and in normal messages. Then predict proba gives the probability of spam for new messages. Run it. The prize message is very likely spam, and the lunch message is not.
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
texts = ["win a free prize now", "free money click now", "claim your free prize",
"lunch at noon tomorrow", "see you at the meeting", "notes from the meeting"]
labels = [1, 1, 1, 0, 0, 0]
vec = CountVectorizer()
model = MultinomialNB().fit(vec.fit_transform(texts), labels)
for msg in ["free prize inside", "meeting moved to noon"]:
p = model.predict_proba(vec.transform([msg]))[0, 1]
print(f"{msg!r}: P(spam) = {p:.2f}")'free prize inside': P(spam) = 0.93 'meeting moved to noon': P(spam) = 0.15
Smoothing
What if a word never appeared in spam during training? Its probability would be zero, and one unseen word would wipe out all the other evidence. Naive Bayes adds a small count to every word, called Laplace smoothing, so nothing is ever exactly impossible. In scikit learn, that's the alpha setting.
| key | value |
|---|---|
| CountVectorizer | text to word counts |
| MultinomialNB | learns word rates |
| Smoothing | add a small count |
| alpha | default 1.0 |
Summary
A naive Bayes filter turns words into counts, and combines their evidence with Bayes' theorem. Smoothing handles unseen words. Next, measuring how different two distributions are.
Key takeaways- CountVectorizer
- MultinomialNB
- Laplace smoothing