Explore a Naive Bayes spam classifier interactively. See how word likelihoods and prior probabilities combine to classify a message.
1Start with a guess, then update it
Before reading an email, the only clue is how common spam is. Each word you read is evidence that moves the guess up or down. That's Bayes' rule.
2Naive: every word counts on its own
The model pretends words are unrelated, as if “free” and “money” never travel together. That's not true, but it makes the counting simple, and it still works surprisingly well.
3Multiply, compare, pick the bigger
For each class, multiply how common the class is by the chance of each word in it. The class with the bigger result wins.
Watch it think
A tiny spam filter
The filter has learned from the 12 emails below (made up for this page, so you can count every word yourself). Pick an email or type your own, then step through it: it starts from how common spam is, and each word pushes the guess towards spam or towards normal email.
mostly spam words
Final answer (press Step to see how it gets there)
Press Step to start from the prior and add one word at a time.
What the filter learned from
Spam (6)
- win free money now
- free prize claim now
- cheap pills free offer
- win a free prize today
- claim your cash prize now
- limited offer win cash
Not spam (6)
- meeting moved to monday
- can we talk about the project today
- free lunch at noon on monday
- here is the offer for the project
- thanks for the meeting notes
- see you at lunch today
Every word it knows, counted
Spam has 26 words in total and normal email 34; there are 35 different words. Each chance is (count + α) ÷ (class total + α × 35), with α = 1. Sorted from most spammy to least.
| word | in spam | in normal | P(word | spam) | P(word | normal) |
|---|---|---|---|---|
| now | 3 | 0 | 0.066 | 0.014 |
| prize | 3 | 0 | 0.066 | 0.014 |
| win | 3 | 0 | 0.066 | 0.014 |
| cash | 2 | 0 | 0.049 | 0.014 |
| claim | 2 | 0 | 0.049 | 0.014 |
| free | 4 | 1 | 0.082 | 0.029 |
| a | 1 | 0 | 0.033 | 0.014 |
| cheap | 1 | 0 | 0.033 | 0.014 |
| limited | 1 | 0 | 0.033 | 0.014 |
| money | 1 | 0 | 0.033 | 0.014 |
| pills | 1 | 0 | 0.033 | 0.014 |
| your | 1 | 0 | 0.033 | 0.014 |
| offer | 2 | 1 | 0.049 | 0.029 |
| today | 1 | 2 | 0.033 | 0.043 |
| about | 0 | 1 | 0.016 | 0.029 |
| can | 0 | 1 | 0.016 | 0.029 |
| here | 0 | 1 | 0.016 | 0.029 |
| is | 0 | 1 | 0.016 | 0.029 |
| moved | 0 | 1 | 0.016 | 0.029 |
| noon | 0 | 1 | 0.016 | 0.029 |
| notes | 0 | 1 | 0.016 | 0.029 |
| on | 0 | 1 | 0.016 | 0.029 |
| see | 0 | 1 | 0.016 | 0.029 |
| talk | 0 | 1 | 0.016 | 0.029 |
| thanks | 0 | 1 | 0.016 | 0.029 |
| to | 0 | 1 | 0.016 | 0.029 |
| we | 0 | 1 | 0.016 | 0.029 |
| you | 0 | 1 | 0.016 | 0.029 |
| at | 0 | 2 | 0.016 | 0.043 |
| for | 0 | 2 | 0.016 | 0.043 |
| lunch | 0 | 2 | 0.016 | 0.043 |
| meeting | 0 | 2 | 0.016 | 0.043 |
| monday | 0 | 2 | 0.016 | 0.043 |
| project | 0 | 2 | 0.016 | 0.043 |
| the | 0 | 4 | 0.016 | 0.072 |
Words in the email you're checking are highlighted. With smoothing off, zeros appear in red.
No magic, just counting
The maths, step by step
We work through the email “free cash today” by hand, with smoothing on (α = 1). Every number is calculated by the same code as the filter above.
- 1
Bayes' rule in plain words
The chance an email is spam, given its words, is: how common spam is, times how likely these words are in spam, divided by how likely these words are at all.P(spam | words) = P(spam) × P(words | spam) ÷ P(words)
P(words) is the same for both classes, so to decide we can skip it and just compare P(class) × P(words | class). We divide at the end only to turn the scores into percentages.
- 2
The naive step
Working out P(words | spam) for the exact sentence would need every possible sentence. So we pretend each word is independent of the others, and multiply their chances.P(“free cash today” | spam) ≈ P(free | spam) × P(cash | spam) × P(today | spam)
That's the “naive” assumption. It's what makes the model so simple to train: just count words.
- 3
Count the words, with smoothing
Each chance is (how often the word appears in that class + α) ÷ (all words in that class + α × the number of different words). Spam has 26 words, normal email 34, and there are 35 different words.P(word | class) = (count + 1) ÷ (class total + 35)
P(free | spam) = (4 + 1) ÷ (26 + 35) = 0.0820
P(free | normal) = (1 + 1) ÷ (34 + 35) = 0.0290
P(cash | spam) = (2 + 1) ÷ (26 + 35) = 0.0492
P(cash | normal) = (0 + 1) ÷ (34 + 35) = 0.0145
P(today | spam) = (1 + 1) ÷ (26 + 35) = 0.0328
P(today | normal) = (2 + 1) ÷ (34 + 35) = 0.0435
- 4
Multiply it all together
Both classes had 6 of the 12 training emails, so each prior is 0.5.spam score = 0.5 × 0.082 × 0.0492 × 0.0328 = 6.608e-5
normal score = 0.5 × 0.029 × 0.0145 × 0.0435 = 9.132e-6
6.608e-5 means 0.00006608. Both are tiny, which is normal: what matters is that the spam score is about 7.2 times bigger.
- 5
Turn the scores into percentages
Divide each score by the two added together. That's the P(words) we skipped in step 1.P(spam | email) = 6.608e-5 ÷ (6.608e-5 + 9.132e-6) = 0.8786
So the filter says 87.9% spam.
- 6
Why real code adds logs instead
A long email multiplies hundreds of small numbers, and the result gets so tiny that a computer rounds it to 0. Taking logs turns multiplying into adding, which stays readable. The answer is exactly the same.ln(spam score) = ln 0.5 + ln 0.082 + ln 0.049 + ln 0.033 = -9.6246
ln(normal score) = ln 0.5 + ln 0.029 + ln 0.014 + ln 0.043 = -11.6037
The bigger (less negative) log wins, and e^(log) gives back the scores from step 4.
- 7
Why smoothing matters
Without smoothing (α = 0), a word that never appeared in spam gets a chance of exactly 0, and multiplying by 0 makes the whole spam score 0, however spammy the other words are.P(meeting | spam), α = 0 = 0 ÷ 26 = 0.0000
So “free offer today meeting” comes out 100% normal with smoothing off, but 58% spam with it on. Try it in the filter above.
An honest caveat
Good at deciding, bad at percentages
scikit-learn's documentation puts it plainly: Naive Bayes is “a decent classifier” but “a bad estimator” of probabilities. Because it treats every word as fresh evidence, repeating a word, or using several words that mean the same thing, piles up certainty that isn't really there. Repeat “free” and watch.
“free free free”
95.8% spam
The decision (spam or not) is often still right. Just don't read 99.99% as “almost certainly”: if you need trustworthy probabilities, calibrate them or use a different model.
When the features are numbers
Gaussian Naive Bayes
Words can be counted, but measurements like weight can't. Gaussian Naive Bayes fits a bell curve (a normal distribution) to each class, using the average and the spread of the training values, and asks which curve is higher at the new value. Here: is a fruit an apple or an orange, from its weight alone?
- apples: mean, spread
- 162.5 g, ±11.5
- oranges: mean, spread
- 202.5 g, ±11.5
68.2% apple · 31.8% orange
Both classes have the same number of examples, so the prior is 50/50 and only the curves' heights decide. The spread is the standard deviation, from the variance averaged over n (as scikit-learn's GaussianNB does).
In practice
Kinds of Naive Bayes, pros and cons
Multinomial
Word counts. The classic for text; this page's spam filter.
Bernoulli
Word present or not (yes/no features). Suits short texts; it also counts the absence of a word as evidence.
Gaussian
Numbers like weight or height: one bell curve per class per feature, as above.
Complement
A variant of Multinomial for imbalanced classes. scikit-learn notes it often beats Multinomial on text classification.
Good at
- Very fast to train (it's just counting) and to use.
- Works with little training data.
- Text classification: spam, topics, sentiment.
- Easy to explain: you can show which words pushed the decision.
Watch out for
- The independence assumption is almost never true, so related words get counted twice.
- Its probabilities are overconfident: trust the decision more than the percentage.
- Without smoothing, one unseen word can wipe out a class.
- It can't learn that a combination of words matters more than each word alone.
How it works: Naive Bayes
Step by step
- Pushes towards spam
- Pushes towards not spam
- Ignored (never seen)
Press Start or Step: the step the animation is on lights up below.
- 1
Start from the prior
Before reading anything, the guess is just how common each class was in training.
score[c] = log P(c) - 2
Add each word's evidence
Look up how likely the word is in each class (counted from training, plus smoothing) and multiply it in, or add its log.
score[c] += log P(word | c) - 3
Skip words it never saw
A word that never appeared in training carries no evidence either way, so it's ignored.
if word not in vocab: continue - 4
Turn scores into chances
Divide each score by the total so the two add up to 100%. Both classes share the same divisor, P(email), so it never changes the winner.
P(c | email) = score[c] / Σ score - 5
Pick the bigger one
The class with the higher chance wins.
predict = argmax_c score[c]
Naive Bayes guesses a class by weighing evidence. For an email it asks: how common is spam in general, and how much more often does each of these words show up in spam than in normal email? Each word nudges the guess one way or the other.
Those word chances come straight from counting the training data: “free” appeared 4 times among the 26 spam words, so it's common in spam. Smoothing adds a pretend count of 1 to every word, so a word that never appeared in one class doesn't make that class impossible.
It's called naive because it treats every word as separate evidence, as if “free” and “money” had nothing to do with each other. That's clearly not true, yet the method still sorts emails surprisingly well, which is why it became a classic method for spam filtering.
Where you'll meet it
- Text: spam filters, sorting support tickets, sentiment, language detection.
- When you have little training data, or need something fast to train and explain.
- As a quick baseline to beat before trying heavier models.
Training is a single pass of counting. Classifying an email is one table look-up per word per class.