This document introduces the problem of classifying emails as spam or ham (not spam) using a naive Bayes classifier. It explains that a naive Bayes classifier calculates the probability that an email contains spam or ham given the presence of certain words or "tokens" by using Bayes' theorem and making the assumption that tokens are independent. The goal is to build a basic spam classifier from scratch in F# that predicts whether a short message service (SMS) message is spam or ham based on the tokens it contains. Readers are instructed to use an existing implementation to create their initial classifier and then improve upon it.

1. Ham or Spam?
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 

2. The goal for tonight
» Take a classic Machine Learning problem
» Write some code and have
fun
» Write a classifier, from scratch, using F#
» Learn some Machine Learning concepts
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 

3. Imagine 20% of your email is Spam…
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 

4. … your default guess should be Ham
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 

5. What if I told you the Subject was…
Subject: Nigerian Diamonds!!!
From: prince94623@spammermail.org
Dear friend,
Based on the further explicit investment
information about your country from my
research i wish to invest in your country under
your supervision.
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 

6. Nigerian
Nigerian
Diamonds
!!!
Diamonds
!!!
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 

7. Nigeria
Ham
Ham
100%
100%
Ham
Spam
Spam
Spam
Nigeria
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 

8. Bayes Theorem
Proba (email is Spam, if contains “Nigeria”) =
P (email contains “Nigeria”, if Spam) x P (email is Spam)
P (email contains “Nigeria”)
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 

9. This can be used to classify text
P(Spam|“Nigeria”) = P(“Nigeria”|Spam) x P(Spam) / P(“Nigeria”)
P(Ham|“Nigeria”) = P(“Nigeria”|Ham) x P(Ham) / P(“Nigeria”)
If P(Spam) > P(Ham), it’s “Crazy Tasty” Spam
Note: we can actually ignore P(“Nigeria”) to make a decision
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 

10. Bayes Theorem weights 2 components
P(Spam|“Nigeria”) = P(“Nigeria”|Spam) x P(Spam)
How likely is it that I
observe the word
“Nigeria” in a Spam
email?
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 
How likely is it that an email is
Spam, “in general”

11. Naïve Bayes Classifier
» Break text into Tokens (“Nigeria”, “Diamond”, …)
» Compute the probability that text is Ham or Spam, given
presence/absence of each Token
» Combine probabilities into one number
P(Spam|Tokens)=
P(T1|Spam)xP(T2|Spam)x … xP(Tn|Spam) x P(Spam) /
P(Tokens)
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 

12. Why “Naïve”?
» Considers that impact of Tokens are independent
» Suppose “Nigerian Diamonds” always shows up together
» [Nigerian], [Diamonds] will be “double-counted”
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 

13. Your mission
» Figure out if SMS is Spam or Ham given a Token
» Use existing implementation to build a basic classifier
» Use your brains to make a better classifier
» Project/guided script available at
»www.github.com/c4fsharp/dojo-ham-or-spam
© Mathias Brandewinder, 2013. Use freely, attributions appreciated 