Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
From Linguistic Rules to Machine Learning              Cohan Sujay Carlos                 Aiaioo Labs               Bangal...
What is a Classifier?A machine learning tool used to apply a label to data.
Classification used in Text Categorization        Politics                   Sports   The UN Security            Warwicksh...
How to Build a Classifier for Text Categorization          How do you tell which label (Politics/Sports) is suitable?  The...
In this case it’s only words    that you will need!
Classification used in Text Categorization                       See the words?   The UN Security                 Warwicks...
Rule-Based Text Categorization                   Gazetteers (word lists)UN Security Council                  WarwickshireA...
Rule-Based to Naïve BayesianHow can you go:from the starting point (word lists)to a really cool classification algorithm  ...
Rule-Based with Weights           Let’s improve the gazetteers with weights   Politics                                 Spo...
Rule-Based with Weights         Let’s improve the gazetteers with weights   Politics                               SportsU...
Rule-Based with Weights   PoliticsUN             1.0   P(Politics|UN)Adopts         0.1   P(Politics|Adopts)Condemnation  ...
Rule-Based with Weights    PoliticsUN          1.0      P(Politics | “UN”)Adopts      0.1      P(Politics | “Adopts”)     ...
Rule-Based with Weights        Politics   UN             1.0      P(Politics | “UN”)   Adopts         0.1      P(Politics ...
Rule-Based with Weights    PoliticsUN           1.0      P(Politics | “UN”)Adopts       0.1      P(Politics | “Adopts”)   ...
Rule-Based with Weights      Politics UN            1.0      P(Politics | “UN”) Adopts        0.1      P(Politics | “Adopt...
Use Bayesian Inversion!In other words, we are looking to turnP(F|E) into P(E|F).There is an equation to do this :P(E|F) = ...
So finally … you have …           Politics   UN                  1.0      P(“UN”|Politics)* P(Politics)/P(“UN”)   Adopts  ...
You have just Learnt               How to Build        A Naïve Bayesian ClassifierStarting from Linguistic Rules (Word Lis...
Upcoming SlideShare
Loading in …5
×

Rules engines to machine learning

679 views

Published on

How do you go from writing rules for classifying data to using powerful machine learning algorithms to do the same in easy comprehensible steps.

Published in: Technology
  • Be the first to comment

Rules engines to machine learning

  1. 1. From Linguistic Rules to Machine Learning Cohan Sujay Carlos Aiaioo Labs Bangalore, India cohan@aiaioo.com
  2. 2. What is a Classifier?A machine learning tool used to apply a label to data.
  3. 3. Classification used in Text Categorization Politics Sports The UN Security Warwickshires Clarke Council adopts its first equalled the first-class clear condemnation of record of seven Syria for its continuing catches for an crackdown on outfielder in an protests, as the army innings but Lancashire continues its advance took control on day into Hama. three.
  4. 4. How to Build a Classifier for Text Categorization How do you tell which label (Politics/Sports) is suitable? The UN Security Warwickshires Clarke Council adopts its first equalled the first-class clear condemnation of record of seven Syria for its continuing catches for an crackdown on outfielder in an protests, as the army innings but Lancashire continues its advance took control on day into Hama. three.
  5. 5. In this case it’s only words that you will need!
  6. 6. Classification used in Text Categorization See the words? The UN Security Warwickshires Clarke Council adopts its first equalled the first-class clear condemnation of record of seven Syria for its continuing catches for an crackdown on outfielder in an protests, as the army innings but Lancashire continues its advance took control on day into Hama. three.
  7. 7. Rule-Based Text Categorization Gazetteers (word lists)UN Security Council WarwickshireAdopts ClarkeCondemnation First-classSyria RecordCrackdown CatchesProtests OutfielderArmy InningsHama Lancashire So you can just use word lists for classification? Yeah, but they won’t work very well. Can you see why word lists alone won’t work very well?
  8. 8. Rule-Based to Naïve BayesianHow can you go:from the starting point (word lists)to a really cool classification algorithm All you need is weights!
  9. 9. Rule-Based with Weights Let’s improve the gazetteers with weights Politics SportsUN 1.0 Warwickshire 0.3Adopts 0.1 Clarke 0.1Condemnation 0.2 First-class 0.6Syria 0.3 Record 0.3Crackdown 0.8 Catches 0.6Protests 1.0 Outfielder 1.0Army 0.8 Innings 0.9Hama 1.0 Lancashire 0.5 These weights are nothing but P(Category|Word).
  10. 10. Rule-Based with Weights Let’s improve the gazetteers with weights Politics SportsUN 1.0 Warwickshire 0.3Adopts 0.1 Clarke 0.1Condemnation 0.2 First-class 0.6Syria 0.3 Record 0.3Crackdown 0.8 Catches 0.6Protests 1.0 Outfielder 1.0Army 0.8 Innings 0.9Hama 1.0 Lancashire 0.5 P(Politics|Word) P(Sports|Word)
  11. 11. Rule-Based with Weights PoliticsUN 1.0 P(Politics|UN)Adopts 0.1 P(Politics|Adopts)Condemnation 0.2 P(Politics|Condemnation)Syria 0.3 P(Politics|Syria)Crackdown 0.8 P(Politics|Crackdown)Protests 1.0 P(Politics|Protests)Army 0.8 P(Politics|Army)Hama 1.0 P(Politics|Hama)
  12. 12. Rule-Based with Weights PoliticsUN 1.0 P(Politics | “UN”)Adopts 0.1 P(Politics | “Adopts”) How can you learn these probabilities automatically?
  13. 13. Rule-Based with Weights Politics UN 1.0 P(Politics | “UN”) Adopts 0.1 P(Politics | “Adopts”) How can you learn these probabilities automatically? Estimation P(Politics | “UN”) = 20/20Statistically not a very accurate estimator - denominator is small.
  14. 14. Rule-Based with Weights PoliticsUN 1.0 P(Politics | “UN”)Adopts 0.1 P(Politics | “Adopts”) How can you learn these probabilities automatically? Instead you EstimateP(“UN”|Politics) = 20/40000{ C(“UN” in politics) / C(all words in category politics) } Statistically this is a better estimator
  15. 15. Rule-Based with Weights Politics UN 1.0 P(Politics | “UN”) Adopts 0.1 P(Politics | “Adopts”) How can you learn these probabilities automatically?A Naïve Bayesian classifier uses a P(Politics | “UN”)estimate calculated from P(“UN”|Politics). That’s so cool! Time to learn how to do that!
  16. 16. Use Bayesian Inversion!In other words, we are looking to turnP(F|E) into P(E|F).There is an equation to do this :P(E|F) = P(F|E) * P(E) / P(F) [Bayesian Inversion]
  17. 17. So finally … you have … Politics UN 1.0 P(“UN”|Politics)* P(Politics)/P(“UN”) Adopts 0.1 P(“Adopts”|Politics)*P(Politics)/P(“Adopts”) That was easy wasn’t it?!These don’t have to be only words. They can be ANY sort of feature (word pairs, syntax).
  18. 18. You have just Learnt How to Build A Naïve Bayesian ClassifierStarting from Linguistic Rules (Word Lists) Cohan Sujay Carlos Aiaioo Labs Bangalore, India cohan@aiaioo.com

×