A	  Praccal	  Overview	  of	  Linear	               Classifier
Discriminave/Generave	  Pair
Quesons•  Advantages/disadvantages	  of	  generave/   discriminave	  model?	  •  Can	  we	  combine	  generave	  and	  dis...
Outline•    Perceptron	  •    Margin	  •    Kernel	  •    Structure	  •    Online	  •    Ensemble
Generic	  Perceptron
Margin
Max	  Margin	  Helps
Percptron	  VS	  SVM
Theory•  Theorecal	  jusficaon:	     –  Large	  margin	  implies	  small	  VC	  dimension	  •  Stascal	  Learning	  Theory	...
Parameter	  Esmaon	  •  SMO	  algorithm	  •  Online	  algorithm
Classifier	  Comparison	  on	  Senment	  Classificaon                              Accuracy
Quesons•  What	  if	  not	  linear	  separable?	  •  Can	  SVM	  be	  used	  when	  some	  data	  do	  not	     have	  lab...
Kernel
Why	  Linear	  Classifier•  Simple	  •  Applicable	  •  Fast
Kernel:	  Nonlinear	  -­‐>	  Linear	  
Kernel	  SVM
Quesons•  What	  is	  the	  best	  kernel?	  •  Can	  we	  use	  mulple	  kernels?	  •  kernels	  for	  NLP?
Structure
Feature	  Funcons
Applicaon1:	  POS	  Tagging
Applicaon2:	  NP	  Chunking
Quesons•  Relaonship	  to	  CRF?	  •  Structured	  SVM?	  •  Maximum	  Margin	  Markov	  Network?
Ensemble	  Perceptron•  Many	  classifiers	  are	  beXer	  than	  one	  •  Vong	  perceptron	  •  Average	  perceptron
Online	  Learning•  Batch	  •  Online	  •  MiniBatch
ImplementaonsAlgorithm                                  Implemena/onPerceptron                                 Sklearn(Pyt...
Perceptron	  and	  Beyond            Kernel                                                      Online                   ...
Examples•  hXp://www.mathworks.com/matlabcentral/   fileexchange/28302-­‐svm-­‐demo	  •  hXp://www.eee.metu.edu.tr/~alatan/...
Linear classifier
Linear classifier
Linear classifier
Linear classifier
Linear classifier
Linear classifier
Linear classifier
Linear classifier
Linear classifier
Linear classifier
Linear classifier
Upcoming SlideShare
Loading in …5
×

Linear classifier

1,809 views

Published on

A practical overview of linear classifier, including perceptron, SVM, structured perceptron and CRF. Some materials are copied from other slides.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,809
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
35
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Linear classifier

  1. 1. A  Praccal  Overview  of  Linear   Classifier
  2. 2. Discriminave/Generave  Pair
  3. 3. Quesons•  Advantages/disadvantages  of  generave/ discriminave  model?  •  Can  we  combine  generave  and  discriminave   model?
  4. 4. Outline•  Perceptron  •  Margin  •  Kernel  •  Structure  •  Online  •  Ensemble
  5. 5. Generic  Perceptron
  6. 6. Margin
  7. 7. Max  Margin  Helps
  8. 8. Percptron  VS  SVM
  9. 9. Theory•  Theorecal  jusficaon:   –  Large  margin  implies  small  VC  dimension  •  Stascal  Learning  Theory  (SLT)   –  VC-­‐dimension  (Vapnik-­‐Chervonenkis)  •  Probably  Approximately  Correct  (PAC)  Theory  
  10. 10. Parameter  Esmaon  •  SMO  algorithm  •  Online  algorithm
  11. 11. Classifier  Comparison  on  Senment  Classificaon Accuracy
  12. 12. Quesons•  What  if  not  linear  separable?  •  Can  SVM  be  used  when  some  data  do  not   have  labels?
  13. 13. Kernel
  14. 14. Why  Linear  Classifier•  Simple  •  Applicable  •  Fast
  15. 15. Kernel:  Nonlinear  -­‐>  Linear  
  16. 16. Kernel  SVM
  17. 17. Quesons•  What  is  the  best  kernel?  •  Can  we  use  mulple  kernels?  •  kernels  for  NLP?
  18. 18. Structure
  19. 19. Feature  Funcons
  20. 20. Applicaon1:  POS  Tagging
  21. 21. Applicaon2:  NP  Chunking
  22. 22. Quesons•  Relaonship  to  CRF?  •  Structured  SVM?  •  Maximum  Margin  Markov  Network?
  23. 23. Ensemble  Perceptron•  Many  classifiers  are  beXer  than  one  •  Vong  perceptron  •  Average  perceptron
  24. 24. Online  Learning•  Batch  •  Online  •  MiniBatch
  25. 25. ImplementaonsAlgorithm Implemena/onPerceptron Sklearn(Python),  Weka(Java)SVM LibSVM,  LibLinear(Linear  kernel)Logisc  regression/Maximum  Entropy LibLinearCRF CRF++,  CRFSuiteNaïve  Bayes Sklean,  Weka
  26. 26. Perceptron  and  Beyond Kernel Online Kernel  SVM Online  SVM Kernel  perceptron Online  CRF PerceptronMargin Vong  perceptron Linear  SVM Ensemble Averaged  perceptron Margin  perceptron Structured  Perceptron Boosng Structured  SVM CRF M3N Structure
  27. 27. Examples•  hXp://www.mathworks.com/matlabcentral/ fileexchange/28302-­‐svm-­‐demo  •  hXp://www.eee.metu.edu.tr/~alatan/ Courses/Demo/AppletSVM.html

×