Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Machine learning Introduction

2,238 views

Published on

Published in: Technology, Education
  • Be the first to comment

Machine learning Introduction

  1. 1. Machine  Learning  Introduc1on   guodong@hulu.com     Machine  learning  introduc0on   Logis1c  regression   Feature  selec1on   Boos1ng,  tree  boos1ng     See  more  ML  posts:  h>p://dongguo.me/    
  2. 2. Machine  Learning  Makes  Life  Be>er  
  3. 3. WHAT  IS  MACHINE  LEARNING?  
  4. 4. Learning   •  What  is  learning   –  Find  rules  from  data/experience   •  Why  learning  is  possible   –  Assume  rules  exist  in  this  world   •  How  to  learn   –  Induc1ve  
  5. 5. What  is  machine  learning   •  “Machine  Learning  is  a  field  of  study  that  gives   computers  the  ability  to  learn  without  being   explicitly  programmed”  -­‐  Arthur  Samuel  (1959)   •  Machine  learning  is  the  study  of  computer   algorithms  that  improve  automa1cally  through   experience”  –  Tom  Mitchell  (1998)  
  6. 6. Overview  of  machine  learning       Machine  Learning   Unsupervised   Learning   Supervised   Learning   Classifica1on   Semi-­‐supervised   Learning   Regression  
  7. 7. Outline   •  Supervised  Learning   •  Case  Study   •  Challenge   •  Resource  
  8. 8. Supervised  learning   •  Concepts   •  Defini1on   •  Models   •  Metrics   •  Open  Ques1ons  
  9. 9. Concepts   Problem       Generate  dataset   Dataset   Train   Sample/instance   Feature  vector   label   model   Predict   Test   Model  Tuning   Feature  selec0on  
  10. 10. What  is  Supervised  learning   •  Find  a  func1on  (from  some  func1on  space)  to   predict  for  unseen  instances,  from  the  labeled   training  data   –  Func1on  space:  determined  by  the  chosen  model   –  Find  the  func1on:  minimize  error  on  training  data  with   some  cost  func1on   •  2  types:  Classifica1on  and  regression  
  11. 11. Formal  defini1on   •  Given  a  training  dataset   r N {xi , yi }i =1 •  And  define  a  loss  func1on   ∧ ∧ L( y, y ), where y = f ( x) •  Target   ∧ f ( x) =arg min G ( f ), f 1 st. G ( f ) = N N ∑ L( y , f ( x )) i =1 i i
  12. 12. Models  for  supervised  learning   •  Classifica1on  and  regression   –  For  classifica1on:  LR(Logis1c  regression),  Naïve  Bayes   –  For  regression:  linear  regression   –  For  Both:  Trees,  KNN,  SVM,  ANN   •  Genera1ve  and  Discrimina1ve   –  Genera1ve:  Naïve  Bayes,  GMM,  HMM   –  Discrimina1ve:  KNN,  LR,  SVM,  ANN,  Trees   •  Parametric  and  nonparametric   –  Parametric:  LR,  Naïve  Bayes,  ANN   –  nonparametric:  Trees,  KNN,  kernel  methods  
  13. 13. Decision  Tree   •  Would  you  like  to  date  somebody?   Gender   male   female   Good   looking?   Yes!   Pass   No!   umm..   Pass   Others…   Accept   Very  good   Accept   else   Pass  
  14. 14. K-­‐Nearest  Neighbor  classifier   K=15   K=1  
  15. 15. Naïve  Bayes   •  Bayes  classifier   •  Condi1onal  Independence  assump1on   •  With  this  assump1on    
  16. 16. Logis1c  regression   •  Logis1c  func1on      
  17. 17. Ar1ficial  neural  network  
  18. 18. Support  vector  machine  
  19. 19. Model  Inference   •  Typical  inference  methods   –  Gradient  descent   –  Expecta1on  Maximiza1on   –  Sampling  based  
  20. 20. Model  ensemble   •  Averaging  or  vo1ng  output  of  mul1ply  classifiers   •  Bagging  (bootstrap  aggrega1ng)   –  Train  mul1ple  base  models   –  Vote  mul1ply  base  classifiers  with  same  weight   –  Improve  model  stability  and  avoid  overfihng   –  Work  well  on  unstable  base  classifier   •  Adaboost  (adap1ve  boos1ng)   –  Sequen1al  base  classifiers   –  Misclassified  instances  have  higher  weight  in  next  base   classifier   –  Weighted  vo1ng  
  21. 21. Evalua1on  metrics   •  Common  Metrics  for  classifica1on   –  Accuracy   –  Precision-­‐Recall   –  AUC   •  For  regression   –  Mean  absolute  error  (MAE)   –  Mean  square  error  (MSE),  RMSE  
  22. 22. Ques1on1:  How  to  choose  a  suitable  model?   Characteris0c   Naïve   Bayes   Trees   K  Nearest   neighbor   Logis0c   regression   Neural   SVM   Networks   Natural  handling   data  of  “mixed”   type   Robustness  to   outliers  in  input   space   Computa1onal   scalability   Interpretability   1   3   1   1   1   1    3   3   3   3     1   1   3   3   1   3   1   1    2   2     1    2   1   1   Predic1ve  power   1   1    3   2   3   3   <Elements  of  Sta-s-cal  Learning>  II  P351      
  23. 23. Ques1on2:  Can  we  find  a  100%  accurate  model?       •  Expected  risk   •  Empirical  risk   •  Choose  a  family          for  candidate  predic1on  func1ons     •  Error  
  24. 24. Case  study:  Predic1ve  Demographic       Feature  extrac1on  (‘show’,  ‘ad  vote’,  ‘ad   selec1on’)   feature  analysis  (remove  ‘ad  selec1on’)   Load  login  profile   ML  problem?  What  kind?    Labels?   Evalua1on  metric?   Possible  features?  (show,  ad  vote,   ad  selec1on,  search…)    Accessible?       Problem   Dataset  genera1on   Choose  a  Model   1.  Familiar?  (NB,  ANN,  LR,  Tree,  SVM)   2.  Computa1onal  cost?  Interpretability?   Precision?     3.  Data:  amount?  noise  ra1o?     Train   Try  more  features(add   ‘OS’,  ‘browser’,  ‘flash’)   Feature  selec1on  (remove   ‘flash’,  and  non   anonymous  features)   Predictor     Try  more  models   Tuning   Evalua1on  (AUC,   Precision-­‐recall)   Test   Challenges   (Noise,  different  Join  distribu1on,  evalua1on)       model  ensemble   Predictor  on  product   Scoring   Online  Update  
  25. 25. Challenges  in  Machine  learning   •  Data   –  Sparse  data  in  high  dimensions   –  Limited  labels     •  Computa1on  Cost   –  Speed  Up  advanced  models   –  Paralleliza1on   •  Applica1on   –  Structured  predic1on  
  26. 26. Resource   •  •  •  •  Conference   Books   Lectures   Dataset  
  27. 27. Top  conference   •  •  •  •  •  ICML   NIPS   IJCAI/AAAI   KDD   Other  related   –  WSDM,  WWW,  SIGIR,  CIKM,  ICDE,  ICDM  
  28. 28. Books   •  •  •  •  Machine  Learning  [link]      by  Mitchell   Pa-ern  Recogni0on  and  Machine  Learning  [link]  by  Bishop   The  Elements  of  Sta0s0cal  Learning  [link]   Scaling  Up  Machine  Learning  [link]  
  29. 29. Lectures   •  Machine  Learning  open  class  –  by  Andrew  Ng   –  Video  in  YouTube   •  Advanced  topics  in  Machine  Learning  –  Cornell   •  h>p://videolectures.net/  
  30. 30. Other  research  resource   •  Research  Organs   –  Yahoo  Research  [link]   –  Google  Research  publica1ons  [link]   •  Dataset   –  UCI  machine  learning  Repository  [link]   –  kaggle.com  
  31. 31. THANKS  

×