SlideShare a Scribd company logo
INTRODUCTION TO   Machine Learning ETHEM ALPAYDIN © The MIT Press, 2004 [email_address] http://www.cmpe.boun.edu.tr/~ethem/i2ml Lecture Slides for
CHAPTER 2:   Supervised Learning
Learning a Class from Examples ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Learn a clasifier x1 x2 x3 x4 y=f(x1,x2,x3,x4) Observations: Learn an approximation g (x1,x2,x3,x4) 0 1 0 1 0 7 0 0 0 1 1 6 0 0 1 1 0 5 1 1 0 0 1 4 1 1 1 0 0 3 0 0 0 1 0 2 0 0 1 0 0 1 y x4 x3 x2 x1 Example
Training set  X
Class  C
Hypothesis class  H Error of  h  on   H
 
S, G, and the Version Space most specific hypothesis,  S most general hypothesis,  G h     H , between  S  and  G  is consistent  and make up the  version space (Mitchell, 1997)
VC Dimension ,[object Object],[object Object],[object Object],[object Object],[object Object],An axis-aligned rectangle shatters 4 points only !
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Probably Approximately Correct (PAC) Learning
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Noise and Model Complexity
Multiple Classes ,  C i  i=1,...,K Train hypotheses  h i ( x ),  i  =1,..., K :
Regression
Model Selection & Generalization ,[object Object],[object Object],[object Object],[object Object],[object Object]
Triple Trade-Off ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Cross-Validation ,[object Object],[object Object],[object Object],[object Object],[object Object]
Dimensions of a Supervised Learner ,[object Object],[object Object],[object Object],[object Object]

More Related Content

What's hot

Alg II 2-7 Transformations
Alg II 2-7 TransformationsAlg II 2-7 Transformations
Alg II 2-7 Transformationsjtentinger
 
Lesson 28: The Fundamental Theorem of Calculus
Lesson 28: The Fundamental Theorem of CalculusLesson 28: The Fundamental Theorem of Calculus
Lesson 28: The Fundamental Theorem of CalculusMatthew Leingang
 
Introdution and designing a learning system
Introdution and designing a learning systemIntrodution and designing a learning system
Introdution and designing a learning system
swapnac12
 
(DL輪読)Matching Networks for One Shot Learning
(DL輪読)Matching Networks for One Shot Learning(DL輪読)Matching Networks for One Shot Learning
(DL輪読)Matching Networks for One Shot Learning
Masahiro Suzuki
 
Raw 2009 -THE ROLE OF LATEST FIXATIONS ON ONGOING VISUAL SEARCH A MODEL TO E...
Raw 2009 -THE ROLE OF LATEST FIXATIONS ON ONGOING VISUAL SEARCH  A MODEL TO E...Raw 2009 -THE ROLE OF LATEST FIXATIONS ON ONGOING VISUAL SEARCH  A MODEL TO E...
Raw 2009 -THE ROLE OF LATEST FIXATIONS ON ONGOING VISUAL SEARCH A MODEL TO E...
Giacomo Veneri
 
Curve sketching 4
Curve sketching 4Curve sketching 4
Curve sketching 4
Dhaval Jalalpara
 
Interest Rate Modeling With Cox Ingersoll Ross
Interest Rate Modeling With Cox Ingersoll RossInterest Rate Modeling With Cox Ingersoll Ross
Interest Rate Modeling With Cox Ingersoll Rossstretyakov
 
Visual Techniques
Visual TechniquesVisual Techniques
Visual Techniques
Md. Shohel Rana
 
Probability and random processes project based learning template.pdf
Probability and random processes project based learning template.pdfProbability and random processes project based learning template.pdf
Probability and random processes project based learning template.pdf
Vedant Srivastava
 
Unit 2.4
Unit 2.4Unit 2.4
Unit 2.4
Mark Ryder
 
9th polynomial test_paper-2
9th polynomial test_paper-29th polynomial test_paper-2
9th polynomial test_paper-2Anshay Gupta
 
Regression_1.pdf
Regression_1.pdfRegression_1.pdf
Regression_1.pdf
Amir Saleh
 
Dr Chris Drovandi (QUT) - Bayesian Indirect Inference Using a Parametric Auxi...
Dr Chris Drovandi (QUT) - Bayesian Indirect Inference Using a Parametric Auxi...Dr Chris Drovandi (QUT) - Bayesian Indirect Inference Using a Parametric Auxi...
Dr Chris Drovandi (QUT) - Bayesian Indirect Inference Using a Parametric Auxi...
QUT_SEF
 
Lecture: Monte Carlo Methods
Lecture: Monte Carlo MethodsLecture: Monte Carlo Methods
Lecture: Monte Carlo Methods
Frank Kienle
 
Mb0048 operations research
Mb0048   operations researchMb0048   operations research
Mb0048 operations researchsmumbahelp
 
Probability concepts and the normal distribution
Probability concepts and the normal distributionProbability concepts and the normal distribution
Probability concepts and the normal distributionHarve Abella
 
Application of interpolation in CSE
Application of interpolation in CSEApplication of interpolation in CSE
Application of interpolation in CSE
Md. Tanvir Hossain
 
A baseline for_few_shot_image_classification
A baseline for_few_shot_image_classificationA baseline for_few_shot_image_classification
A baseline for_few_shot_image_classification
DongHeeKim39
 

What's hot (20)

Alg II 2-7 Transformations
Alg II 2-7 TransformationsAlg II 2-7 Transformations
Alg II 2-7 Transformations
 
Lesson 28: The Fundamental Theorem of Calculus
Lesson 28: The Fundamental Theorem of CalculusLesson 28: The Fundamental Theorem of Calculus
Lesson 28: The Fundamental Theorem of Calculus
 
Introdution and designing a learning system
Introdution and designing a learning systemIntrodution and designing a learning system
Introdution and designing a learning system
 
ppt
pptppt
ppt
 
(DL輪読)Matching Networks for One Shot Learning
(DL輪読)Matching Networks for One Shot Learning(DL輪読)Matching Networks for One Shot Learning
(DL輪読)Matching Networks for One Shot Learning
 
Raw 2009 -THE ROLE OF LATEST FIXATIONS ON ONGOING VISUAL SEARCH A MODEL TO E...
Raw 2009 -THE ROLE OF LATEST FIXATIONS ON ONGOING VISUAL SEARCH  A MODEL TO E...Raw 2009 -THE ROLE OF LATEST FIXATIONS ON ONGOING VISUAL SEARCH  A MODEL TO E...
Raw 2009 -THE ROLE OF LATEST FIXATIONS ON ONGOING VISUAL SEARCH A MODEL TO E...
 
Curve sketching 4
Curve sketching 4Curve sketching 4
Curve sketching 4
 
Interest Rate Modeling With Cox Ingersoll Ross
Interest Rate Modeling With Cox Ingersoll RossInterest Rate Modeling With Cox Ingersoll Ross
Interest Rate Modeling With Cox Ingersoll Ross
 
Visual Techniques
Visual TechniquesVisual Techniques
Visual Techniques
 
Probability and random processes project based learning template.pdf
Probability and random processes project based learning template.pdfProbability and random processes project based learning template.pdf
Probability and random processes project based learning template.pdf
 
Unit 2.4
Unit 2.4Unit 2.4
Unit 2.4
 
9th polynomial test_paper-2
9th polynomial test_paper-29th polynomial test_paper-2
9th polynomial test_paper-2
 
Regression_1.pdf
Regression_1.pdfRegression_1.pdf
Regression_1.pdf
 
Dr Chris Drovandi (QUT) - Bayesian Indirect Inference Using a Parametric Auxi...
Dr Chris Drovandi (QUT) - Bayesian Indirect Inference Using a Parametric Auxi...Dr Chris Drovandi (QUT) - Bayesian Indirect Inference Using a Parametric Auxi...
Dr Chris Drovandi (QUT) - Bayesian Indirect Inference Using a Parametric Auxi...
 
Lecture: Monte Carlo Methods
Lecture: Monte Carlo MethodsLecture: Monte Carlo Methods
Lecture: Monte Carlo Methods
 
Mb0048 operations research
Mb0048   operations researchMb0048   operations research
Mb0048 operations research
 
Probability concepts and the normal distribution
Probability concepts and the normal distributionProbability concepts and the normal distribution
Probability concepts and the normal distribution
 
Alg1 lesson 8-1
Alg1 lesson 8-1Alg1 lesson 8-1
Alg1 lesson 8-1
 
Application of interpolation in CSE
Application of interpolation in CSEApplication of interpolation in CSE
Application of interpolation in CSE
 
A baseline for_few_shot_image_classification
A baseline for_few_shot_image_classificationA baseline for_few_shot_image_classification
A baseline for_few_shot_image_classification
 

Viewers also liked

Machine Learning
Machine LearningMachine Learning
Machine Learningbutest
 
10-228
10-22810-228
10-228butest
 
Hawaii - JONATHAN WESTON CREATIVE MULTIMEDIA VIDEO PRODUCER PMP
Hawaii - JONATHAN WESTON CREATIVE MULTIMEDIA VIDEO PRODUCER PMPHawaii - JONATHAN WESTON CREATIVE MULTIMEDIA VIDEO PRODUCER PMP
Hawaii - JONATHAN WESTON CREATIVE MULTIMEDIA VIDEO PRODUCER PMPbutest
 
WALD LECTURE 1
WALD LECTURE 1WALD LECTURE 1
WALD LECTURE 1butest
 
Word version - Home - Lethbridge College
Word version - Home - Lethbridge CollegeWord version - Home - Lethbridge College
Word version - Home - Lethbridge Collegebutest
 
Classification of CT..
Classification of CT..Classification of CT..
Classification of CT..butest
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALbutest
 
1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同butest
 
EL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBEEL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBEbutest
 

Viewers also liked (10)

Machine Learning
Machine LearningMachine Learning
Machine Learning
 
10-228
10-22810-228
10-228
 
Hawaii - JONATHAN WESTON CREATIVE MULTIMEDIA VIDEO PRODUCER PMP
Hawaii - JONATHAN WESTON CREATIVE MULTIMEDIA VIDEO PRODUCER PMPHawaii - JONATHAN WESTON CREATIVE MULTIMEDIA VIDEO PRODUCER PMP
Hawaii - JONATHAN WESTON CREATIVE MULTIMEDIA VIDEO PRODUCER PMP
 
[PDF]
[PDF][PDF]
[PDF]
 
WALD LECTURE 1
WALD LECTURE 1WALD LECTURE 1
WALD LECTURE 1
 
Word version - Home - Lethbridge College
Word version - Home - Lethbridge CollegeWord version - Home - Lethbridge College
Word version - Home - Lethbridge College
 
Classification of CT..
Classification of CT..Classification of CT..
Classification of CT..
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIAL
 
1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同
 
EL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBEEL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBE
 

Similar to Alpaydin - Chapter 2

i2ml3e-chap2.pptx
i2ml3e-chap2.pptxi2ml3e-chap2.pptx
i2ml3e-chap2.pptx
waseem214905
 
Lecture5 xing
Lecture5 xingLecture5 xing
Lecture5 xing
Tianlu Wang
 
Supervised_Learning.ppt
Supervised_Learning.pptSupervised_Learning.ppt
Supervised_Learning.ppt
Hari629251
 
Computational Learning Theory
Computational Learning TheoryComputational Learning Theory
Computational Learning Theorybutest
 
lecture_mooney.ppt
lecture_mooney.pptlecture_mooney.ppt
lecture_mooney.pptbutest
 
Introduction to Machine Learning
Introduction to Machine Learning Introduction to Machine Learning
Introduction to Machine Learning
FogGuru MSCA Project
 
Introduction to machine learning
Introduction to machine learningIntroduction to machine learning
Introduction to machine learningbutest
 
AML_030607.ppt
AML_030607.pptAML_030607.ppt
AML_030607.pptbutest
 
Cs229 notes4
Cs229 notes4Cs229 notes4
Cs229 notes4
VuTran231
 
Machine Learning
Machine LearningMachine Learning
Machine Learningbutest
 
Decision tree learning
Decision tree learningDecision tree learning
Decision tree learning
Dr. Radhey Shyam
 
Lecture 3 (Supervised learning)
Lecture 3 (Supervised learning)Lecture 3 (Supervised learning)
Lecture 3 (Supervised learning)
VARUN KUMAR
 
ML unit-1.pptx
ML unit-1.pptxML unit-1.pptx
ML unit-1.pptx
SwarnaKumariChinni
 
Imprecision in learning: an overview
Imprecision in learning: an overviewImprecision in learning: an overview
Imprecision in learning: an overview
Sebastien Destercke
 
01_CS115 Machine Learning Overview .pdf
01_CS115 Machine Learning Overview  .pdf01_CS115 Machine Learning Overview  .pdf
01_CS115 Machine Learning Overview .pdf
NguynTrngNhn42
 
Statistical Machine________ Learning.ppt
Statistical Machine________ Learning.pptStatistical Machine________ Learning.ppt
Statistical Machine________ Learning.ppt
SandeepGupta229023
 
Machine Learning for NLP
Machine Learning for NLPMachine Learning for NLP
Machine Learning for NLPbutest
 
Text classification
Text classificationText classification
Text classification
Fraboni Ec
 
Text classification
Text classificationText classification
Text classification
David Hoen
 

Similar to Alpaydin - Chapter 2 (20)

i2ml3e-chap2.pptx
i2ml3e-chap2.pptxi2ml3e-chap2.pptx
i2ml3e-chap2.pptx
 
Lecture5 xing
Lecture5 xingLecture5 xing
Lecture5 xing
 
Supervised_Learning.ppt
Supervised_Learning.pptSupervised_Learning.ppt
Supervised_Learning.ppt
 
Computational Learning Theory
Computational Learning TheoryComputational Learning Theory
Computational Learning Theory
 
lecture_mooney.ppt
lecture_mooney.pptlecture_mooney.ppt
lecture_mooney.ppt
 
Introduction to Machine Learning
Introduction to Machine Learning Introduction to Machine Learning
Introduction to Machine Learning
 
Introduction to machine learning
Introduction to machine learningIntroduction to machine learning
Introduction to machine learning
 
.ppt
.ppt.ppt
.ppt
 
AML_030607.ppt
AML_030607.pptAML_030607.ppt
AML_030607.ppt
 
Cs229 notes4
Cs229 notes4Cs229 notes4
Cs229 notes4
 
Machine Learning
Machine LearningMachine Learning
Machine Learning
 
Decision tree learning
Decision tree learningDecision tree learning
Decision tree learning
 
Lecture 3 (Supervised learning)
Lecture 3 (Supervised learning)Lecture 3 (Supervised learning)
Lecture 3 (Supervised learning)
 
ML unit-1.pptx
ML unit-1.pptxML unit-1.pptx
ML unit-1.pptx
 
Imprecision in learning: an overview
Imprecision in learning: an overviewImprecision in learning: an overview
Imprecision in learning: an overview
 
01_CS115 Machine Learning Overview .pdf
01_CS115 Machine Learning Overview  .pdf01_CS115 Machine Learning Overview  .pdf
01_CS115 Machine Learning Overview .pdf
 
Statistical Machine________ Learning.ppt
Statistical Machine________ Learning.pptStatistical Machine________ Learning.ppt
Statistical Machine________ Learning.ppt
 
Machine Learning for NLP
Machine Learning for NLPMachine Learning for NLP
Machine Learning for NLP
 
Text classification
Text classificationText classification
Text classification
 
Text classification
Text classificationText classification
Text classification
 

More from butest

Timeline: The Life of Michael Jackson
Timeline: The Life of Michael JacksonTimeline: The Life of Michael Jackson
Timeline: The Life of Michael Jacksonbutest
 
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...butest
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALbutest
 
Com 380, Summer II
Com 380, Summer IICom 380, Summer II
Com 380, Summer IIbutest
 
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet JazzThe MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazzbutest
 
MICHAEL JACKSON.doc
MICHAEL JACKSON.docMICHAEL JACKSON.doc
MICHAEL JACKSON.docbutest
 
Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1butest
 
Facebook
Facebook Facebook
Facebook butest
 
Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...butest
 
Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...butest
 
NEWS ANNOUNCEMENT
NEWS ANNOUNCEMENTNEWS ANNOUNCEMENT
NEWS ANNOUNCEMENTbutest
 
C-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.docC-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.docbutest
 
MAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.docMAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.docbutest
 
Mac OS X Guide.doc
Mac OS X Guide.docMac OS X Guide.doc
Mac OS X Guide.docbutest
 
WEB DESIGN!
WEB DESIGN!WEB DESIGN!
WEB DESIGN!butest
 
Download
DownloadDownload
Downloadbutest
 
resume.doc
resume.docresume.doc
resume.docbutest
 
Download.doc.doc
Download.doc.docDownload.doc.doc
Download.doc.docbutest
 

More from butest (20)

Timeline: The Life of Michael Jackson
Timeline: The Life of Michael JacksonTimeline: The Life of Michael Jackson
Timeline: The Life of Michael Jackson
 
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIAL
 
Com 380, Summer II
Com 380, Summer IICom 380, Summer II
Com 380, Summer II
 
PPT
PPTPPT
PPT
 
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet JazzThe MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
 
MICHAEL JACKSON.doc
MICHAEL JACKSON.docMICHAEL JACKSON.doc
MICHAEL JACKSON.doc
 
Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1
 
Facebook
Facebook Facebook
Facebook
 
Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...
 
Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...
 
NEWS ANNOUNCEMENT
NEWS ANNOUNCEMENTNEWS ANNOUNCEMENT
NEWS ANNOUNCEMENT
 
C-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.docC-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.doc
 
MAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.docMAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.doc
 
Mac OS X Guide.doc
Mac OS X Guide.docMac OS X Guide.doc
Mac OS X Guide.doc
 
hier
hierhier
hier
 
WEB DESIGN!
WEB DESIGN!WEB DESIGN!
WEB DESIGN!
 
Download
DownloadDownload
Download
 
resume.doc
resume.docresume.doc
resume.doc
 
Download.doc.doc
Download.doc.docDownload.doc.doc
Download.doc.doc
 

Alpaydin - Chapter 2

  • 1. INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, 2004 [email_address] http://www.cmpe.boun.edu.tr/~ethem/i2ml Lecture Slides for
  • 2. CHAPTER 2: Supervised Learning
  • 3.
  • 4. Learn a clasifier x1 x2 x3 x4 y=f(x1,x2,x3,x4) Observations: Learn an approximation g (x1,x2,x3,x4) 0 1 0 1 0 7 0 0 0 1 1 6 0 0 1 1 0 5 1 1 0 0 1 4 1 1 1 0 0 3 0 0 0 1 0 2 0 0 1 0 0 1 y x4 x3 x2 x1 Example
  • 7. Hypothesis class H Error of h on H
  • 8.  
  • 9. S, G, and the Version Space most specific hypothesis, S most general hypothesis, G h  H , between S and G is consistent and make up the version space (Mitchell, 1997)
  • 10.
  • 11.
  • 12.
  • 13. Multiple Classes , C i i=1,...,K Train hypotheses h i ( x ), i =1,..., K :
  • 15.
  • 16.
  • 17.
  • 18.