SlideShare a Scribd company logo
1 of 22
Download to read offline
Introduction to Machine
       Learning
                      Lecture 9
 Bayesian decision theory – An introduction

                   Albert Orriols i Puig
                  aorriols@salle.url.edu
                      i l @ ll       ld

         Artificial Intelligence – Machine Learning
             Enginyeria i Arquitectura La Salle
                 gy           q
                    Universitat Ramon Llull
Recap of Lecture 5-8

                          LET’S START WITH DATA
                             CLASSIFICATION




                                                               Slide 2
Artificial Intelligence                     Machine Learning
Recap of Lectures 5-8
        We want to build decision trees
        How can I automatically
        generate these types
        of trees?
                Decide which attribute we
                should put in each node
                Decide a split point




                Rely on information theory
                We also saw many other improvements



                                                          Slide 3
Artificial Intelligence                Machine Learning
Recap of Lecture 5-8
        From kNN to CBR
                             15-NN                      1-NN




        Key aspects
                Value of k
                Distance functions


                                                               Slide 4
Artificial Intelligence              Machine Learning
Today’s Agenda


        Could we use probability to classify?
                     p         y           y
        Where all began
        Some anecdotes on the correct use of
        probabilities
           b biliti




                                                    Slide 5
Artificial Intelligence       Introduction to C++
Why Bother about Prob.?

        The world is a very uncertain place
        Almost 40 years of AI and ML dealing with uncertain
        domains
        Some researchers decided to employ ideas from
        probability to model concepts


        Before saying more let’s go to the beginning
                      more… let s




                                                              Slide 6
Artificial Intelligence       Machine Learning
Meeting the Reverend Thomas Bayes


    Two main works:
         Divine Benevolence or an Attempt to
                Benevolence,
         Prove That the Principal End of the Divine
         Providence and Government is the
         Happiness of Hi C t
         H    i      f His Creatures (1731)
         An Introduction to the Doctrine of Fluxions,
         and a Defence of the Mathematicians
         Against the Objections of the Author of the
         Analyst (published anonymously in 1736)


    But we are especially interested in:
         Essay Towards Solving a Problem in the Doctrine of Chances (1764)
         which was actually published p
                          yp          posthumously by Richard Price
                                                 yy


                                                                             Slide 7
Artificial Intelligence                Machine Learning
Where These Ideas Came From?
        Bayes build his theory upon several ideas
          y                  yp
        Immanuel Kant (1724-1804)
                Copernican revolution: our understanding
                of the external world had its foundations
                not merely in experience, but in both experience
                and a priori concepts, th offering a
                   d     ii          t thus ff i
                non-empiricist critique of rationalist philosophy


        Isaac Newton (1643-1727)
                Universal gravitation
                three laws of motion which dominated
                the scientific view of the physical universe
                for the next three centuries



                                                                    Slide 8
Artificial Intelligence                   Machine Learning
What Was Bayes’ Point
        Bayesian p
          y      probability
                           y
                Notion of probability interpreted as partial belief rather than as
                frequency
        Bayesian estimation
                Calculate the validity of a proposition
                On the basis of a prior estimate of its probability and new
                relevant evidence
                E.g.:
                          Before Bayes, forward probability
                          Bf     B      f     d    b bilit
                              given a specified number of white and black balls in an urn, what
                              is the probability of drawing a black ball?
                                     p         y          g
                          Bayes turned its attention to the converse problem
                              given that one or more balls have been drawn, what can be said
                              about the number of white and black balls in the urn?

                                                                                          Slide 9
Artificial Intelligence                        Machine Learning
Bayes’ Theorem
        Outputs the most probable hypothesis h∈H, given the data D +
        knowledge about prior probabilities of hypotheses in H
        Terminology:
                P(h|D): probability that h holds given data D. Posterior probability of h;
                confidence that h holds given D.
                P(h): prior probability of h (background knowledge we have about that h is a
                correct hypothesis)
                P(D): prior probability that training data D will be observed
                P(D|h): probability of observing D given h holds



                                           P (D | h )P (h )
                              P (h | D ) =
                                               P (D )



                                                                                             Slide 10
Artificial Intelligence                       Machine Learning
Bayes’ Theorem

           Given H the space of possible hypothesis
           The
           Th most probable h
                      b bl hypothesis i the one that maximizes P(h|D)
                                h i is h         h      ii     P(h|D):




                                          P (D | h )P (h )
hMAP ≡ arg max P (h | D ) = arg max                        = arg max P (D | h )P (h )
                                              P (D )
                   h∈H




                                                                             Slide 11
  Artificial Intelligence           Machine Learning
Is the Pope the Pope?
        The chances that a randomly chosen human being is the Pope
                                  y                  g          p
        are about 1 in 6 billion
        Benedict XVI is the Pope
                              p
        What are the chances that Benedict XVI is human?
        (Beck-Bornholdt
        (Beck Bornholdt and Dubben, 1996)
                            Dubben




                Analogy to syllogistic reasoning: 1 in 6 billion
                                                                   Slide 12
Artificial Intelligence                 Machine Learning
So, Is the Pope an Alien?
         Where is the trick?
                 Probability of the data given a
                 hypothesis H: P(D|H)
                  ypo es s        (|)
                 Probability of the hypothesis
                 ge
                 given the da a P(H|D)
                         e data: ( | )
                 P(D|H) is different from P(H|D)
         So, i th P
         S is the Pope An alien?
                       A li ?
                 Probability of being an alien P(A)
                 Probability of being human P(H)
                 Probability that the pope is an alien
                                 P( Pope | Alien) P( Alien)
P( Alien | Pope) =
             p
                                       Human) + P( P
                   P( P
                      Pope | H
                             Human) P( H            Pope | Ali ) P( Ali )
                                                            Alien Alien
                                                                  Slide 13
 Artificial Intelligence                 Machine Learning
So, Is the Pope an Alien?
        What’s missing?
                     g
                P(Pope|Alien)
                P(Human)
                P(H    )
                P(Alien)


        Considering
                Low values of P(Alien) and P(Pope|Alien)
                And large values of P(Human)
                                  f(       )


                We could “probably” say that the pope is not an alien!



                                                                         Slide 14
Artificial Intelligence                Machine Learning
More examples: Monty Hall
        Stick or switch




                                             Slide 15
Artificial Intelligence   Machine Learning
Stick or Switch
        I chose door number 3
                Door 2 is uncovered
                a d contains sheep
                and co a s a s eep




        They give me the chance to change the door
                Should I?
                Use probability, not faith,
                to give an answer!




                                                           Slide 16
Artificial Intelligence                 Machine Learning
Stick or Switch




                           I should switch!
                                                  Slide 17
Artificial Intelligence        Machine Learning
Yet Another Example: The Defendant’s Fallacy


        The history of a murder
                A suspect was caught
                                  h
                DNA test was positive
                DNA test fails only 1 over 1 million times


        So, my suspect must be guilty, right?
                More specifically, it will be guilty with p = 0.999999. Agree?




                                                                             Slide 18
Artificial Intelligence                 Machine Learning
The Defendant’s Fallacy
        Where is the trick now?
                P(coincides | innocent) as opposed to P(innocent|coincides)
                          P(coincides | innocent) commonly misused as the probability
                          of being innocent
                          P(innocent | coincides) is the probability of being guilty
                           (                     )       p         y        gg y
                          having that the test was positive!



        Does this really matter?
                Let’s
                L t’ assume a city of 10 million i h bit t
                               it f       illi inhabitants
                We apply the test to all the 10 million inhabitants
                How many of them will be positive?
                          10


                                                                                       Slide 19
Artificial Intelligence                       Machine Learning
The Defendant’s Fallacy
        Two arguments
              g
                The prosecutor: There is 0.000001 that the suspect is innocent
                The d f d t In thi it f
                Th defendant: I this city of 10M people, the probability of th
                                                      l th      b bilit f the
                suspect being innocent is approximately 90%

        Who is right?
                The d f d t
                Th defendant
                Prove for that? You do the math




                                                                          Slide 20
Artificial Intelligence               Machine Learning
Next Class



        How we can use these concepts in machine learning




                                                            Slide 21
Artificial Intelligence     Introduction to C++
Introduction to Machine
       Learning
                      Lecture 9
 Bayesian decision theory – An introduction

                   Albert Orriols i Puig
                  aorriols@salle.url.edu
                      i l @ ll       ld

         Artificial Intelligence – Machine Learning
             Enginyeria i Arquitectura La Salle
                 gy           q
                    Universitat Ramon Llull

More Related Content

What's hot

Machine Learning-Linear regression
Machine Learning-Linear regressionMachine Learning-Linear regression
Machine Learning-Linear regressionkishanthkumaar
 
Machine Learning with Decision trees
Machine Learning with Decision treesMachine Learning with Decision trees
Machine Learning with Decision treesKnoldus Inc.
 
Decision trees in Machine Learning
Decision trees in Machine Learning Decision trees in Machine Learning
Decision trees in Machine Learning Mohammad Junaid Khan
 
2.3 bayesian classification
2.3 bayesian classification2.3 bayesian classification
2.3 bayesian classificationKrish_ver2
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reductionmrizwan969
 
AI 7 | Constraint Satisfaction Problem
AI 7 | Constraint Satisfaction ProblemAI 7 | Constraint Satisfaction Problem
AI 7 | Constraint Satisfaction ProblemMohammad Imam Hossain
 
Belief Networks & Bayesian Classification
Belief Networks & Bayesian ClassificationBelief Networks & Bayesian Classification
Belief Networks & Bayesian ClassificationAdnan Masood
 
Training Neural Networks
Training Neural NetworksTraining Neural Networks
Training Neural NetworksDatabricks
 
Classification and Regression
Classification and RegressionClassification and Regression
Classification and RegressionMegha Sharma
 
Classification Based Machine Learning Algorithms
Classification Based Machine Learning AlgorithmsClassification Based Machine Learning Algorithms
Classification Based Machine Learning AlgorithmsMd. Main Uddin Rony
 
ML - Simple Linear Regression
ML - Simple Linear RegressionML - Simple Linear Regression
ML - Simple Linear RegressionAndrew Ferlitsch
 
Feature selection
Feature selectionFeature selection
Feature selectionDong Guo
 
NAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIERNAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIERKnoldus Inc.
 
Bias and variance trade off
Bias and variance trade offBias and variance trade off
Bias and variance trade offVARUN KUMAR
 
Parametric & Non-Parametric Machine Learning (Supervised ML)
Parametric & Non-Parametric Machine Learning (Supervised ML)Parametric & Non-Parametric Machine Learning (Supervised ML)
Parametric & Non-Parametric Machine Learning (Supervised ML)Rehan Guha
 

What's hot (20)

Machine Learning-Linear regression
Machine Learning-Linear regressionMachine Learning-Linear regression
Machine Learning-Linear regression
 
Machine Learning with Decision trees
Machine Learning with Decision treesMachine Learning with Decision trees
Machine Learning with Decision trees
 
Decision trees in Machine Learning
Decision trees in Machine Learning Decision trees in Machine Learning
Decision trees in Machine Learning
 
Naive bayes
Naive bayesNaive bayes
Naive bayes
 
2.3 bayesian classification
2.3 bayesian classification2.3 bayesian classification
2.3 bayesian classification
 
Dimensionality Reduction
Dimensionality ReductionDimensionality Reduction
Dimensionality Reduction
 
Support Vector Machines ( SVM )
Support Vector Machines ( SVM ) Support Vector Machines ( SVM )
Support Vector Machines ( SVM )
 
AI 7 | Constraint Satisfaction Problem
AI 7 | Constraint Satisfaction ProblemAI 7 | Constraint Satisfaction Problem
AI 7 | Constraint Satisfaction Problem
 
Belief Networks & Bayesian Classification
Belief Networks & Bayesian ClassificationBelief Networks & Bayesian Classification
Belief Networks & Bayesian Classification
 
Bayes Belief Networks
Bayes Belief NetworksBayes Belief Networks
Bayes Belief Networks
 
Training Neural Networks
Training Neural NetworksTraining Neural Networks
Training Neural Networks
 
Classification and Regression
Classification and RegressionClassification and Regression
Classification and Regression
 
Classification Based Machine Learning Algorithms
Classification Based Machine Learning AlgorithmsClassification Based Machine Learning Algorithms
Classification Based Machine Learning Algorithms
 
ML - Simple Linear Regression
ML - Simple Linear RegressionML - Simple Linear Regression
ML - Simple Linear Regression
 
Bayesian networks
Bayesian networksBayesian networks
Bayesian networks
 
Ridge regression
Ridge regressionRidge regression
Ridge regression
 
Feature selection
Feature selectionFeature selection
Feature selection
 
NAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIERNAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIER
 
Bias and variance trade off
Bias and variance trade offBias and variance trade off
Bias and variance trade off
 
Parametric & Non-Parametric Machine Learning (Supervised ML)
Parametric & Non-Parametric Machine Learning (Supervised ML)Parametric & Non-Parametric Machine Learning (Supervised ML)
Parametric & Non-Parametric Machine Learning (Supervised ML)
 

Similar to Lecture9 - Bayesian-Decision-Theory

Lecture1 AI1 Introduction to artificial intelligence
Lecture1 AI1 Introduction to artificial intelligenceLecture1 AI1 Introduction to artificial intelligence
Lecture1 AI1 Introduction to artificial intelligenceAlbert Orriols-Puig
 
4-ML-UNIT-IV-Bayesian Learning.pptx
4-ML-UNIT-IV-Bayesian Learning.pptx4-ML-UNIT-IV-Bayesian Learning.pptx
4-ML-UNIT-IV-Bayesian Learning.pptxSaitama84
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learningkkkc
 
Supermathematics and Artificial General Intelligence
Supermathematics and Artificial General IntelligenceSupermathematics and Artificial General Intelligence
Supermathematics and Artificial General IntelligenceJordan Bennett
 
Artificial intelligence
Artificial intelligenceArtificial intelligence
Artificial intelligenceUmesh Meher
 
Artificial Intelligence AI Topics History and Overview
Artificial Intelligence AI Topics History and OverviewArtificial Intelligence AI Topics History and Overview
Artificial Intelligence AI Topics History and Overviewbutest
 
Artificial Intelligence AI Topics History and Overview
Artificial Intelligence AI Topics History and OverviewArtificial Intelligence AI Topics History and Overview
Artificial Intelligence AI Topics History and Overviewbutest
 
Artificial Intelligence
Artificial IntelligenceArtificial Intelligence
Artificial IntelligenceIman Ardekani
 
Class 27: Pythonic Objects
Class 27: Pythonic ObjectsClass 27: Pythonic Objects
Class 27: Pythonic ObjectsDavid Evans
 
Striving to Demystify Bayesian Computational Modelling
Striving to Demystify Bayesian Computational ModellingStriving to Demystify Bayesian Computational Modelling
Striving to Demystify Bayesian Computational ModellingMarco Wirthlin
 
(Reverse) Engineering Intelligence - Noah Goodman - H+ Summit @ Harvard
(Reverse) Engineering Intelligence - Noah Goodman - H+ Summit @ Harvard(Reverse) Engineering Intelligence - Noah Goodman - H+ Summit @ Harvard
(Reverse) Engineering Intelligence - Noah Goodman - H+ Summit @ HarvardHumanity Plus
 
Physics of Intelligence
Physics of IntelligencePhysics of Intelligence
Physics of IntelligenceRobert Fry
 
Naturalized Epistemology North American Computing and Philosophy 2007
Naturalized Epistemology North American Computing and Philosophy 2007 Naturalized Epistemology North American Computing and Philosophy 2007
Naturalized Epistemology North American Computing and Philosophy 2007 Gordana Dodig-Crnkovic
 
Cognitive systems institute talk 8 june 2017 - v.1.0
Cognitive systems institute talk   8 june 2017 - v.1.0Cognitive systems institute talk   8 june 2017 - v.1.0
Cognitive systems institute talk 8 june 2017 - v.1.0diannepatricia
 
Foundations of Intelligence Agents
Foundations of Intelligence AgentsFoundations of Intelligence Agents
Foundations of Intelligence Agentsmahutte
 
Why are Good Theorys Good- Review
Why are Good Theorys Good- ReviewWhy are Good Theorys Good- Review
Why are Good Theorys Good- ReviewSinu G S
 
Learning
LearningLearning
Learningbutest
 

Similar to Lecture9 - Bayesian-Decision-Theory (20)

Lecture10 - Naïve Bayes
Lecture10 - Naïve BayesLecture10 - Naïve Bayes
Lecture10 - Naïve Bayes
 
Lecture1 AI1 Introduction to artificial intelligence
Lecture1 AI1 Introduction to artificial intelligenceLecture1 AI1 Introduction to artificial intelligence
Lecture1 AI1 Introduction to artificial intelligence
 
Dli milano rl_parton_sep
Dli milano rl_parton_sepDli milano rl_parton_sep
Dli milano rl_parton_sep
 
AI_1 Introduction of AI
AI_1 Introduction of AIAI_1 Introduction of AI
AI_1 Introduction of AI
 
4-ML-UNIT-IV-Bayesian Learning.pptx
4-ML-UNIT-IV-Bayesian Learning.pptx4-ML-UNIT-IV-Bayesian Learning.pptx
4-ML-UNIT-IV-Bayesian Learning.pptx
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learning
 
Supermathematics and Artificial General Intelligence
Supermathematics and Artificial General IntelligenceSupermathematics and Artificial General Intelligence
Supermathematics and Artificial General Intelligence
 
Artificial intelligence
Artificial intelligenceArtificial intelligence
Artificial intelligence
 
Artificial Intelligence AI Topics History and Overview
Artificial Intelligence AI Topics History and OverviewArtificial Intelligence AI Topics History and Overview
Artificial Intelligence AI Topics History and Overview
 
Artificial Intelligence AI Topics History and Overview
Artificial Intelligence AI Topics History and OverviewArtificial Intelligence AI Topics History and Overview
Artificial Intelligence AI Topics History and Overview
 
Artificial Intelligence
Artificial IntelligenceArtificial Intelligence
Artificial Intelligence
 
Class 27: Pythonic Objects
Class 27: Pythonic ObjectsClass 27: Pythonic Objects
Class 27: Pythonic Objects
 
Striving to Demystify Bayesian Computational Modelling
Striving to Demystify Bayesian Computational ModellingStriving to Demystify Bayesian Computational Modelling
Striving to Demystify Bayesian Computational Modelling
 
(Reverse) Engineering Intelligence - Noah Goodman - H+ Summit @ Harvard
(Reverse) Engineering Intelligence - Noah Goodman - H+ Summit @ Harvard(Reverse) Engineering Intelligence - Noah Goodman - H+ Summit @ Harvard
(Reverse) Engineering Intelligence - Noah Goodman - H+ Summit @ Harvard
 
Physics of Intelligence
Physics of IntelligencePhysics of Intelligence
Physics of Intelligence
 
Naturalized Epistemology North American Computing and Philosophy 2007
Naturalized Epistemology North American Computing and Philosophy 2007 Naturalized Epistemology North American Computing and Philosophy 2007
Naturalized Epistemology North American Computing and Philosophy 2007
 
Cognitive systems institute talk 8 june 2017 - v.1.0
Cognitive systems institute talk   8 june 2017 - v.1.0Cognitive systems institute talk   8 june 2017 - v.1.0
Cognitive systems institute talk 8 june 2017 - v.1.0
 
Foundations of Intelligence Agents
Foundations of Intelligence AgentsFoundations of Intelligence Agents
Foundations of Intelligence Agents
 
Why are Good Theorys Good- Review
Why are Good Theorys Good- ReviewWhy are Good Theorys Good- Review
Why are Good Theorys Good- Review
 
Learning
LearningLearning
Learning
 

More from Albert Orriols-Puig (20)

HAIS09-BeyondHomemadeArtificialDatasets
HAIS09-BeyondHomemadeArtificialDatasetsHAIS09-BeyondHomemadeArtificialDatasets
HAIS09-BeyondHomemadeArtificialDatasets
 
Lecture24
Lecture24Lecture24
Lecture24
 
Lecture23
Lecture23Lecture23
Lecture23
 
Lecture22
Lecture22Lecture22
Lecture22
 
Lecture21
Lecture21Lecture21
Lecture21
 
Lecture20
Lecture20Lecture20
Lecture20
 
Lecture19
Lecture19Lecture19
Lecture19
 
Lecture18
Lecture18Lecture18
Lecture18
 
Lecture17
Lecture17Lecture17
Lecture17
 
Lecture16 - Advances topics on association rules PART III
Lecture16 - Advances topics on association rules PART IIILecture16 - Advances topics on association rules PART III
Lecture16 - Advances topics on association rules PART III
 
Lecture15 - Advances topics on association rules PART II
Lecture15 - Advances topics on association rules PART IILecture15 - Advances topics on association rules PART II
Lecture15 - Advances topics on association rules PART II
 
Lecture14 - Advanced topics in association rules
Lecture14 - Advanced topics in association rulesLecture14 - Advanced topics in association rules
Lecture14 - Advanced topics in association rules
 
Lecture13 - Association Rules
Lecture13 - Association RulesLecture13 - Association Rules
Lecture13 - Association Rules
 
Lecture12 - SVM
Lecture12 - SVMLecture12 - SVM
Lecture12 - SVM
 
Lecture11 - neural networks
Lecture11 - neural networksLecture11 - neural networks
Lecture11 - neural networks
 
Lecture7 - IBk
Lecture7 - IBkLecture7 - IBk
Lecture7 - IBk
 
Lecture8 - From CBR to IBk
Lecture8 - From CBR to IBkLecture8 - From CBR to IBk
Lecture8 - From CBR to IBk
 
Lecture6 - C4.5
Lecture6 - C4.5Lecture6 - C4.5
Lecture6 - C4.5
 
Lecture5 - C4.5
Lecture5 - C4.5Lecture5 - C4.5
Lecture5 - C4.5
 
Lecture4 - Machine Learning
Lecture4 - Machine LearningLecture4 - Machine Learning
Lecture4 - Machine Learning
 

Recently uploaded

TEACHER REFLECTION FORM (NEW SET........).docx
TEACHER REFLECTION FORM (NEW SET........).docxTEACHER REFLECTION FORM (NEW SET........).docx
TEACHER REFLECTION FORM (NEW SET........).docxruthvilladarez
 
Congestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentationCongestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentationdeepaannamalai16
 
How to Add Barcode on PDF Report in Odoo 17
How to Add Barcode on PDF Report in Odoo 17How to Add Barcode on PDF Report in Odoo 17
How to Add Barcode on PDF Report in Odoo 17Celine George
 
Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17Celine George
 
ANG SEKTOR NG agrikultura.pptx QUARTER 4
ANG SEKTOR NG agrikultura.pptx QUARTER 4ANG SEKTOR NG agrikultura.pptx QUARTER 4
ANG SEKTOR NG agrikultura.pptx QUARTER 4MiaBumagat1
 
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTSGRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTSJoshuaGantuangco2
 
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17Celine George
 
Dust Of Snow By Robert Frost Class-X English CBSE
Dust Of Snow By Robert Frost Class-X English CBSEDust Of Snow By Robert Frost Class-X English CBSE
Dust Of Snow By Robert Frost Class-X English CBSEaurabinda banchhor
 
Integumentary System SMP B. Pharm Sem I.ppt
Integumentary System SMP B. Pharm Sem I.pptIntegumentary System SMP B. Pharm Sem I.ppt
Integumentary System SMP B. Pharm Sem I.pptshraddhaparab530
 
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptxMULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptxAnupkumar Sharma
 
Karra SKD Conference Presentation Revised.pptx
Karra SKD Conference Presentation Revised.pptxKarra SKD Conference Presentation Revised.pptx
Karra SKD Conference Presentation Revised.pptxAshokKarra1
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptxmary850239
 
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)lakshayb543
 
Influencing policy (training slides from Fast Track Impact)
Influencing policy (training slides from Fast Track Impact)Influencing policy (training slides from Fast Track Impact)
Influencing policy (training slides from Fast Track Impact)Mark Reed
 
ClimART Action | eTwinning Project
ClimART Action    |    eTwinning ProjectClimART Action    |    eTwinning Project
ClimART Action | eTwinning Projectjordimapav
 

Recently uploaded (20)

TEACHER REFLECTION FORM (NEW SET........).docx
TEACHER REFLECTION FORM (NEW SET........).docxTEACHER REFLECTION FORM (NEW SET........).docx
TEACHER REFLECTION FORM (NEW SET........).docx
 
Congestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentationCongestive Cardiac Failure..presentation
Congestive Cardiac Failure..presentation
 
How to Add Barcode on PDF Report in Odoo 17
How to Add Barcode on PDF Report in Odoo 17How to Add Barcode on PDF Report in Odoo 17
How to Add Barcode on PDF Report in Odoo 17
 
Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17Field Attribute Index Feature in Odoo 17
Field Attribute Index Feature in Odoo 17
 
Paradigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTAParadigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTA
 
ANG SEKTOR NG agrikultura.pptx QUARTER 4
ANG SEKTOR NG agrikultura.pptx QUARTER 4ANG SEKTOR NG agrikultura.pptx QUARTER 4
ANG SEKTOR NG agrikultura.pptx QUARTER 4
 
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptxINCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
 
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTSGRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
GRADE 4 - SUMMATIVE TEST QUARTER 4 ALL SUBJECTS
 
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
 
Dust Of Snow By Robert Frost Class-X English CBSE
Dust Of Snow By Robert Frost Class-X English CBSEDust Of Snow By Robert Frost Class-X English CBSE
Dust Of Snow By Robert Frost Class-X English CBSE
 
Integumentary System SMP B. Pharm Sem I.ppt
Integumentary System SMP B. Pharm Sem I.pptIntegumentary System SMP B. Pharm Sem I.ppt
Integumentary System SMP B. Pharm Sem I.ppt
 
YOUVE GOT EMAIL_FINALS_EL_DORADO_2024.pptx
YOUVE GOT EMAIL_FINALS_EL_DORADO_2024.pptxYOUVE GOT EMAIL_FINALS_EL_DORADO_2024.pptx
YOUVE GOT EMAIL_FINALS_EL_DORADO_2024.pptx
 
LEFT_ON_C'N_ PRELIMS_EL_DORADO_2024.pptx
LEFT_ON_C'N_ PRELIMS_EL_DORADO_2024.pptxLEFT_ON_C'N_ PRELIMS_EL_DORADO_2024.pptx
LEFT_ON_C'N_ PRELIMS_EL_DORADO_2024.pptx
 
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptxMULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
MULTIDISCIPLINRY NATURE OF THE ENVIRONMENTAL STUDIES.pptx
 
Karra SKD Conference Presentation Revised.pptx
Karra SKD Conference Presentation Revised.pptxKarra SKD Conference Presentation Revised.pptx
Karra SKD Conference Presentation Revised.pptx
 
FINALS_OF_LEFT_ON_C'N_EL_DORADO_2024.pptx
FINALS_OF_LEFT_ON_C'N_EL_DORADO_2024.pptxFINALS_OF_LEFT_ON_C'N_EL_DORADO_2024.pptx
FINALS_OF_LEFT_ON_C'N_EL_DORADO_2024.pptx
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx
 
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
 
Influencing policy (training slides from Fast Track Impact)
Influencing policy (training slides from Fast Track Impact)Influencing policy (training slides from Fast Track Impact)
Influencing policy (training slides from Fast Track Impact)
 
ClimART Action | eTwinning Project
ClimART Action    |    eTwinning ProjectClimART Action    |    eTwinning Project
ClimART Action | eTwinning Project
 

Lecture9 - Bayesian-Decision-Theory

  • 1. Introduction to Machine Learning Lecture 9 Bayesian decision theory – An introduction Albert Orriols i Puig aorriols@salle.url.edu i l @ ll ld Artificial Intelligence – Machine Learning Enginyeria i Arquitectura La Salle gy q Universitat Ramon Llull
  • 2. Recap of Lecture 5-8 LET’S START WITH DATA CLASSIFICATION Slide 2 Artificial Intelligence Machine Learning
  • 3. Recap of Lectures 5-8 We want to build decision trees How can I automatically generate these types of trees? Decide which attribute we should put in each node Decide a split point Rely on information theory We also saw many other improvements Slide 3 Artificial Intelligence Machine Learning
  • 4. Recap of Lecture 5-8 From kNN to CBR 15-NN 1-NN Key aspects Value of k Distance functions Slide 4 Artificial Intelligence Machine Learning
  • 5. Today’s Agenda Could we use probability to classify? p y y Where all began Some anecdotes on the correct use of probabilities b biliti Slide 5 Artificial Intelligence Introduction to C++
  • 6. Why Bother about Prob.? The world is a very uncertain place Almost 40 years of AI and ML dealing with uncertain domains Some researchers decided to employ ideas from probability to model concepts Before saying more let’s go to the beginning more… let s Slide 6 Artificial Intelligence Machine Learning
  • 7. Meeting the Reverend Thomas Bayes Two main works: Divine Benevolence or an Attempt to Benevolence, Prove That the Principal End of the Divine Providence and Government is the Happiness of Hi C t H i f His Creatures (1731) An Introduction to the Doctrine of Fluxions, and a Defence of the Mathematicians Against the Objections of the Author of the Analyst (published anonymously in 1736) But we are especially interested in: Essay Towards Solving a Problem in the Doctrine of Chances (1764) which was actually published p yp posthumously by Richard Price yy Slide 7 Artificial Intelligence Machine Learning
  • 8. Where These Ideas Came From? Bayes build his theory upon several ideas y yp Immanuel Kant (1724-1804) Copernican revolution: our understanding of the external world had its foundations not merely in experience, but in both experience and a priori concepts, th offering a d ii t thus ff i non-empiricist critique of rationalist philosophy Isaac Newton (1643-1727) Universal gravitation three laws of motion which dominated the scientific view of the physical universe for the next three centuries Slide 8 Artificial Intelligence Machine Learning
  • 9. What Was Bayes’ Point Bayesian p y probability y Notion of probability interpreted as partial belief rather than as frequency Bayesian estimation Calculate the validity of a proposition On the basis of a prior estimate of its probability and new relevant evidence E.g.: Before Bayes, forward probability Bf B f d b bilit given a specified number of white and black balls in an urn, what is the probability of drawing a black ball? p y g Bayes turned its attention to the converse problem given that one or more balls have been drawn, what can be said about the number of white and black balls in the urn? Slide 9 Artificial Intelligence Machine Learning
  • 10. Bayes’ Theorem Outputs the most probable hypothesis h∈H, given the data D + knowledge about prior probabilities of hypotheses in H Terminology: P(h|D): probability that h holds given data D. Posterior probability of h; confidence that h holds given D. P(h): prior probability of h (background knowledge we have about that h is a correct hypothesis) P(D): prior probability that training data D will be observed P(D|h): probability of observing D given h holds P (D | h )P (h ) P (h | D ) = P (D ) Slide 10 Artificial Intelligence Machine Learning
  • 11. Bayes’ Theorem Given H the space of possible hypothesis The Th most probable h b bl hypothesis i the one that maximizes P(h|D) h i is h h ii P(h|D): P (D | h )P (h ) hMAP ≡ arg max P (h | D ) = arg max = arg max P (D | h )P (h ) P (D ) h∈H Slide 11 Artificial Intelligence Machine Learning
  • 12. Is the Pope the Pope? The chances that a randomly chosen human being is the Pope y g p are about 1 in 6 billion Benedict XVI is the Pope p What are the chances that Benedict XVI is human? (Beck-Bornholdt (Beck Bornholdt and Dubben, 1996) Dubben Analogy to syllogistic reasoning: 1 in 6 billion Slide 12 Artificial Intelligence Machine Learning
  • 13. So, Is the Pope an Alien? Where is the trick? Probability of the data given a hypothesis H: P(D|H) ypo es s (|) Probability of the hypothesis ge given the da a P(H|D) e data: ( | ) P(D|H) is different from P(H|D) So, i th P S is the Pope An alien? A li ? Probability of being an alien P(A) Probability of being human P(H) Probability that the pope is an alien P( Pope | Alien) P( Alien) P( Alien | Pope) = p Human) + P( P P( P Pope | H Human) P( H Pope | Ali ) P( Ali ) Alien Alien Slide 13 Artificial Intelligence Machine Learning
  • 14. So, Is the Pope an Alien? What’s missing? g P(Pope|Alien) P(Human) P(H ) P(Alien) Considering Low values of P(Alien) and P(Pope|Alien) And large values of P(Human) f( ) We could “probably” say that the pope is not an alien! Slide 14 Artificial Intelligence Machine Learning
  • 15. More examples: Monty Hall Stick or switch Slide 15 Artificial Intelligence Machine Learning
  • 16. Stick or Switch I chose door number 3 Door 2 is uncovered a d contains sheep and co a s a s eep They give me the chance to change the door Should I? Use probability, not faith, to give an answer! Slide 16 Artificial Intelligence Machine Learning
  • 17. Stick or Switch I should switch! Slide 17 Artificial Intelligence Machine Learning
  • 18. Yet Another Example: The Defendant’s Fallacy The history of a murder A suspect was caught h DNA test was positive DNA test fails only 1 over 1 million times So, my suspect must be guilty, right? More specifically, it will be guilty with p = 0.999999. Agree? Slide 18 Artificial Intelligence Machine Learning
  • 19. The Defendant’s Fallacy Where is the trick now? P(coincides | innocent) as opposed to P(innocent|coincides) P(coincides | innocent) commonly misused as the probability of being innocent P(innocent | coincides) is the probability of being guilty ( ) p y gg y having that the test was positive! Does this really matter? Let’s L t’ assume a city of 10 million i h bit t it f illi inhabitants We apply the test to all the 10 million inhabitants How many of them will be positive? 10 Slide 19 Artificial Intelligence Machine Learning
  • 20. The Defendant’s Fallacy Two arguments g The prosecutor: There is 0.000001 that the suspect is innocent The d f d t In thi it f Th defendant: I this city of 10M people, the probability of th l th b bilit f the suspect being innocent is approximately 90% Who is right? The d f d t Th defendant Prove for that? You do the math Slide 20 Artificial Intelligence Machine Learning
  • 21. Next Class How we can use these concepts in machine learning Slide 21 Artificial Intelligence Introduction to C++
  • 22. Introduction to Machine Learning Lecture 9 Bayesian decision theory – An introduction Albert Orriols i Puig aorriols@salle.url.edu i l @ ll ld Artificial Intelligence – Machine Learning Enginyeria i Arquitectura La Salle gy q Universitat Ramon Llull