This document provides a summary of Lecture 9 on Bayesian decision theory and machine learning. The lecture begins with a recap of previous lectures on topics like decision trees, k-nearest neighbors, and using probabilities for classification. It then discusses Thomas Bayes and the origins of Bayesian probability. Key concepts from Bayes' theorem are explained, like calculating posterior probabilities. Examples are provided to illustrate Bayesian reasoning, such as calculating the probability that the Pope is an alien or whether to switch doors in the Monty Hall problem. The lecture concludes by discussing how these Bayesian concepts can be applied to machine learning.
Naive Bayes is a kind of classifier which uses the Bayes Theorem. It predicts membership probabilities for each class such as the probability that given record or data point belongs to a particular class.
What is the Expectation Maximization (EM) Algorithm?Kazuki Yoshida
Review of Do and Batzoglou. "What is the expectation maximization algorith?" Nat. Biotechnol. 2008;26:897. Also covers the Data Augmentation and Stan implementation. Resources at https://github.com/kaz-yos/em_da_repo
Naive Bayes is a kind of classifier which uses the Bayes Theorem. It predicts membership probabilities for each class such as the probability that given record or data point belongs to a particular class.
What is the Expectation Maximization (EM) Algorithm?Kazuki Yoshida
Review of Do and Batzoglou. "What is the expectation maximization algorith?" Nat. Biotechnol. 2008;26:897. Also covers the Data Augmentation and Stan implementation. Resources at https://github.com/kaz-yos/em_da_repo
This presentation was prepared as part of the curriculum studies for CSCI-659 Topics in Artificial Intelligence Course - Machine Learning in Computational Linguistics.
It was prepared under guidance of Prof. Sandra Kubler.
Lecture 4 Decision Trees (2): Entropy, Information Gain, Gain RatioMarina Santini
attribute selection, constructing decision trees, decision trees, divide and conquer, entropy, gain ratio, information gain, machine leaning, pruning, rules, suprisal
DBScan stands for Density-Based Spatial Clustering of Applications with Noise.
DBScan Concepts
DBScan Parameters
DBScan Connectivity and Reachability
DBScan Algorithm , Flowchart and Example
Advantages and Disadvantages of DBScan
DBScan Complexity
Outliers related question and its solution.
Slide explaining the distinction between bagging and boosting while understanding the bias variance trade-off. Followed by some lesser known scope of supervised learning. understanding the effect of tree split metric in deciding feature importance. Then understanding the effect of threshold on classification accuracy. Additionally, how to adjust model threshold for classification in supervised learning.
Note: Limitation of Accuracy metric (baseline accuracy), alternative metrics, their use case and their advantage and limitations were briefly discussed.
Machine Learning With Logistic RegressionKnoldus Inc.
Machine learning is the subfield of computer science that gives computers the ability to learn without being programmed. Logistic Regression is a type of classification algorithm, based on linear regression to evaluate output and to minimize the error.
This presentation was prepared as part of the curriculum studies for CSCI-659 Topics in Artificial Intelligence Course - Machine Learning in Computational Linguistics.
It was prepared under guidance of Prof. Sandra Kubler.
Lecture 4 Decision Trees (2): Entropy, Information Gain, Gain RatioMarina Santini
attribute selection, constructing decision trees, decision trees, divide and conquer, entropy, gain ratio, information gain, machine leaning, pruning, rules, suprisal
DBScan stands for Density-Based Spatial Clustering of Applications with Noise.
DBScan Concepts
DBScan Parameters
DBScan Connectivity and Reachability
DBScan Algorithm , Flowchart and Example
Advantages and Disadvantages of DBScan
DBScan Complexity
Outliers related question and its solution.
Slide explaining the distinction between bagging and boosting while understanding the bias variance trade-off. Followed by some lesser known scope of supervised learning. understanding the effect of tree split metric in deciding feature importance. Then understanding the effect of threshold on classification accuracy. Additionally, how to adjust model threshold for classification in supervised learning.
Note: Limitation of Accuracy metric (baseline accuracy), alternative metrics, their use case and their advantage and limitations were briefly discussed.
Machine Learning With Logistic RegressionKnoldus Inc.
Machine learning is the subfield of computer science that gives computers the ability to learn without being programmed. Logistic Regression is a type of classification algorithm, based on linear regression to evaluate output and to minimize the error.
Supermathematics and Artificial General IntelligenceJordan Bennett
In a clear way, I outline how Supermathematics may apply in Artificial General Intelligence.
I describe standard Super-Hamiltonian usage, with respect to Dwave's "Quantum Boltzmann Machine".
Striving to Demystify Bayesian Computational ModellingMarco Wirthlin
Abstract
Bayesian approaches to computational modelling have experienced a slow, but steady gain in recognition and
usage in academia and industry alike, accompanying the growing availability of evermore powerful computing
platforms at shrinking costs. Why would one use such techniques? How are those models conceived and
implemented? Which is the recommended workflow? Why make life hard when there are P-values?
In his talk, Marco Wirthlin will first attempt an introduction to statistical notions supporting Bayesian computation and
explain the difference to the Frequentist framework. In the second half, an example of a recommended workflow is
outlined on a simple toy model, with simulated data. Live coding will be used as much as possible to illustrate
concepts on an implementational level in the R language. Ample literature and media references for self-learning
will be provided during the talk.
Context and Licence
This talk was performed in the context of the “R Lunch” on the 29 of October 2019 at the University of Geneva and
was organized by Elise Tancoigne (@tancoigne) & Xavier Adam (@xvrdm). Many thanks for inviting me! :D
Code (if any) is licenced under the BSD (3 clause), while the text licence is CC BY-NC 4.0. Any derived work has
been cited. Please contact me if you see non-attributed work (marco.wirthlin@gmail.com).
What computational principles explain the success of human intelligence? I will describe recent work that combines together the unbounded flexibility of mathematical logic with the robustness of statistical inference. This combination brings us several steps closer to understanding human intelligence -- and to the tools for true intelligence engineering.
Noah D. Goodman is a research scientist in the Department of Brain and Cognitive Sciences at MIT, and a member of the Computer Science and Artificial Intelligence Laboratory. He studies the computational basis of human thought, merging behavioral experiments with formal methods from statistics and logic. He received his Ph.D. in mathematics from the University of Texas at Austin. After a brief stint as a Chicago real estate developer, he joined the Computational Cognitive Science group at MIT. Goodman has published more than thirty publications in psychology, cognitive science, artificial intelligence, and mathematics. Several of these papers have won awards.
Naturalized Epistemology North American Computing and Philosophy 2007 Gordana Dodig-Crnkovic
Gordana Dodig-Crnkovic
Knowledge Generation as Natural Computation,
Journal of Systemics, Cybernetics and Informatics, Vol 6, No 2, 2008
http://www.iiisci.org/Journal/CV$/sci/pdfs/G774PI.pdf
Cognitive systems institute talk 8 june 2017 - v.1.0diannepatricia
José Hernández-Orallo, Full Professor, Department of Information Systems and Computation at the Universitat Politecnica de València, presentation “Evaluating Cognitive Systems: Task-oriented or Ability-oriented?” as part of the Cognitive Systems Institute Speaker Series.
The approaches to Artificial Intelligence (AI) in the last century may be labelled as (a) trying to understand and copy (human) nature, (b) being based on heuristic considerations, (c) being formal but from the outset (provably) limited, (d) being (mere) frameworks that leave crucial aspects unspecified. This decade has spawned the first theory of AI, which (e) is principled, formal, complete, and general. This theory, called Universal AI, is about ultimate super-intelligence. It can serve as a gold standard for General AI, and implicitly proposes a formal definition of machine intelligence. After a brief review of the various approaches to (general) AI, I will give an introduction to Universal AI, concentrating on the philosophical, mathematical, and computational aspects behind it. I will also discuss various implications and future challenges.
In this third session of the Elements of AI Luxembourg series of webinars, our guest speaker Dr. Alexander Steen talks about Logic and Automated Reasoning. More information, and a recording of the session, can be found on our reddit page:
eofai.lu/reddit
Similar to Lecture9 - Bayesian-Decision-Theory (20)
The Art Pastor's Guide to Sabbath | Steve ThomasonSteve Thomason
What is the purpose of the Sabbath Law in the Torah. It is interesting to compare how the context of the law shifts from Exodus to Deuteronomy. Who gets to rest, and why?
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
How to Create Map Views in the Odoo 17 ERPCeline George
The map views are useful for providing a geographical representation of data. They allow users to visualize and analyze the data in a more intuitive manner.
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
The Roman Empire A Historical Colossus.pdfkaushalkr1407
The Roman Empire, a vast and enduring power, stands as one of history's most remarkable civilizations, leaving an indelible imprint on the world. It emerged from the Roman Republic, transitioning into an imperial powerhouse under the leadership of Augustus Caesar in 27 BCE. This transformation marked the beginning of an era defined by unprecedented territorial expansion, architectural marvels, and profound cultural influence.
The empire's roots lie in the city of Rome, founded, according to legend, by Romulus in 753 BCE. Over centuries, Rome evolved from a small settlement to a formidable republic, characterized by a complex political system with elected officials and checks on power. However, internal strife, class conflicts, and military ambitions paved the way for the end of the Republic. Julius Caesar’s dictatorship and subsequent assassination in 44 BCE created a power vacuum, leading to a civil war. Octavian, later Augustus, emerged victorious, heralding the Roman Empire’s birth.
Under Augustus, the empire experienced the Pax Romana, a 200-year period of relative peace and stability. Augustus reformed the military, established efficient administrative systems, and initiated grand construction projects. The empire's borders expanded, encompassing territories from Britain to Egypt and from Spain to the Euphrates. Roman legions, renowned for their discipline and engineering prowess, secured and maintained these vast territories, building roads, fortifications, and cities that facilitated control and integration.
The Roman Empire’s society was hierarchical, with a rigid class system. At the top were the patricians, wealthy elites who held significant political power. Below them were the plebeians, free citizens with limited political influence, and the vast numbers of slaves who formed the backbone of the economy. The family unit was central, governed by the paterfamilias, the male head who held absolute authority.
Culturally, the Romans were eclectic, absorbing and adapting elements from the civilizations they encountered, particularly the Greeks. Roman art, literature, and philosophy reflected this synthesis, creating a rich cultural tapestry. Latin, the Roman language, became the lingua franca of the Western world, influencing numerous modern languages.
Roman architecture and engineering achievements were monumental. They perfected the arch, vault, and dome, constructing enduring structures like the Colosseum, Pantheon, and aqueducts. These engineering marvels not only showcased Roman ingenuity but also served practical purposes, from public entertainment to water supply.
Ethnobotany and Ethnopharmacology:
Ethnobotany in herbal drug evaluation,
Impact of Ethnobotany in traditional medicine,
New development in herbals,
Bio-prospecting tools for drug discovery,
Role of Ethnopharmacology in drug evaluation,
Reverse Pharmacology.
Instructions for Submissions thorugh G- Classroom.pptxJheel Barad
This presentation provides a briefing on how to upload submissions and documents in Google Classroom. It was prepared as part of an orientation for new Sainik School in-service teacher trainees. As a training officer, my goal is to ensure that you are comfortable and proficient with this essential tool for managing assignments and fostering student engagement.
The Indian economy is classified into different sectors to simplify the analysis and understanding of economic activities. For Class 10, it's essential to grasp the sectors of the Indian economy, understand their characteristics, and recognize their importance. This guide will provide detailed notes on the Sectors of the Indian Economy Class 10, using specific long-tail keywords to enhance comprehension.
For more information, visit-www.vavaclasses.com
Chapter 3 - Islamic Banking Products and Services.pptx
Lecture9 - Bayesian-Decision-Theory
1. Introduction to Machine
Learning
Lecture 9
Bayesian decision theory – An introduction
Albert Orriols i Puig
aorriols@salle.url.edu
i l @ ll ld
Artificial Intelligence – Machine Learning
Enginyeria i Arquitectura La Salle
gy q
Universitat Ramon Llull
2. Recap of Lecture 5-8
LET’S START WITH DATA
CLASSIFICATION
Slide 2
Artificial Intelligence Machine Learning
3. Recap of Lectures 5-8
We want to build decision trees
How can I automatically
generate these types
of trees?
Decide which attribute we
should put in each node
Decide a split point
Rely on information theory
We also saw many other improvements
Slide 3
Artificial Intelligence Machine Learning
4. Recap of Lecture 5-8
From kNN to CBR
15-NN 1-NN
Key aspects
Value of k
Distance functions
Slide 4
Artificial Intelligence Machine Learning
5. Today’s Agenda
Could we use probability to classify?
p y y
Where all began
Some anecdotes on the correct use of
probabilities
b biliti
Slide 5
Artificial Intelligence Introduction to C++
6. Why Bother about Prob.?
The world is a very uncertain place
Almost 40 years of AI and ML dealing with uncertain
domains
Some researchers decided to employ ideas from
probability to model concepts
Before saying more let’s go to the beginning
more… let s
Slide 6
Artificial Intelligence Machine Learning
7. Meeting the Reverend Thomas Bayes
Two main works:
Divine Benevolence or an Attempt to
Benevolence,
Prove That the Principal End of the Divine
Providence and Government is the
Happiness of Hi C t
H i f His Creatures (1731)
An Introduction to the Doctrine of Fluxions,
and a Defence of the Mathematicians
Against the Objections of the Author of the
Analyst (published anonymously in 1736)
But we are especially interested in:
Essay Towards Solving a Problem in the Doctrine of Chances (1764)
which was actually published p
yp posthumously by Richard Price
yy
Slide 7
Artificial Intelligence Machine Learning
8. Where These Ideas Came From?
Bayes build his theory upon several ideas
y yp
Immanuel Kant (1724-1804)
Copernican revolution: our understanding
of the external world had its foundations
not merely in experience, but in both experience
and a priori concepts, th offering a
d ii t thus ff i
non-empiricist critique of rationalist philosophy
Isaac Newton (1643-1727)
Universal gravitation
three laws of motion which dominated
the scientific view of the physical universe
for the next three centuries
Slide 8
Artificial Intelligence Machine Learning
9. What Was Bayes’ Point
Bayesian p
y probability
y
Notion of probability interpreted as partial belief rather than as
frequency
Bayesian estimation
Calculate the validity of a proposition
On the basis of a prior estimate of its probability and new
relevant evidence
E.g.:
Before Bayes, forward probability
Bf B f d b bilit
given a specified number of white and black balls in an urn, what
is the probability of drawing a black ball?
p y g
Bayes turned its attention to the converse problem
given that one or more balls have been drawn, what can be said
about the number of white and black balls in the urn?
Slide 9
Artificial Intelligence Machine Learning
10. Bayes’ Theorem
Outputs the most probable hypothesis h∈H, given the data D +
knowledge about prior probabilities of hypotheses in H
Terminology:
P(h|D): probability that h holds given data D. Posterior probability of h;
confidence that h holds given D.
P(h): prior probability of h (background knowledge we have about that h is a
correct hypothesis)
P(D): prior probability that training data D will be observed
P(D|h): probability of observing D given h holds
P (D | h )P (h )
P (h | D ) =
P (D )
Slide 10
Artificial Intelligence Machine Learning
11. Bayes’ Theorem
Given H the space of possible hypothesis
The
Th most probable h
b bl hypothesis i the one that maximizes P(h|D)
h i is h h ii P(h|D):
P (D | h )P (h )
hMAP ≡ arg max P (h | D ) = arg max = arg max P (D | h )P (h )
P (D )
h∈H
Slide 11
Artificial Intelligence Machine Learning
12. Is the Pope the Pope?
The chances that a randomly chosen human being is the Pope
y g p
are about 1 in 6 billion
Benedict XVI is the Pope
p
What are the chances that Benedict XVI is human?
(Beck-Bornholdt
(Beck Bornholdt and Dubben, 1996)
Dubben
Analogy to syllogistic reasoning: 1 in 6 billion
Slide 12
Artificial Intelligence Machine Learning
13. So, Is the Pope an Alien?
Where is the trick?
Probability of the data given a
hypothesis H: P(D|H)
ypo es s (|)
Probability of the hypothesis
ge
given the da a P(H|D)
e data: ( | )
P(D|H) is different from P(H|D)
So, i th P
S is the Pope An alien?
A li ?
Probability of being an alien P(A)
Probability of being human P(H)
Probability that the pope is an alien
P( Pope | Alien) P( Alien)
P( Alien | Pope) =
p
Human) + P( P
P( P
Pope | H
Human) P( H Pope | Ali ) P( Ali )
Alien Alien
Slide 13
Artificial Intelligence Machine Learning
14. So, Is the Pope an Alien?
What’s missing?
g
P(Pope|Alien)
P(Human)
P(H )
P(Alien)
Considering
Low values of P(Alien) and P(Pope|Alien)
And large values of P(Human)
f( )
We could “probably” say that the pope is not an alien!
Slide 14
Artificial Intelligence Machine Learning
15. More examples: Monty Hall
Stick or switch
Slide 15
Artificial Intelligence Machine Learning
16. Stick or Switch
I chose door number 3
Door 2 is uncovered
a d contains sheep
and co a s a s eep
They give me the chance to change the door
Should I?
Use probability, not faith,
to give an answer!
Slide 16
Artificial Intelligence Machine Learning
17. Stick or Switch
I should switch!
Slide 17
Artificial Intelligence Machine Learning
18. Yet Another Example: The Defendant’s Fallacy
The history of a murder
A suspect was caught
h
DNA test was positive
DNA test fails only 1 over 1 million times
So, my suspect must be guilty, right?
More specifically, it will be guilty with p = 0.999999. Agree?
Slide 18
Artificial Intelligence Machine Learning
19. The Defendant’s Fallacy
Where is the trick now?
P(coincides | innocent) as opposed to P(innocent|coincides)
P(coincides | innocent) commonly misused as the probability
of being innocent
P(innocent | coincides) is the probability of being guilty
( ) p y gg y
having that the test was positive!
Does this really matter?
Let’s
L t’ assume a city of 10 million i h bit t
it f illi inhabitants
We apply the test to all the 10 million inhabitants
How many of them will be positive?
10
Slide 19
Artificial Intelligence Machine Learning
20. The Defendant’s Fallacy
Two arguments
g
The prosecutor: There is 0.000001 that the suspect is innocent
The d f d t In thi it f
Th defendant: I this city of 10M people, the probability of th
l th b bilit f the
suspect being innocent is approximately 90%
Who is right?
The d f d t
Th defendant
Prove for that? You do the math
Slide 20
Artificial Intelligence Machine Learning
21. Next Class
How we can use these concepts in machine learning
Slide 21
Artificial Intelligence Introduction to C++
22. Introduction to Machine
Learning
Lecture 9
Bayesian decision theory – An introduction
Albert Orriols i Puig
aorriols@salle.url.edu
i l @ ll ld
Artificial Intelligence – Machine Learning
Enginyeria i Arquitectura La Salle
gy q
Universitat Ramon Llull
