SlideShare a Scribd company logo
1 of 71
Download to read offline
Probability Theory
Anthony J. Evans
Professor of Economics, ESCP Europe
www.anthonyjevans.com
(cc) Anthony J. Evans 2019 | http://creativecommons.org/licenses/by-nc-sa/3.0/
Contents
• Measuring probability
• Combining probabilities
• Bayes theorem
• Factorials
• Permutations
• Combinations
2
Probability
• Def: “The branch of mathematics that describes the
pattern of chance outcomes”*
• A number between 0 and 1 that represents the chance that
an event will occur
0 1
Impossible Certain
Risk
*Moore & McCabe (1999), p. 288
Probability
Increasing likelihood of occurrence
Uncertainty
3
Measuring Probability
Number of times the event (E ) can occur
P(E) =
Total number of possible outcomes
Classical
Relative
Frequency
• Assigning probabilities based on the assumption
of equally likely outcomes
Number of times the event (E ) actually occurs
P(E) =
Total number of observed outcomes
• Assigning probabilities based on experimentation
or historical data.
4
Measuring Probability: Classical Method
• What is the probability of rolling a 2 on a fair dice?
• Possible outcomes: [1,2,3,4,5,6]; n=6
P(2) = 1/6
5
Measuring Probability: Relative Frequency Method
• Lucas Tool Rental would like to assign probabilities to the
number of floor polishers it rents per day
• Office records show the following frequencies of daily
rentals for the last 40 days:
Number of Number
polishers rented of days Probability
0 4 .10 = 4/40
1 6 .15 = 6/40
2 18 .45 etc.
3 10 .25
4 2 .05
40 1.00
6
Consider the following possible event:
a) Picking the Ace of Spades from a full deck of cards
• What is P(A)? 1/52
7
Sample Space
Event A
Complement events
• The complement (~A or AC) of event A is defined as the
event consisting of all sample points that are not in A
a) Picking the Ace of Spades from a full deck of cards
• What is P(AC)? = 51/52
8
Sample Space
Event A AC
Union probability
• Two events are said to be mutually exclusive if they cannot
occur at the same time – i.e. the events have no sample
points in common
• The probability that one of several mutually exclusive
events occurs, is simply the sum of the individual
probabilities
– “OR” means “+”
• P(AUB) = P(A) + P(B)
9
Sample Space
Event A Event B
)( BAP È
Union probability - example
a) Rolling a die and getting a 4
b) Rolling a die and getting a 5
• P(AUB) = 1/6 + 1/6 = 2/6 = 1/3
10
Sample Space
Event A Event B
)( BAP È
Joint probability
• Two events are said to be independent if the outcomes
have no effect on each other
• The probability that two independent events occur is
simply the multiplication of the individual probabilities
– “AND” means “X”
• P(A∩B) = P(A) x P(B)
11
)( BAP Ç
Sample Space
Event A Event B
Joint probability - example
a) Picking any Spade from a full deck of cards
b) Rolling a die and getting a 3
• P(A∩B) = 1/4 x 1/6 = 1/24
12
)( BAP Ç
Sample Space
Event A Event B
KRUPNIK INVESTMENT
• Analysts for Krupnik Investments have assigned
probabilities for six scenarios, and the respective
profitability of portfolios A and B
• Note: there’s no way to verify the assigned probabilities
per outcome.
13
KRUPNIK INVESTMENT: Getting Started
• Event A: Portfolio A is profitable
P(A) = Outcomes 1, 2, 3 or 6
= 0.3 + 0.06 + 0.2 + 0.14 = 70%
• Event B: Portfolio B is profitable
P(B) = Outcomes 1, 3, 5 or 6
= 0.3 + 0.2 + 0.24 + 0.14 = 88%
14
KRUPNIK INVESTMENT: Complement
• What is the probability of 5C?
• P(5c
) = 76%
15
KRUPNIK INVESTMENT: A more complicated example
• Are Events A and B mutually exclusive?
– No: it is possible for a sample point to be part of both Event A and Event
B (e.g. Outcome 1)
• Are Events A and B independent?
– No: the profitability of A may well influence the profitability of B
– We can’t know for sure, so assume they are not independent and then
later on we can test for this to become certain
16
KRUPNIK INVESTMENT: Union
• What’s the likelihood that either A or B are profitable?
• What is the probability of the union of Event A and Event B?
= Outcome 1, 2, 3, 5 or 6
= 0.3 + 0.06 + 0.2 + 0.24 + 0.14
= 94%
By inspection
)( BAP È
17
KRUPNIK INVESTMENT: Joint
• What’s the likelihood that both A and B are profitable?
• What is the joint probability of Event A and Event B?
= Outcomes 1, 3, 6
= 0.3 + 0.2 + 0.14
= 64%
By inspection
18
)( BAP Ç
KRUPNIK INVESTMENT: Union
• If the events are not mutually exclusive, they overlap
• The union of events A and B is the event containing all
sample points that are in A or B or both
• What is the likelihood that either A or B occur?
– The addition law:
– P(AUB) = P(A) + P(B) – P(A ∩ B)
19
Sample Space
Event A Event B
)( BAP È
)( BAP Ç
We need to
avoid
double
counting
KRUPNIK INVESTMENT: Union
• What’s the likelihood that either A or B are profitable?
• What is the probability of the union of Event A and Event B?
• P(AUB) = P(A) + P(B) – P(A ∩ B)
= 0.7 + 0.88 – 0.64
= 94%
Addition Law
)( BAP È
20
KRUPNIK INVESTMENT: Conditional
• The conditional probability of event A wrt B is the set of
all sample points that are in both A and B
• The conditional is denoted by A|B
• P(A|B)= P(A∩B) / P(B)
21
)|( BAP
Sample Space
Event B
Conditional
KRUPNIK INVESTMENT: Conditional
• What’s the likelihood that A occurs, given B occurring?
• What is the conditional probability of Event A wrt Event B?
– It’s the ratio of (A∩B) wrt P(B)
• P(A|B) = P(A∩B) / P(B)
= 0.64 / 0.88
= 73%
22
)|( BAP
KRUPNIK INVESTMENT: Conditional
• What’s the likelihood that B occurs, given A occurring?
• What is the conditional probability of Event B wrt Event A?
– It’s the ratio of (A∩B) wrt P(B)
• P(B|A) = P(A∩B) / P(A)
= 0.64 / 0.70
= 91%
23
€
P(B | A)
KRUPNIK INVESTMENT: Joint
• If the events are not independent, they overlap
• The intersection of events A and B is the set of all sample
points that are in both A and B
• The intersection is denoted by (A∩B)
• What’s the likelihood that both A and B occur?
– The multiplication law:
– P(A∩B) = P(A) x P(B|A)
24
)( BAP Ç
Sample Space
Event A Event B
)( BAP Ç
KRUPNIK INVESTMENT: Joint
• What’s the likelihood that both A and B are profitable?
• What is the joint probability of Event A and Event B?
• P(A∩B) = P(A) x P(B|A)
= 0.70 x (0.914)
= 64%
Multiplication Law
25
)( BAP Ç
Union
… mutually
exclusive?
… independent?
Relationship between A and B
NO
YES
YES
NO
• P(AUB) = P(A) + P(B)
• P(AUB) = P(A) + P(B) – P(A ∩ B)
• P(A∩B) = P(A) x P(B)
• P(A∩B) = P(A) x P(B|A)
Are events
A and B…
Formula
…A or B happening
…A and B happening
Combining probabilities: Rules
Joint
26
Marginal
The probability
of A occurring
Union
The probability
of A or B
occurring
Joint
The probability
of A and B
occurring
Conditional
The probability
of A occurring
given that B has
occurred
BA BA BA
)(AP )( BAP È )( BAP Ç )|( BAP
Combining probabilities: Summary
27
Conditional Probability: Defective Machine Example
• When defective items are found, it is not immediately
possible to say from which machine they came
• We can use conditional probability to identify the machine
which is most likely to have produced a defective item
A company uses three machines, labelled A,B and C
• 40% of production goes through A, 50% to B, and 10% to C
• The wastage rate for machine A is 6%, 1% for machine B,
and 10% for machine C
• What is the probability that a piece of waste has come
from machine A?
28
BAThis is saying
“what’s the
probability that
there is a piece
of waste from
machine A”
B This is saying
“what’s the
probability that
this piece of
waste came
from machine A”
Conditional Probability: Defective Machine Example
0.10
0.50
0.10
Acceptable
Defective
Acceptable
Acceptable
Defective
Defective
0.90
0.01
0.99
0.06
0.40 0.94
Production process
A
B
C
Machine
0.4*0.94 = 0.376
Joint probability
0.4*0.06 = 0.024
0.5*0.99 = 0.495
0.5*0.01 = 0.005
0.1*0.90 = 0.09
0.1*0.10 = 0.01
29
Conditional Probability: Defective Machine Example
What’s the probability
that a defective product
will come out of the
production line?
Given that the product is
defective, what’s the
probability that it came out
of machine A?
)(
)(
)|(
machineanyfromdefectiveP
AmachinefromdefectiveP
defectiveAP =
Problem Solution
P(defective)=0.024+0.005+0.010 = 0.039
039.0
024.0
=
%62=
30
P(A|B) = P(A∩B) / P(B)
Conditional Probability: Defective Machine Example
Given that the product is
defective, what’s the
probability that it came out
of machine B?
)(
)(
)|(
machineanyfromdefectiveP
BmachinefromdefectiveP
defectiveBP =
039.0
005.0
=
%13=
Given that the product is
defective, what’s the
probability that it came out
of machine C?
)(
)(
)|(
machineanyfromdefectiveP
CmachinefromdefectiveP
defectiveCP =
039.0
010.0
=
= 26%
31
Probability theory and the dating market
• Suppose you are told you must marry, and that you must choose your spouse out
of 100 applicants. You may interview each applicant once. After each interview
you must decide whether to marry that person. If you decline, you lose the
opportunity forever. If you work your way through 99 applicants without choosing
one, you must marry the 100th
• If you interview half the potential partners then stop at the next best one - that
is, the first one better than the best person you've already interviewed - you will
marry the very best candidate about 25 per cent of the time. Once again,
probability explains why. A quarter of the time, the second best partner will be
in the first 50 people and the very best in the second. So 25 per cent of the
time, the rule "stop at the next best one" will see you marrying the best
candidate. Much of the rest of the time, you will end up marrying the 100th
person, who has a 1 in 100 chance of being the worst, but hey, this is
probability, not certainty.
• John Gilbert and Frederick Mosteller of Harvard University proved that you could
raise your odds to 37 per cent by interviewing 37 people then stopping at the
next best. The number 37 comes from dividing 100 by e, the base of the natural
logarithms, which is roughly equal to 2.72. Gilbert and Mosteller's law works no
matter how many candidates there are - you simply divide the number of options
by e
32“What's luck got to do with it? The math of gambling”, Helen Thomson, New Scientist, 11 August 2009
The Monty Hall problem
• Suppose the contestants on a game show are given the
choice of three doors: Behind one door is a car; behind the
others, goats. After a contestant picks a door, the host,
who knows what’s behind all the doors, opens one of the
unchosen doors, which reveals a goat. He then says to the
contestant, “Do you want to switch to the other unopened
door?”
• Should the contestant switch?
33See Mlodinow, 2008, p.43 & 53
Applet: http://math.ucsd.edu/~crypto/Monty/monty.html
The Monty Hall problem (2)
34Text taken from Oxford University Entrance Exam, “Economics and Management” May 2000
Bayes Theorem
• Bayes Theorem is a way to calculate conditional probability
• It allows us to update original probabilities in light of new
data
New
Information
Application
of Bayes’
Theorem
Posterior
Probabilities
Prior
Probabilities
35
Bayes Theorem
• P(A) = probability of A
• P(A|B) = probability of A given B
• P(~A) = probability of not A
[ ] [ ])(~)|~()()|(
)()|(
)|(
APABPAPABP
APABP
BAP
×+×
×
=
36
Bayes Theorem: Mammograms
• 1% of women at age forty who participate in routine screening have breast
cancer. 80% of women with breast cancer will get positive mammograms. 9.6%
of women without breast cancer will also get positive mammograms. A woman
in this age group had a positive mammography in a routine screening. What is
the probability that she actually has breast cancer?
37This example is from Yudowsky but the explanation and visualisation is from https://oscarbonilla.com/2009/05/visualizing-bayes-theorem/
A = Women with cancer
Consider all women
B = Positive mammogram
(Need to cover 80% of event A and
9.6% of Ac)
Most positive results do not
have cancer
Bayes Theorem: Mammograms
• 1% of women at age forty who participate in routine screening have breast
cancer. 80% of women with breast cancer will get positive mammograms. 9.6%
of women without breast cancer will also get positive mammograms. A woman
in this age group had a positive mammography in a routine screening. What is
the probability that she actually has breast cancer?
38This example is from Yudowsky but the explanation and visualisation is from https://oscarbonilla.com/2009/05/visualizing-bayes-theorem/
P(A)=0.01 0.80=P(B|A)
[ ] [ ])(~)|~()()|(
)()|(
)|(
APABPAPABP
APABP
BAP
×+×
×
=
𝑃 𝐴 𝐵 =
0.80×0.01
[0.80×0.01] + [0.096×0.99]
0.096=P(B|~A)
𝑃 𝐴 𝐵 = 7.76%
Application: mammograms
• For every 2,000 mammograms
– 1 life saved
– 200 false positives, which lead to 10 painful and
unnecessary surgeries
• ~ 84% success rate
• Younger women are more likely to get a false positive
– Recommended age for having one has risen as a result
• If you’re between 40-50 years old there’s a 50%-60% chance
of a false positive
39https://www.stuffyoushouldknow.com/podcasts/what-are-false-positives.htm
Bayes Theorem: Drug Testing Example
• A certain drug test is 99% accurate, that is, the test
will correctly identify a drug user as testing positive
99% of the time, and will correctly identify a non-
user as testing negative 99% of the time.
• This would seem to be a relatively accurate test, but
Bayes' theorem will reveal a potential flaw.
• Let's assume a corporation decides to test its
employees for opium use, and 0.5% of the employees
use the drug.
• We want to know the probability that, given a
positive drug test, an employee is actually a drug
user. Let “A" be the event of being a drug user and
“~A" indicate being a non-user
• Let “B" be the event of a positive drug test
Source: http://en.wikipedia.org/wiki/Bayes'_theorem 40
Bayes Theorem: Drug Testing Example
We need to know the following:
• P(A), or the probability that the employee is a drug user,
regardless of any other information. This is 0.005, since
0.5% of the employees are drug users.
• P(~A), or the probability that the employee is not a drug
user. This is 1-Pr(A), or 0.995.
• P(B|A), or the probability that the test is positive, given
that the employee is a drug user. This is 0.99, since the
test is 99% accurate.
• P(B|~A), or the probability that the test is positive, given
that the employee is not a drug user. This is 0.01, since
the test will produce a false positive for 1% of non-users.
41
Bayes Theorem: Drug Testing Example
• Compute the probability that an employee that
tested positive is actually a drug user:
• The probability that the employee is actually a drug
user is only about 33%.
• The rarer the condition for which we are testing, the
greater percentage of the positive tests will be false
positives
• This illustrates why it is important to do follow-up
tests
[ ] [ ]995.001.0005.099.0
005.099.0
)|(
´+´
´
=BAP
00995.000495.0
00495.0
)|(
+
=BAP
3322.0)|( =BAP
42
False positives
Does the person have the
disease?
Yes No
What is the
test result?
Positive
True
positive
False
positive
Negative
False
negative
True
negative
43
Factorials
The factorial of a number (n) is the product of all positive
integers less than or equal to n
(By definition 0!=1)
Example: What’s 10!
10! = 10*9*8*7*6*5*4*3*2*1
= 3,628,800
n1)3...(n21=!n ⋅−⋅⋅
44
Permutations
A permutation is an ordered sequence
• A possible permutation of the number sequence [1-6] is
– 3,5,2,1,6,4
• Number of permutations of
– n different things
– k at a time
• Used when order matters
)!(
!
kn
n
Pk
n
−
=
45
Permutations: Example
• What are the odds of getting 6 numbers (between 1-49) in
the right order?
520,347,068,10=
)!649(
!49
6
49
-
=P
46
Combinations
A combination is an un-ordered collection of unique
elements
•Number of combinations of
– n things
– k at a time
•Used when order doesn't matters
– In permutations abc is different from bca
– In combinations we are concerned only that a, b, and c
have been selected
• abc and bca are the same combination
)!(!
!
knk
n
k
n
Ck
n
-
=÷
ø
ö
ç
è
æ
=
47
Combinations: Example
• You have 5 shirts, but you will select only 3 for your
vacation. In how many different ways can you do this?
)!35(!3
!5
3
5
3
5
-
=÷
ø
ö
ç
è
æ
=C
103
5
=C
48
Combinations: Example
What are the odds of winning the UK Lotto?
• Need to get 6 numbers (between 1-49) in any order
)!649(!6
!49
6
49
-
=C
mC 146
49
=
49
Summary
• There’s a difference between risk and uncertainty –
quantities techniques focus purely on risk
• Probability can be difficult to get your head around, so
keep things intuitive and do sanity tests
• Casinos are a good place for revision
50
Solutions
51
Measuring Probability: Classical Method
• What is the probability of rolling a 2 on a fair dice?
• Possible outcomes: [1,2,3,4,5,6]; n=6
P(2) = 1/6
52
Consider the following possible event:
a) Picking the Ace of Spades from a full deck of cards
• What is P(A)? 1/52
53
Sample Space
Event A
Complement events
• The complement (~A or AC) of event A is defined as the
event consisting of all sample points that are not in A
a) Picking the Ace of Spades from a full deck of cards
• What is P(AC)? = 51/52
54
Sample Space
Event A AC
Union probability - example
a) Rolling a die and getting a 4
b) Rolling a die and getting a 5
• P(AUB) = 1/6 + 1/6 = 2/6 = 1/3
55
Sample Space
Event A Event B
)( BAP È
Joint probability - example
a) Picking any Spade from a full deck of cards
b) Rolling a die and getting a 3
• P(A∩B) = 1/4 x 1/6 = 1/24
56
)( BAP Ç
Sample Space
Event A Event B
KRUPNIK INVESTMENT: Getting Started
• Event A: Portfolio A is profitable
P(A) = Outcomes 1, 2, 3 or 6
= 0.3 + 0.06 + 0.2 + 0.14 = 70%
• Event B: Portfolio B is profitable
P(B) = Outcomes 1, 3, 5 or 6
= 0.3 + 0.2 + 0.24 + 0.14 = 88%
57
KRUPNIK INVESTMENT: Complement
• What is the probability of 5C?
• P(5c
) = 76%
58
KRUPNIK INVESTMENT: Union
• What’s the likelihood that either A or B are profitable?
• What is the probability of the union of Event A and Event B?
= Outcome 1, 2, 3, 5 or 6
= 0.3 + 0.06 + 0.2 + 0.24 + 0.14
= 94%
By inspection
)( BAP È
59
KRUPNIK INVESTMENT: Joint
• What’s the likelihood that both A and B are profitable?
• What is the joint probability of Event A and Event B?
= Outcomes 1, 3, 6
= 0.3 + 0.2 + 0.14
= 64%
By inspection
60
)( BAP Ç
KRUPNIK INVESTMENT: Union
• What’s the likelihood that either A or B are profitable?
• What is the probability of the union of Event A and Event B?
• P(AUB) = P(A) + P(B) – P(A ∩ B)
= 0.70 + 0.88 – 0.64
= 94%
Addition Law
)( BAP È
61
KRUPNIK INVESTMENT: Conditional
• What’s the likelihood that A occurs, given B occurring?
• What is the conditional probability of Event A wrt Event B?
– It’s the ratio of (A∩B) wrt P(B)
• P(A|B) = P(A∩B) / P(B)
= 0.64 / 0.88
= 73%
62
)|( BAP
KRUPNIK INVESTMENT: Conditional
• What’s the likelihood that B occurs, given A occurring?
• What is the conditional probability of Event B wrt Event A?
– It’s the ratio of (A∩B) wrt P(B)
• P(B|A) = P(A∩B) / P(A)
= 0.64 / 0.70
= 91%
63
€
P(B | A)
KRUPNIK INVESTMENT: Joint
• What’s the likelihood that both A and B are profitable?
• What is the joint probability of Event A and Event B?
• P(A∩B) = P(A) x P(B|A)
= 0.70 x (0.914)
= 64%
Multiplication Law
64
)( BAP Ç
Conditional Probability: Defective Machine Example
Given that the product is
defective, what’s the
probability that it came out
of machine B?
)(
)(
)|(
machineanyfromdefectiveP
BmachinefromdefectiveP
defectiveBP =
039.0
005.0
=
%13=
Given that the product is
defective, what’s the
probability that it came out
of machine C?
)(
)(
)|(
machineanyfromdefectiveP
CmachinefromdefectiveP
defectiveCP =
039.0
010.0
=
= 26%
65
Bayes Theorem: Mammograms
• 1% of women at age forty who participate in routine screening have breast
cancer. 80% of women with breast cancer will get positive mammograms. 9.6%
of women without breast cancer will also get positive mammograms. A woman
in this age group had a positive mammography in a routine screening. What is
the probability that she actually has breast cancer?
66This example is from Yudowsky but the explanation and visualisation is from https://oscarbonilla.com/2009/05/visualizing-bayes-theorem/
P(A)=0.01 0.80=P(B|A)
[ ] [ ])(~)|~()()|(
)()|(
)|(
APABPAPABP
APABP
BAP
×+×
×
=
𝑃 𝐴 𝐵 =
0.80×0.01
[0.80×0.01] + [0.096×0.99]
0.096=P(B|~A)
𝑃 𝐴 𝐵 = 7.76%
Factorials
The factorial of a number (n) is the product of all positive
integers less than or equal to n
(By definition 0!=1)
Example: What’s 10!
10! = 10*9*8*7*6*5*4*3*2*1
= 3,628,800
n1)3...(n21=!n ⋅−⋅⋅
67
Permutations: Example
• What are the odds of getting 6 numbers (between 1-49) in
the right order?
520,347,068,10=
)!649(
!49
6
49
-
=P
68
Combinations: Example
• You have 5 shirts, but you will select only 3 for your
vacation. In how many different ways can you do this?
)!35(!3
!5
3
5
3
5
-
=÷
ø
ö
ç
è
æ
=C
103
5
=C
69
Combinations: Example
What are the odds of winning the UK Lotto?
• Need to get 6 numbers (between 1-49) in any order
)!649(!6
!49
6
49
-
=C
mC 146
49
=
70
• This presentation forms part of a free, online course
on analytics
• http://econ.anthonyjevans.com/courses/analytics/
71

More Related Content

What's hot

03+probability+distributions.ppt
03+probability+distributions.ppt03+probability+distributions.ppt
03+probability+distributions.ppt
abhinav3874
 
Theoretical probability distributions
Theoretical probability distributionsTheoretical probability distributions
Theoretical probability distributions
Hasnain Baber
 

What's hot (20)

Conditional probability
Conditional probabilityConditional probability
Conditional probability
 
Probability
ProbabilityProbability
Probability
 
Probability
ProbabilityProbability
Probability
 
Probablity ppt maths
Probablity ppt mathsProbablity ppt maths
Probablity ppt maths
 
Statistics & probability
Statistics & probabilityStatistics & probability
Statistics & probability
 
Binomial distribution
Binomial distributionBinomial distribution
Binomial distribution
 
Basic probability concept
Basic probability conceptBasic probability concept
Basic probability concept
 
PROBABILITY
PROBABILITYPROBABILITY
PROBABILITY
 
Statistics: Probability
Statistics: ProbabilityStatistics: Probability
Statistics: Probability
 
Conditional Probability
Conditional ProbabilityConditional Probability
Conditional Probability
 
Probability concept and Probability distribution
Probability concept and Probability distributionProbability concept and Probability distribution
Probability concept and Probability distribution
 
Basic probability Concepts and its application By Khubaib Raza
Basic probability Concepts and its application By Khubaib RazaBasic probability Concepts and its application By Khubaib Raza
Basic probability Concepts and its application By Khubaib Raza
 
Bayes Theorem
Bayes TheoremBayes Theorem
Bayes Theorem
 
Basic concepts of probability
Basic concepts of probability Basic concepts of probability
Basic concepts of probability
 
Probability
ProbabilityProbability
Probability
 
Probability Distribution
Probability DistributionProbability Distribution
Probability Distribution
 
03+probability+distributions.ppt
03+probability+distributions.ppt03+probability+distributions.ppt
03+probability+distributions.ppt
 
Probability
ProbabilityProbability
Probability
 
Intro to probability
Intro to probabilityIntro to probability
Intro to probability
 
Theoretical probability distributions
Theoretical probability distributionsTheoretical probability distributions
Theoretical probability distributions
 

Similar to Probability Theory

Stat lesson 4.2 rules of computing probability
Stat lesson 4.2 rules of computing probabilityStat lesson 4.2 rules of computing probability
Stat lesson 4.2 rules of computing probability
pipamutuc
 
Unit 4--probability and probability distribution (1).pptx
Unit 4--probability and probability distribution (1).pptxUnit 4--probability and probability distribution (1).pptx
Unit 4--probability and probability distribution (1).pptx
akshay353895
 

Similar to Probability Theory (20)

Addition rule and multiplication rule
Addition rule and multiplication rule  Addition rule and multiplication rule
Addition rule and multiplication rule
 
Conditional Probability
Conditional ProbabilityConditional Probability
Conditional Probability
 
probability
probabilityprobability
probability
 
Stat lesson 4.2 rules of computing probability
Stat lesson 4.2 rules of computing probabilityStat lesson 4.2 rules of computing probability
Stat lesson 4.2 rules of computing probability
 
Day 3.pptx
Day 3.pptxDay 3.pptx
Day 3.pptx
 
Basic Concepts of Probability
Basic Concepts of ProbabilityBasic Concepts of Probability
Basic Concepts of Probability
 
Probability concepts for Data Analytics
Probability concepts for Data AnalyticsProbability concepts for Data Analytics
Probability concepts for Data Analytics
 
Reliability-Engineering.pdf
Reliability-Engineering.pdfReliability-Engineering.pdf
Reliability-Engineering.pdf
 
Probability models & basic rules
Probability models & basic rulesProbability models & basic rules
Probability models & basic rules
 
Stats chapter 6
Stats chapter 6Stats chapter 6
Stats chapter 6
 
Probabilty1.pptx
Probabilty1.pptxProbabilty1.pptx
Probabilty1.pptx
 
Probability & Bayesian Theorem
Probability & Bayesian TheoremProbability & Bayesian Theorem
Probability & Bayesian Theorem
 
1-Probability-Conditional-Bayes.pdf
1-Probability-Conditional-Bayes.pdf1-Probability-Conditional-Bayes.pdf
1-Probability-Conditional-Bayes.pdf
 
3.2 probablity
3.2 probablity3.2 probablity
3.2 probablity
 
Probability revised
Probability revisedProbability revised
Probability revised
 
Probabilidad 2020 2 v2
Probabilidad 2020 2 v2Probabilidad 2020 2 v2
Probabilidad 2020 2 v2
 
BHARAT & KAJAL.pptx
BHARAT & KAJAL.pptxBHARAT & KAJAL.pptx
BHARAT & KAJAL.pptx
 
Unit 4--probability and probability distribution (1).pptx
Unit 4--probability and probability distribution (1).pptxUnit 4--probability and probability distribution (1).pptx
Unit 4--probability and probability distribution (1).pptx
 
Complements conditional probability bayes theorem
Complements  conditional probability bayes theorem  Complements  conditional probability bayes theorem
Complements conditional probability bayes theorem
 
Probabilities distributions
Probabilities distributionsProbabilities distributions
Probabilities distributions
 

More from Anthony J. Evans

More from Anthony J. Evans (16)

Time Series
Time SeriesTime Series
Time Series
 
The Suitcase Case
The Suitcase CaseThe Suitcase Case
The Suitcase Case
 
Correlation
Correlation Correlation
Correlation
 
Nonparametric Statistics
Nonparametric StatisticsNonparametric Statistics
Nonparametric Statistics
 
Student's T Test
Student's T TestStudent's T Test
Student's T Test
 
Significance Tests
Significance TestsSignificance Tests
Significance Tests
 
Taxi for Professor Evans
Taxi for Professor EvansTaxi for Professor Evans
Taxi for Professor Evans
 
Inferential Statistics
Inferential StatisticsInferential Statistics
Inferential Statistics
 
Probability Distributions
Probability Distributions Probability Distributions
Probability Distributions
 
Descriptive Statistics
Descriptive StatisticsDescriptive Statistics
Descriptive Statistics
 
Statistical Literacy
Statistical Literacy Statistical Literacy
Statistical Literacy
 
Quantitative Methods
Quantitative Methods Quantitative Methods
Quantitative Methods
 
Collecting and Presenting Data
Collecting and Presenting DataCollecting and Presenting Data
Collecting and Presenting Data
 
Numeracy Skills 1
Numeracy Skills 1Numeracy Skills 1
Numeracy Skills 1
 
The Dynamic AD AS Model
The Dynamic AD AS ModelThe Dynamic AD AS Model
The Dynamic AD AS Model
 
Numeracy Skills 2
Numeracy Skills 2Numeracy Skills 2
Numeracy Skills 2
 

Recently uploaded

原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
pwgnohujw
 
一比一原版麦考瑞大学毕业证成绩单如何办理
一比一原版麦考瑞大学毕业证成绩单如何办理一比一原版麦考瑞大学毕业证成绩单如何办理
一比一原版麦考瑞大学毕业证成绩单如何办理
cyebo
 
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
zifhagzkk
 
一比一原版纽卡斯尔大学毕业证成绩单如何办理
一比一原版纽卡斯尔大学毕业证成绩单如何办理一比一原版纽卡斯尔大学毕业证成绩单如何办理
一比一原版纽卡斯尔大学毕业证成绩单如何办理
cyebo
 
一比一原版阿德莱德大学毕业证成绩单如何办理
一比一原版阿德莱德大学毕业证成绩单如何办理一比一原版阿德莱德大学毕业证成绩单如何办理
一比一原版阿德莱德大学毕业证成绩单如何办理
pyhepag
 
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
ju0dztxtn
 
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
Amil baba
 
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotecAbortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh +966572737505 get cytotec
 
Audience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptxAudience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptx
Stephen266013
 
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
pyhepag
 

Recently uploaded (20)

原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
 
一比一原版麦考瑞大学毕业证成绩单如何办理
一比一原版麦考瑞大学毕业证成绩单如何办理一比一原版麦考瑞大学毕业证成绩单如何办理
一比一原版麦考瑞大学毕业证成绩单如何办理
 
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
如何办理(Dalhousie毕业证书)达尔豪斯大学毕业证成绩单留信学历认证
 
Predictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting TechniquesPredictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting Techniques
 
Atlantic Grupa Case Study (Mintec Data AI)
Atlantic Grupa Case Study (Mintec Data AI)Atlantic Grupa Case Study (Mintec Data AI)
Atlantic Grupa Case Study (Mintec Data AI)
 
一比一原版纽卡斯尔大学毕业证成绩单如何办理
一比一原版纽卡斯尔大学毕业证成绩单如何办理一比一原版纽卡斯尔大学毕业证成绩单如何办理
一比一原版纽卡斯尔大学毕业证成绩单如何办理
 
Sensing the Future: Anomaly Detection and Event Prediction in Sensor Networks
Sensing the Future: Anomaly Detection and Event Prediction in Sensor NetworksSensing the Future: Anomaly Detection and Event Prediction in Sensor Networks
Sensing the Future: Anomaly Detection and Event Prediction in Sensor Networks
 
What is Insertion Sort. Its basic information
What is Insertion Sort. Its basic informationWhat is Insertion Sort. Its basic information
What is Insertion Sort. Its basic information
 
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...
 
一比一原版阿德莱德大学毕业证成绩单如何办理
一比一原版阿德莱德大学毕业证成绩单如何办理一比一原版阿德莱德大学毕业证成绩单如何办理
一比一原版阿德莱德大学毕业证成绩单如何办理
 
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
 
The Significance of Transliteration Enhancing
The Significance of Transliteration EnhancingThe Significance of Transliteration Enhancing
The Significance of Transliteration Enhancing
 
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
 
Northern New England Tableau User Group (TUG) May 2024
Northern New England Tableau User Group (TUG) May 2024Northern New England Tableau User Group (TUG) May 2024
Northern New England Tableau User Group (TUG) May 2024
 
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
NO1 Best Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialist I...
 
MATERI MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI  MANAJEMEN OF PENYAKIT TETANUS.pptMATERI  MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI MANAJEMEN OF PENYAKIT TETANUS.ppt
 
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotecAbortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
 
Audience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptxAudience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptx
 
Digital Marketing Demystified: Expert Tips from Samantha Rae Coolbeth
Digital Marketing Demystified: Expert Tips from Samantha Rae CoolbethDigital Marketing Demystified: Expert Tips from Samantha Rae Coolbeth
Digital Marketing Demystified: Expert Tips from Samantha Rae Coolbeth
 
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
一比一原版加利福尼亚大学尔湾分校毕业证成绩单如何办理
 

Probability Theory

  • 1. Probability Theory Anthony J. Evans Professor of Economics, ESCP Europe www.anthonyjevans.com (cc) Anthony J. Evans 2019 | http://creativecommons.org/licenses/by-nc-sa/3.0/
  • 2. Contents • Measuring probability • Combining probabilities • Bayes theorem • Factorials • Permutations • Combinations 2
  • 3. Probability • Def: “The branch of mathematics that describes the pattern of chance outcomes”* • A number between 0 and 1 that represents the chance that an event will occur 0 1 Impossible Certain Risk *Moore & McCabe (1999), p. 288 Probability Increasing likelihood of occurrence Uncertainty 3
  • 4. Measuring Probability Number of times the event (E ) can occur P(E) = Total number of possible outcomes Classical Relative Frequency • Assigning probabilities based on the assumption of equally likely outcomes Number of times the event (E ) actually occurs P(E) = Total number of observed outcomes • Assigning probabilities based on experimentation or historical data. 4
  • 5. Measuring Probability: Classical Method • What is the probability of rolling a 2 on a fair dice? • Possible outcomes: [1,2,3,4,5,6]; n=6 P(2) = 1/6 5
  • 6. Measuring Probability: Relative Frequency Method • Lucas Tool Rental would like to assign probabilities to the number of floor polishers it rents per day • Office records show the following frequencies of daily rentals for the last 40 days: Number of Number polishers rented of days Probability 0 4 .10 = 4/40 1 6 .15 = 6/40 2 18 .45 etc. 3 10 .25 4 2 .05 40 1.00 6
  • 7. Consider the following possible event: a) Picking the Ace of Spades from a full deck of cards • What is P(A)? 1/52 7 Sample Space Event A
  • 8. Complement events • The complement (~A or AC) of event A is defined as the event consisting of all sample points that are not in A a) Picking the Ace of Spades from a full deck of cards • What is P(AC)? = 51/52 8 Sample Space Event A AC
  • 9. Union probability • Two events are said to be mutually exclusive if they cannot occur at the same time – i.e. the events have no sample points in common • The probability that one of several mutually exclusive events occurs, is simply the sum of the individual probabilities – “OR” means “+” • P(AUB) = P(A) + P(B) 9 Sample Space Event A Event B )( BAP È
  • 10. Union probability - example a) Rolling a die and getting a 4 b) Rolling a die and getting a 5 • P(AUB) = 1/6 + 1/6 = 2/6 = 1/3 10 Sample Space Event A Event B )( BAP È
  • 11. Joint probability • Two events are said to be independent if the outcomes have no effect on each other • The probability that two independent events occur is simply the multiplication of the individual probabilities – “AND” means “X” • P(A∩B) = P(A) x P(B) 11 )( BAP Ç Sample Space Event A Event B
  • 12. Joint probability - example a) Picking any Spade from a full deck of cards b) Rolling a die and getting a 3 • P(A∩B) = 1/4 x 1/6 = 1/24 12 )( BAP Ç Sample Space Event A Event B
  • 13. KRUPNIK INVESTMENT • Analysts for Krupnik Investments have assigned probabilities for six scenarios, and the respective profitability of portfolios A and B • Note: there’s no way to verify the assigned probabilities per outcome. 13
  • 14. KRUPNIK INVESTMENT: Getting Started • Event A: Portfolio A is profitable P(A) = Outcomes 1, 2, 3 or 6 = 0.3 + 0.06 + 0.2 + 0.14 = 70% • Event B: Portfolio B is profitable P(B) = Outcomes 1, 3, 5 or 6 = 0.3 + 0.2 + 0.24 + 0.14 = 88% 14
  • 15. KRUPNIK INVESTMENT: Complement • What is the probability of 5C? • P(5c ) = 76% 15
  • 16. KRUPNIK INVESTMENT: A more complicated example • Are Events A and B mutually exclusive? – No: it is possible for a sample point to be part of both Event A and Event B (e.g. Outcome 1) • Are Events A and B independent? – No: the profitability of A may well influence the profitability of B – We can’t know for sure, so assume they are not independent and then later on we can test for this to become certain 16
  • 17. KRUPNIK INVESTMENT: Union • What’s the likelihood that either A or B are profitable? • What is the probability of the union of Event A and Event B? = Outcome 1, 2, 3, 5 or 6 = 0.3 + 0.06 + 0.2 + 0.24 + 0.14 = 94% By inspection )( BAP È 17
  • 18. KRUPNIK INVESTMENT: Joint • What’s the likelihood that both A and B are profitable? • What is the joint probability of Event A and Event B? = Outcomes 1, 3, 6 = 0.3 + 0.2 + 0.14 = 64% By inspection 18 )( BAP Ç
  • 19. KRUPNIK INVESTMENT: Union • If the events are not mutually exclusive, they overlap • The union of events A and B is the event containing all sample points that are in A or B or both • What is the likelihood that either A or B occur? – The addition law: – P(AUB) = P(A) + P(B) – P(A ∩ B) 19 Sample Space Event A Event B )( BAP È )( BAP Ç We need to avoid double counting
  • 20. KRUPNIK INVESTMENT: Union • What’s the likelihood that either A or B are profitable? • What is the probability of the union of Event A and Event B? • P(AUB) = P(A) + P(B) – P(A ∩ B) = 0.7 + 0.88 – 0.64 = 94% Addition Law )( BAP È 20
  • 21. KRUPNIK INVESTMENT: Conditional • The conditional probability of event A wrt B is the set of all sample points that are in both A and B • The conditional is denoted by A|B • P(A|B)= P(A∩B) / P(B) 21 )|( BAP Sample Space Event B Conditional
  • 22. KRUPNIK INVESTMENT: Conditional • What’s the likelihood that A occurs, given B occurring? • What is the conditional probability of Event A wrt Event B? – It’s the ratio of (A∩B) wrt P(B) • P(A|B) = P(A∩B) / P(B) = 0.64 / 0.88 = 73% 22 )|( BAP
  • 23. KRUPNIK INVESTMENT: Conditional • What’s the likelihood that B occurs, given A occurring? • What is the conditional probability of Event B wrt Event A? – It’s the ratio of (A∩B) wrt P(B) • P(B|A) = P(A∩B) / P(A) = 0.64 / 0.70 = 91% 23 € P(B | A)
  • 24. KRUPNIK INVESTMENT: Joint • If the events are not independent, they overlap • The intersection of events A and B is the set of all sample points that are in both A and B • The intersection is denoted by (A∩B) • What’s the likelihood that both A and B occur? – The multiplication law: – P(A∩B) = P(A) x P(B|A) 24 )( BAP Ç Sample Space Event A Event B )( BAP Ç
  • 25. KRUPNIK INVESTMENT: Joint • What’s the likelihood that both A and B are profitable? • What is the joint probability of Event A and Event B? • P(A∩B) = P(A) x P(B|A) = 0.70 x (0.914) = 64% Multiplication Law 25 )( BAP Ç
  • 26. Union … mutually exclusive? … independent? Relationship between A and B NO YES YES NO • P(AUB) = P(A) + P(B) • P(AUB) = P(A) + P(B) – P(A ∩ B) • P(A∩B) = P(A) x P(B) • P(A∩B) = P(A) x P(B|A) Are events A and B… Formula …A or B happening …A and B happening Combining probabilities: Rules Joint 26
  • 27. Marginal The probability of A occurring Union The probability of A or B occurring Joint The probability of A and B occurring Conditional The probability of A occurring given that B has occurred BA BA BA )(AP )( BAP È )( BAP Ç )|( BAP Combining probabilities: Summary 27
  • 28. Conditional Probability: Defective Machine Example • When defective items are found, it is not immediately possible to say from which machine they came • We can use conditional probability to identify the machine which is most likely to have produced a defective item A company uses three machines, labelled A,B and C • 40% of production goes through A, 50% to B, and 10% to C • The wastage rate for machine A is 6%, 1% for machine B, and 10% for machine C • What is the probability that a piece of waste has come from machine A? 28 BAThis is saying “what’s the probability that there is a piece of waste from machine A” B This is saying “what’s the probability that this piece of waste came from machine A”
  • 29. Conditional Probability: Defective Machine Example 0.10 0.50 0.10 Acceptable Defective Acceptable Acceptable Defective Defective 0.90 0.01 0.99 0.06 0.40 0.94 Production process A B C Machine 0.4*0.94 = 0.376 Joint probability 0.4*0.06 = 0.024 0.5*0.99 = 0.495 0.5*0.01 = 0.005 0.1*0.90 = 0.09 0.1*0.10 = 0.01 29
  • 30. Conditional Probability: Defective Machine Example What’s the probability that a defective product will come out of the production line? Given that the product is defective, what’s the probability that it came out of machine A? )( )( )|( machineanyfromdefectiveP AmachinefromdefectiveP defectiveAP = Problem Solution P(defective)=0.024+0.005+0.010 = 0.039 039.0 024.0 = %62= 30 P(A|B) = P(A∩B) / P(B)
  • 31. Conditional Probability: Defective Machine Example Given that the product is defective, what’s the probability that it came out of machine B? )( )( )|( machineanyfromdefectiveP BmachinefromdefectiveP defectiveBP = 039.0 005.0 = %13= Given that the product is defective, what’s the probability that it came out of machine C? )( )( )|( machineanyfromdefectiveP CmachinefromdefectiveP defectiveCP = 039.0 010.0 = = 26% 31
  • 32. Probability theory and the dating market • Suppose you are told you must marry, and that you must choose your spouse out of 100 applicants. You may interview each applicant once. After each interview you must decide whether to marry that person. If you decline, you lose the opportunity forever. If you work your way through 99 applicants without choosing one, you must marry the 100th • If you interview half the potential partners then stop at the next best one - that is, the first one better than the best person you've already interviewed - you will marry the very best candidate about 25 per cent of the time. Once again, probability explains why. A quarter of the time, the second best partner will be in the first 50 people and the very best in the second. So 25 per cent of the time, the rule "stop at the next best one" will see you marrying the best candidate. Much of the rest of the time, you will end up marrying the 100th person, who has a 1 in 100 chance of being the worst, but hey, this is probability, not certainty. • John Gilbert and Frederick Mosteller of Harvard University proved that you could raise your odds to 37 per cent by interviewing 37 people then stopping at the next best. The number 37 comes from dividing 100 by e, the base of the natural logarithms, which is roughly equal to 2.72. Gilbert and Mosteller's law works no matter how many candidates there are - you simply divide the number of options by e 32“What's luck got to do with it? The math of gambling”, Helen Thomson, New Scientist, 11 August 2009
  • 33. The Monty Hall problem • Suppose the contestants on a game show are given the choice of three doors: Behind one door is a car; behind the others, goats. After a contestant picks a door, the host, who knows what’s behind all the doors, opens one of the unchosen doors, which reveals a goat. He then says to the contestant, “Do you want to switch to the other unopened door?” • Should the contestant switch? 33See Mlodinow, 2008, p.43 & 53 Applet: http://math.ucsd.edu/~crypto/Monty/monty.html
  • 34. The Monty Hall problem (2) 34Text taken from Oxford University Entrance Exam, “Economics and Management” May 2000
  • 35. Bayes Theorem • Bayes Theorem is a way to calculate conditional probability • It allows us to update original probabilities in light of new data New Information Application of Bayes’ Theorem Posterior Probabilities Prior Probabilities 35
  • 36. Bayes Theorem • P(A) = probability of A • P(A|B) = probability of A given B • P(~A) = probability of not A [ ] [ ])(~)|~()()|( )()|( )|( APABPAPABP APABP BAP ×+× × = 36
  • 37. Bayes Theorem: Mammograms • 1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammograms. 9.6% of women without breast cancer will also get positive mammograms. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer? 37This example is from Yudowsky but the explanation and visualisation is from https://oscarbonilla.com/2009/05/visualizing-bayes-theorem/ A = Women with cancer Consider all women B = Positive mammogram (Need to cover 80% of event A and 9.6% of Ac) Most positive results do not have cancer
  • 38. Bayes Theorem: Mammograms • 1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammograms. 9.6% of women without breast cancer will also get positive mammograms. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer? 38This example is from Yudowsky but the explanation and visualisation is from https://oscarbonilla.com/2009/05/visualizing-bayes-theorem/ P(A)=0.01 0.80=P(B|A) [ ] [ ])(~)|~()()|( )()|( )|( APABPAPABP APABP BAP ×+× × = 𝑃 𝐴 𝐵 = 0.80×0.01 [0.80×0.01] + [0.096×0.99] 0.096=P(B|~A) 𝑃 𝐴 𝐵 = 7.76%
  • 39. Application: mammograms • For every 2,000 mammograms – 1 life saved – 200 false positives, which lead to 10 painful and unnecessary surgeries • ~ 84% success rate • Younger women are more likely to get a false positive – Recommended age for having one has risen as a result • If you’re between 40-50 years old there’s a 50%-60% chance of a false positive 39https://www.stuffyoushouldknow.com/podcasts/what-are-false-positives.htm
  • 40. Bayes Theorem: Drug Testing Example • A certain drug test is 99% accurate, that is, the test will correctly identify a drug user as testing positive 99% of the time, and will correctly identify a non- user as testing negative 99% of the time. • This would seem to be a relatively accurate test, but Bayes' theorem will reveal a potential flaw. • Let's assume a corporation decides to test its employees for opium use, and 0.5% of the employees use the drug. • We want to know the probability that, given a positive drug test, an employee is actually a drug user. Let “A" be the event of being a drug user and “~A" indicate being a non-user • Let “B" be the event of a positive drug test Source: http://en.wikipedia.org/wiki/Bayes'_theorem 40
  • 41. Bayes Theorem: Drug Testing Example We need to know the following: • P(A), or the probability that the employee is a drug user, regardless of any other information. This is 0.005, since 0.5% of the employees are drug users. • P(~A), or the probability that the employee is not a drug user. This is 1-Pr(A), or 0.995. • P(B|A), or the probability that the test is positive, given that the employee is a drug user. This is 0.99, since the test is 99% accurate. • P(B|~A), or the probability that the test is positive, given that the employee is not a drug user. This is 0.01, since the test will produce a false positive for 1% of non-users. 41
  • 42. Bayes Theorem: Drug Testing Example • Compute the probability that an employee that tested positive is actually a drug user: • The probability that the employee is actually a drug user is only about 33%. • The rarer the condition for which we are testing, the greater percentage of the positive tests will be false positives • This illustrates why it is important to do follow-up tests [ ] [ ]995.001.0005.099.0 005.099.0 )|( ´+´ ´ =BAP 00995.000495.0 00495.0 )|( + =BAP 3322.0)|( =BAP 42
  • 43. False positives Does the person have the disease? Yes No What is the test result? Positive True positive False positive Negative False negative True negative 43
  • 44. Factorials The factorial of a number (n) is the product of all positive integers less than or equal to n (By definition 0!=1) Example: What’s 10! 10! = 10*9*8*7*6*5*4*3*2*1 = 3,628,800 n1)3...(n21=!n ⋅−⋅⋅ 44
  • 45. Permutations A permutation is an ordered sequence • A possible permutation of the number sequence [1-6] is – 3,5,2,1,6,4 • Number of permutations of – n different things – k at a time • Used when order matters )!( ! kn n Pk n − = 45
  • 46. Permutations: Example • What are the odds of getting 6 numbers (between 1-49) in the right order? 520,347,068,10= )!649( !49 6 49 - =P 46
  • 47. Combinations A combination is an un-ordered collection of unique elements •Number of combinations of – n things – k at a time •Used when order doesn't matters – In permutations abc is different from bca – In combinations we are concerned only that a, b, and c have been selected • abc and bca are the same combination )!(! ! knk n k n Ck n - =÷ ø ö ç è æ = 47
  • 48. Combinations: Example • You have 5 shirts, but you will select only 3 for your vacation. In how many different ways can you do this? )!35(!3 !5 3 5 3 5 - =÷ ø ö ç è æ =C 103 5 =C 48
  • 49. Combinations: Example What are the odds of winning the UK Lotto? • Need to get 6 numbers (between 1-49) in any order )!649(!6 !49 6 49 - =C mC 146 49 = 49
  • 50. Summary • There’s a difference between risk and uncertainty – quantities techniques focus purely on risk • Probability can be difficult to get your head around, so keep things intuitive and do sanity tests • Casinos are a good place for revision 50
  • 52. Measuring Probability: Classical Method • What is the probability of rolling a 2 on a fair dice? • Possible outcomes: [1,2,3,4,5,6]; n=6 P(2) = 1/6 52
  • 53. Consider the following possible event: a) Picking the Ace of Spades from a full deck of cards • What is P(A)? 1/52 53 Sample Space Event A
  • 54. Complement events • The complement (~A or AC) of event A is defined as the event consisting of all sample points that are not in A a) Picking the Ace of Spades from a full deck of cards • What is P(AC)? = 51/52 54 Sample Space Event A AC
  • 55. Union probability - example a) Rolling a die and getting a 4 b) Rolling a die and getting a 5 • P(AUB) = 1/6 + 1/6 = 2/6 = 1/3 55 Sample Space Event A Event B )( BAP È
  • 56. Joint probability - example a) Picking any Spade from a full deck of cards b) Rolling a die and getting a 3 • P(A∩B) = 1/4 x 1/6 = 1/24 56 )( BAP Ç Sample Space Event A Event B
  • 57. KRUPNIK INVESTMENT: Getting Started • Event A: Portfolio A is profitable P(A) = Outcomes 1, 2, 3 or 6 = 0.3 + 0.06 + 0.2 + 0.14 = 70% • Event B: Portfolio B is profitable P(B) = Outcomes 1, 3, 5 or 6 = 0.3 + 0.2 + 0.24 + 0.14 = 88% 57
  • 58. KRUPNIK INVESTMENT: Complement • What is the probability of 5C? • P(5c ) = 76% 58
  • 59. KRUPNIK INVESTMENT: Union • What’s the likelihood that either A or B are profitable? • What is the probability of the union of Event A and Event B? = Outcome 1, 2, 3, 5 or 6 = 0.3 + 0.06 + 0.2 + 0.24 + 0.14 = 94% By inspection )( BAP È 59
  • 60. KRUPNIK INVESTMENT: Joint • What’s the likelihood that both A and B are profitable? • What is the joint probability of Event A and Event B? = Outcomes 1, 3, 6 = 0.3 + 0.2 + 0.14 = 64% By inspection 60 )( BAP Ç
  • 61. KRUPNIK INVESTMENT: Union • What’s the likelihood that either A or B are profitable? • What is the probability of the union of Event A and Event B? • P(AUB) = P(A) + P(B) – P(A ∩ B) = 0.70 + 0.88 – 0.64 = 94% Addition Law )( BAP È 61
  • 62. KRUPNIK INVESTMENT: Conditional • What’s the likelihood that A occurs, given B occurring? • What is the conditional probability of Event A wrt Event B? – It’s the ratio of (A∩B) wrt P(B) • P(A|B) = P(A∩B) / P(B) = 0.64 / 0.88 = 73% 62 )|( BAP
  • 63. KRUPNIK INVESTMENT: Conditional • What’s the likelihood that B occurs, given A occurring? • What is the conditional probability of Event B wrt Event A? – It’s the ratio of (A∩B) wrt P(B) • P(B|A) = P(A∩B) / P(A) = 0.64 / 0.70 = 91% 63 € P(B | A)
  • 64. KRUPNIK INVESTMENT: Joint • What’s the likelihood that both A and B are profitable? • What is the joint probability of Event A and Event B? • P(A∩B) = P(A) x P(B|A) = 0.70 x (0.914) = 64% Multiplication Law 64 )( BAP Ç
  • 65. Conditional Probability: Defective Machine Example Given that the product is defective, what’s the probability that it came out of machine B? )( )( )|( machineanyfromdefectiveP BmachinefromdefectiveP defectiveBP = 039.0 005.0 = %13= Given that the product is defective, what’s the probability that it came out of machine C? )( )( )|( machineanyfromdefectiveP CmachinefromdefectiveP defectiveCP = 039.0 010.0 = = 26% 65
  • 66. Bayes Theorem: Mammograms • 1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammograms. 9.6% of women without breast cancer will also get positive mammograms. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer? 66This example is from Yudowsky but the explanation and visualisation is from https://oscarbonilla.com/2009/05/visualizing-bayes-theorem/ P(A)=0.01 0.80=P(B|A) [ ] [ ])(~)|~()()|( )()|( )|( APABPAPABP APABP BAP ×+× × = 𝑃 𝐴 𝐵 = 0.80×0.01 [0.80×0.01] + [0.096×0.99] 0.096=P(B|~A) 𝑃 𝐴 𝐵 = 7.76%
  • 67. Factorials The factorial of a number (n) is the product of all positive integers less than or equal to n (By definition 0!=1) Example: What’s 10! 10! = 10*9*8*7*6*5*4*3*2*1 = 3,628,800 n1)3...(n21=!n ⋅−⋅⋅ 67
  • 68. Permutations: Example • What are the odds of getting 6 numbers (between 1-49) in the right order? 520,347,068,10= )!649( !49 6 49 - =P 68
  • 69. Combinations: Example • You have 5 shirts, but you will select only 3 for your vacation. In how many different ways can you do this? )!35(!3 !5 3 5 3 5 - =÷ ø ö ç è æ =C 103 5 =C 69
  • 70. Combinations: Example What are the odds of winning the UK Lotto? • Need to get 6 numbers (between 1-49) in any order )!649(!6 !49 6 49 - =C mC 146 49 = 70
  • 71. • This presentation forms part of a free, online course on analytics • http://econ.anthonyjevans.com/courses/analytics/ 71