aryuty

1. Lecture (5)
Conditional Probability,
Bayes Theorem and
Independent Events with Examples
University of Baghdad ,
Collage of Sciences,
Department of Computer Sciences
Diploma Class Syllabus Academic
First Semester
Dr. Ali H. Kashmar
Year 2019/2020

2. Topics
• Conditional Probability
• Independent Events
• Multiplication Rule of Probability
• Total probability rule
• Bayes rule
• Pairwise Independent
• Mutually or complete Independent
• Probability Distributions for Discrete Random
Variables
• Examples

3. The probability of an event based on the fact that some other event has
occurred, will occur, or is occurring.
P(B/A) =
Conditional Probability
The probability of event B occurring given that event A has occurred is
usually stated as “the conditional probability of B, given A; P(B/A)
Conditional Probability


)
(
)
(
A
P
B
A
P
)
(
)
(
A
P
B
and
A
P

4. Example: 1
A number from the sample space S = {2, 3, 4, 5, 6, 7, 8, 9} is randomly
selected. Given the defined events A and B,
A: selected number is odd, and
B: selected number is a multiple of 3
Find the following probabilities.
a) B = {3, 6, 9}
Conditional Probability
b) P(A and B) =
a) P(B) b) P(A and B) c) P(B/A)
Conditional Probability
P(B) = 3/8
P({3, 5, 7, 9}  {3, 6, 9})
= P({3, 9}) = 2/8 = 1/4
c) Probability of B given A has occurred:
P(B/A) =
P(A)
P(A and B)
4/8
1/4
1/2
= =

5. Example: 2
Given a family with two children, find the probability that both are boys,
given that at least one is a boy.
P({gb, bg, bb}) =
Conditional Probability
P(A and B) =
A = at least one boy
Conditional Probability
P(A) =
3/4
P({gb, bg, bb}  {bb}) = P({bb}) =
= 1/3
1/4
Conditional Probability P(B/A) =
3/4
1/4
B = both are boys
S= {gg, gb, bg, bb}
A = {gb, bg, bb}=3/4
B = {bb}=1/4
)
(
)
(
A
P
B
and
A
P
)
(
)
(
A
P
B
and
A
P
=

6. Two events are Independent if the occurrence of one of them has no
effect on the probability of the other.
Theorem: 1
If A and B are independent events, then P(B/A) = P(B) and
P(A/B) = P(A)
Where P(A) > 0 and P(B) > 0
P(B/A) = P(B) need to proof?
Independent Events
or
Independent Events
P(A/B) = P(A) need to proof?

7. Theorem:2
If A and B are independent events, then A and 𝐵𝑐
, B
and 𝐴𝑐
are independent.
Proof: Homework

8. Example: 3
A single card is randomly selected from a standard 52-card deck. Given
the defined events A and B,
A: the selected card is an ace, B: the selected card is red,
find the following probabilities.
a) P(B) =
Independent Events
b) P(A and B) =
a) P(B) b) P(A and B) c) P(B/A)
Independent Events
= 1/2
P({Ah, Ad, Ac, As}  {all red}) = P({Ah, Ad}) = 2/52
Events A and B are independent as P(B) = P(B/A).
c) P(B/A) =
P(A)
P(A and B)
4/52
2/52
1/2
= =
52
26

9. Example 4
X, Y, Z, H four students,
• P(X|Y)= prob. Of X is success given Y is success too
• P(X|Y)=P(X)
• Thus: P(A|B)=P(A)
• P(A∩B)=P(A).P(B)
• A and B are independent events

10. Example 5
Two coins be tossed, let
A={H in the 2nd throw}
B={ H in the 1st throw}, Are A and B independent events?
Sol:
Ω={HH,HT,TH,TT}
A={HH,TH} P(A)=2/4=1/2
B={HH,HT} P(B)=2/4=1/2
A∩B={HH] p(A∩B)=1/4
P(A|B)=
p(A∩B)
𝑃(𝐵)
=
1/4
1/2
= 1/2 i.e P(A|B)= P(A)
P(A).P(B)=(1/2).(1/2)=1/4
Then : A and B are independent events

11. If A and B are any two events then
P(A and B) = P(A)  P(B/A)
Multiplication Rule of Probability
Multiplication Rule of Probability
P(A and B) = P(A)  P(B)
If A and B are independent events then
A jar contains 4 red marbles, 3 blue marbles, and 2 yellow marbles.
What is the probability that a red marble is selected and then a blue one
without replacement?
P(Red and Blue) = P(Red)  P(Blue/Red)
= 4/9  3/8
= 12/72
= 1/8 = 0.1667
Example:6

12. Multiplication Rule of Probability
Multiplication Rule of Probability
A jar contains 4 red marbles, 3 blue marbles, and 2 yellow marbles.
What is the probability that a red marble is selected and then a blue one
with replacement?
P(Red and Blue) = P(Red)  P(Blue)
= 4/9  3/9
= 12/81
= 4/27 = 0.148
Example:7

13. The Law of Total Probability
P(A) = P(A  S1) + P(A  S2) + … + P(A  Sk)
= P(S1)P(A|S1) + P(S2)P(A|S2) + … + P(Sk)P(A|Sk)
Let S1 , S2 , S3 ,..., Sk be mutually exclusive and
exhaustive events (that is, one and only one
must happen). Then the probability of any event
A can be written as

14. The Law of Total Probability
A
A Sk
A  S1
S2….
S1
Sk
P(A) = P(A  S1) + P(A  S2) + … + P(A  Sk)
= P(S1)P(A|S1) + P(S2)P(A|S2) + … + P(Sk)P(A|Sk)
= 𝒊=𝟏
𝒌
𝑷 𝑺𝒊 𝑷(𝑨|𝑺𝒊)

15. Homework
1). we have 3 urns (U1, U2, U3)
U1={1w,2b}, U2={1b,2w}, U3={3b,3w}
• A dice is rolled:
– If 1, 2, 3, U1 is selected
– If 4, U2 is selected
– If 5,6 U3 is selected
• A={choose one white ball} Find p(A)?
• B={2 balls are selected, one white and one block}
Find p(B)?

16. 2) U1={1w, 2b, 3r}, U2={2w, 1b, 1r}, U3={3w, 3b, 2r}
3-balls are selected.
2-dice are rolled.
Let x be the no. of the first dice
Let y be the no. of the second dice
If x>y U1 is selected
If x=y U2 is selected
If x<y U3 is selected
A={3-balls are selected and at least one red}.
B={3-balls are selected and at most 1-red and 1-block}.
C={3-balls are selected and one ball from each kind}.
Find: p(A),p(A∩B), p(A∩B ∩ C), p(B),p(A∩ C), p(B ∩ C) and p(C)

17. Bayes’ rule
,...
2
,
1
)
|
(
)
(
)
|
(
)
(
)
|
(
1





j
where
H
B
P
H
P
H
B
P
H
P
B
H
P
j
j
j
j
j
j
• Theorem:
Let {𝐻𝑛} be a disjoint sequences of events, such
that P( 𝐻𝑛 ) > 0 for n=1,2,… and 𝐻𝑛 = Ω , Let
B∈ 𝑆 with P(𝐵) > 0 , then:

18. Bayes’ Rule: Proof
can be re-arranged to:
)
(
)
(
)
|
(
j
j
j
H
P
B
H
P
H
B
P


)
|
(
)
(
)
(
1




j
j
j H
B
P
H
P
B
P
)
(
)
(
)
|
(
B
P
B
H
P
B
H
P
j
j


and, by using total probability def.
We define:
)
|
(
).
(
)
( B
H
P
B
P
B
H
P j
j 
 )
|
(
).
(
)
( j
j
j H
B
P
H
P
B
H
P 

)
(
)
|
(
).
(
)
(
)
(
)
|
(
B
P
H
B
P
H
P
B
P
B
H
P
B
H
P
j
j
j
j 



19. Bayes’ Rule:
p(𝐻𝑗): called prior probability
p(𝐻𝑗|B): called posturer probability
)
|
(
)
(
)
|
(
)
(
)
|
(
1
j
j
j
j
j
j
H
B
P
H
P
H
B
P
H
P
B
H
P






20. Bayes’ Rule:
• Why do we care?
• Why is Bayes’ Rule useful?
• It turns out that sometimes it is very useful to
be able to “flip” conditional probabilities.
That is, we may know the probability of A
given B, but the probability of B given A may
not be obvious. An example will help…

21. We know:
P(F) =
P(M) =
P(H|F) =
P(H|M) =
Example 8
From a previous information, we know that 49% of the
population are female. Of the female patients, 8% are
high risk for heart attack, while 12% of the male patients
are high risk. A single person is selected at random and
found to be high risk. What is the probability that it is a
male? Define H: high risk F: female M: male
61
.
)
08
(.
49
.
)
12
(.
51
.
)
12
(.
51
.
)
|
(
)
(
)
|
(
)
(
)
|
(
)
(
)
|
(





F
H
P
F
P
M
H
P
M
P
M
H
P
M
P
H
M
P
.12
.08
.51
.49

22. Example 9
Suppose a unusual illness infects one out of
every 1000 people in a population. And
suppose that there is a good, but not perfect,
test for this disease: if a person has the
disease, the test comes back positive 99% of
the time. On the other hand, the test also
produces some false positives: 2% of
uninfected people are also test positive. And
someone just tested positive. What are his
chances of having this disease?

23. We know:
P(A) = .001 P(Ac) =.999
P(B|A) = .99 P(B|Ac) =.02
Example 9
Define A: has the disease B: test positive
0472
.
02
.
999
.
99
.
001
.
99
.
001
.
)
|
(
)
(
)
|
(
)
(
)
|
(
)
(
)
|
(







 c
A
B
P
c
A
P
A
B
P
A
P
A
B
P
A
P
B
A
P
We want to know P(A|B)=?

24. Example 10
• Three jars contain colored balls as described in the
table below.
– One jar is chosen at random and a ball is selected. If the ball
is red, what is the probability that it came from the 2nd jar?
Jar # Red White Blue
1 3 4 1
2 1 2 3
3 4 3 2

25. Example 10
• We will define the following events:
– J1 is the event that first jar is chosen
– J2 is the event that second jar is chosen
– J3 is the event that third jar is chosen
– R is the event that a red ball is selected

26. Example 10
• The events J1 , J2 , and J3 mutually exclusive
– Why?
• You can’t chose two different jars at the same time
• Because of this, our sample space has been
divided or partitioned along these three
events

27. Venn Diagram
• Let’s look at the Venn Diagram

28. Venn Diagram
• All of the red balls are in the first, second, and
third jar so their set overlaps all three sets of
our partition

29. Finding Probabilities
• What are the probabilities for each of the
events in our sample space?
• How do we find them?
     
B
P
B
A
P
B
A
P |



30. Computing Probabilities
• Similar calculations
show:
     
8
1
3
1
8
3
| 1
1
1 



 J
P
J
R
P
R
J
P
     
     
27
4
3
1
9
4
|
18
1
3
1
6
1
|
3
3
3
2
2
2










J
P
J
R
P
R
J
P
J
P
J
R
P
R
J
P

31. Venn Diagram
• Updating our Venn Diagram with these
probabilities:

32. Where are we going with this?
• Our original problem was:
– One jar is chosen at random and a ball is selected.
If the ball is red, what is the probability that it
came from the 2nd jar?
• In terms of the events we’ve defined we want:
   
 
R
P
R
J
P
R
J
P

 2
2 |

33. Finding our Probability
   
 
 
     
R
J
P
R
J
P
R
J
P
R
J
P
R
P
R
J
P
R
J
P









3
2
1
2
2
2 |

34. Arithmetic!
   
     
17
.
0
71
12
27
4
18
1
8
1
18
1
|
3
2
1
2
2




































R
J
P
R
J
P
R
J
P
R
J
P
R
J
P

35. Pairwise Independent
Mutually or complete Independent
Pairwise Independent: let U be a family of events from S,
we say that the events U are pairwise independent iff for any
pair of distinct event A,B∈ U, i.e: P(A∩B)=P(A).P(B)
Mutually or complete Independent: A family of events
U is said to be mutually or complete independent family iff for
every finite subcollection
{𝐴𝑖1, 𝐴𝑖2, … , 𝐴𝑖𝑘} of U, the following relation holds:
𝑃 𝐴𝑖1 ∩ 𝐴𝑖2, … ,∩ 𝐴𝑖𝑘 =
𝑗=1
𝑘
𝑃(𝐴𝑖𝑗)

36. Random Variables
• A quantitative variable x is a random variable if
the value that it assumes, corresponding to the
outcome of an experiment is a chance or
random event.
• Random variables can be discrete or continuous.
• Examples:
x = SAT score for a randomly selected student
x = number of people in a room at a randomly
selected time of day
x = number on the upper face of a randomly tossed
die

37. Probability Distributions for
Discrete Random Variables
The probability distribution for a discrete
random variable x resembles the relative
frequency distributions. It is a graph, Table or
formula that gives the possible values of x and
the probability p(x) associated with each value.
1
)
(
and
1
)
(
0
have
must
We



 x
p
x
p

38. Example 11
Toss a fair coin three times and
define x = number of heads.
1/8
1/8
1/8
1/8
1/8
1/8
1/8
1/8
P(x = 0) = 1/8
P(x = 1) = 3/8
P(x = 2) = 3/8
P(x = 3) = 1/8
HHH
HHT
HTH
THH
HTT
THT
TTH
TTT
x
3
2
2
2
1
1
1
0
x p(x)
0 1/8
1 3/8
2 3/8
3 1/8
Probability
Histogram for x

39. Example 12
Toss two dice and define
x = sum of two dice. x p(x)
2 1/36
3 2/36
4 3/36
5 4/36
6 5/36
7 6/36
8 5/36
9 4/36
10 3/36
11 2/36
12 1/36

40. Probability Distributions
Probability distributions can be used to describe
the population.
–Shape: Symmetric, skewed, mound-shaped…
–Outliers: unusual or unlikely measurements
–Center and spread: mean and standard
deviation. A population mean is called m and a
population standard deviation is called s.

41. The Mean
and Standard Deviation
Let x be a discrete random variable with
probability distribution p(x). Then the mean,
variance and standard deviation of x are given
as
2
2
2
:
deviation
Standard
)
(
)
(
:
Variance
)
(
:
Mean
s
s
m
s
m






x
p
x
x
xp

42. Example 13
Toss a fair coin 3 times and record x
the number of heads.
x p(x) xp(x) (x-m2p(x)
0 1/8 0 (-1.5)2(1/8)
1 3/8 3/8 (-0.5)2(3/8)
2 3/8 6/8 (0.5)2(3/8)
3 1/8 3/8 (1.5)2(1/8)
5
.
1
8
12
)
( 


 x
xp
m
)
(
)
( 2
2
x
p
x m
s 


688
.
75
.
75
.
28125
.
09375
.
09375
.
28125
.
2







s
s

43. Example 14
The probability distribution for x the
number of heads in tossing 3 fair
coins.
• Shape?
• Outliers
?
• Center?
• Spread?
Symmetric;
mound-shaped
None
m = 1.5
s = .688
m