- 1. Lecture (5) Conditional Probability, Bayes Theorem and Independent Events with Examples University of Baghdad , Collage of Sciences, Department of Computer Sciences Diploma Class Syllabus Academic First Semester Dr. Ali H. Kashmar Year 2019/2020
- 2. Topics • Conditional Probability • Independent Events • Multiplication Rule of Probability • Total probability rule • Bayes rule • Pairwise Independent • Mutually or complete Independent • Probability Distributions for Discrete Random Variables • Examples
- 3. The probability of an event based on the fact that some other event has occurred, will occur, or is occurring. P(B/A) = Conditional Probability The probability of event B occurring given that event A has occurred is usually stated as “the conditional probability of B, given A; P(B/A) Conditional Probability ) ( ) ( A P B A P ) ( ) ( A P B and A P
- 4. Example: 1 A number from the sample space S = {2, 3, 4, 5, 6, 7, 8, 9} is randomly selected. Given the defined events A and B, A: selected number is odd, and B: selected number is a multiple of 3 Find the following probabilities. a) B = {3, 6, 9} Conditional Probability b) P(A and B) = a) P(B) b) P(A and B) c) P(B/A) Conditional Probability P(B) = 3/8 P({3, 5, 7, 9} {3, 6, 9}) = P({3, 9}) = 2/8 = 1/4 c) Probability of B given A has occurred: P(B/A) = P(A) P(A and B) 4/8 1/4 1/2 = =
- 5. Example: 2 Given a family with two children, find the probability that both are boys, given that at least one is a boy. P({gb, bg, bb}) = Conditional Probability P(A and B) = A = at least one boy Conditional Probability P(A) = 3/4 P({gb, bg, bb} {bb}) = P({bb}) = = 1/3 1/4 Conditional Probability P(B/A) = 3/4 1/4 B = both are boys S= {gg, gb, bg, bb} A = {gb, bg, bb}=3/4 B = {bb}=1/4 ) ( ) ( A P B and A P ) ( ) ( A P B and A P =
- 6. Two events are Independent if the occurrence of one of them has no effect on the probability of the other. Theorem: 1 If A and B are independent events, then P(B/A) = P(B) and P(A/B) = P(A) Where P(A) > 0 and P(B) > 0 P(B/A) = P(B) need to proof? Independent Events or Independent Events P(A/B) = P(A) need to proof?
- 7. Theorem:2 If A and B are independent events, then A and 𝐵𝑐 , B and 𝐴𝑐 are independent. Proof: Homework
- 8. Example: 3 A single card is randomly selected from a standard 52-card deck. Given the defined events A and B, A: the selected card is an ace, B: the selected card is red, find the following probabilities. a) P(B) = Independent Events b) P(A and B) = a) P(B) b) P(A and B) c) P(B/A) Independent Events = 1/2 P({Ah, Ad, Ac, As} {all red}) = P({Ah, Ad}) = 2/52 Events A and B are independent as P(B) = P(B/A). c) P(B/A) = P(A) P(A and B) 4/52 2/52 1/2 = = 52 26
- 9. Example 4 X, Y, Z, H four students, • P(X|Y)= prob. Of X is success given Y is success too • P(X|Y)=P(X) • Thus: P(A|B)=P(A) • P(A∩B)=P(A).P(B) • A and B are independent events
- 10. Example 5 Two coins be tossed, let A={H in the 2nd throw} B={ H in the 1st throw}, Are A and B independent events? Sol: Ω={HH,HT,TH,TT} A={HH,TH} P(A)=2/4=1/2 B={HH,HT} P(B)=2/4=1/2 A∩B={HH] p(A∩B)=1/4 P(A|B)= p(A∩B) 𝑃(𝐵) = 1/4 1/2 = 1/2 i.e P(A|B)= P(A) P(A).P(B)=(1/2).(1/2)=1/4 Then : A and B are independent events
- 11. If A and B are any two events then P(A and B) = P(A) P(B/A) Multiplication Rule of Probability Multiplication Rule of Probability P(A and B) = P(A) P(B) If A and B are independent events then A jar contains 4 red marbles, 3 blue marbles, and 2 yellow marbles. What is the probability that a red marble is selected and then a blue one without replacement? P(Red and Blue) = P(Red) P(Blue/Red) = 4/9 3/8 = 12/72 = 1/8 = 0.1667 Example:6
- 12. Multiplication Rule of Probability Multiplication Rule of Probability A jar contains 4 red marbles, 3 blue marbles, and 2 yellow marbles. What is the probability that a red marble is selected and then a blue one with replacement? P(Red and Blue) = P(Red) P(Blue) = 4/9 3/9 = 12/81 = 4/27 = 0.148 Example:7
- 13. The Law of Total Probability P(A) = P(A S1) + P(A S2) + … + P(A Sk) = P(S1)P(A|S1) + P(S2)P(A|S2) + … + P(Sk)P(A|Sk) Let S1 , S2 , S3 ,..., Sk be mutually exclusive and exhaustive events (that is, one and only one must happen). Then the probability of any event A can be written as
- 14. The Law of Total Probability A A Sk A S1 S2…. S1 Sk P(A) = P(A S1) + P(A S2) + … + P(A Sk) = P(S1)P(A|S1) + P(S2)P(A|S2) + … + P(Sk)P(A|Sk) = 𝒊=𝟏 𝒌 𝑷 𝑺𝒊 𝑷(𝑨|𝑺𝒊)
- 15. Homework 1). we have 3 urns (U1, U2, U3) U1={1w,2b}, U2={1b,2w}, U3={3b,3w} • A dice is rolled: – If 1, 2, 3, U1 is selected – If 4, U2 is selected – If 5,6 U3 is selected • A={choose one white ball} Find p(A)? • B={2 balls are selected, one white and one block} Find p(B)?
- 16. 2) U1={1w, 2b, 3r}, U2={2w, 1b, 1r}, U3={3w, 3b, 2r} 3-balls are selected. 2-dice are rolled. Let x be the no. of the first dice Let y be the no. of the second dice If x>y U1 is selected If x=y U2 is selected If x<y U3 is selected A={3-balls are selected and at least one red}. B={3-balls are selected and at most 1-red and 1-block}. C={3-balls are selected and one ball from each kind}. Find: p(A),p(A∩B), p(A∩B ∩ C), p(B),p(A∩ C), p(B ∩ C) and p(C)
- 17. Bayes’ rule ,... 2 , 1 ) | ( ) ( ) | ( ) ( ) | ( 1 j where H B P H P H B P H P B H P j j j j j j • Theorem: Let {𝐻𝑛} be a disjoint sequences of events, such that P( 𝐻𝑛 ) > 0 for n=1,2,… and 𝐻𝑛 = Ω , Let B∈ 𝑆 with P(𝐵) > 0 , then:
- 18. Bayes’ Rule: Proof can be re-arranged to: ) ( ) ( ) | ( j j j H P B H P H B P ) | ( ) ( ) ( 1 j j j H B P H P B P ) ( ) ( ) | ( B P B H P B H P j j and, by using total probability def. We define: ) | ( ). ( ) ( B H P B P B H P j j ) | ( ). ( ) ( j j j H B P H P B H P ) ( ) | ( ). ( ) ( ) ( ) | ( B P H B P H P B P B H P B H P j j j j
- 19. Bayes’ Rule: p(𝐻𝑗): called prior probability p(𝐻𝑗|B): called posturer probability ) | ( ) ( ) | ( ) ( ) | ( 1 j j j j j j H B P H P H B P H P B H P
- 20. Bayes’ Rule: • Why do we care? • Why is Bayes’ Rule useful? • It turns out that sometimes it is very useful to be able to “flip” conditional probabilities. That is, we may know the probability of A given B, but the probability of B given A may not be obvious. An example will help…
- 21. We know: P(F) = P(M) = P(H|F) = P(H|M) = Example 8 From a previous information, we know that 49% of the population are female. Of the female patients, 8% are high risk for heart attack, while 12% of the male patients are high risk. A single person is selected at random and found to be high risk. What is the probability that it is a male? Define H: high risk F: female M: male 61 . ) 08 (. 49 . ) 12 (. 51 . ) 12 (. 51 . ) | ( ) ( ) | ( ) ( ) | ( ) ( ) | ( F H P F P M H P M P M H P M P H M P .12 .08 .51 .49
- 22. Example 9 Suppose a unusual illness infects one out of every 1000 people in a population. And suppose that there is a good, but not perfect, test for this disease: if a person has the disease, the test comes back positive 99% of the time. On the other hand, the test also produces some false positives: 2% of uninfected people are also test positive. And someone just tested positive. What are his chances of having this disease?
- 23. We know: P(A) = .001 P(Ac) =.999 P(B|A) = .99 P(B|Ac) =.02 Example 9 Define A: has the disease B: test positive 0472 . 02 . 999 . 99 . 001 . 99 . 001 . ) | ( ) ( ) | ( ) ( ) | ( ) ( ) | ( c A B P c A P A B P A P A B P A P B A P We want to know P(A|B)=?
- 24. Example 10 • Three jars contain colored balls as described in the table below. – One jar is chosen at random and a ball is selected. If the ball is red, what is the probability that it came from the 2nd jar? Jar # Red White Blue 1 3 4 1 2 1 2 3 3 4 3 2
- 25. Example 10 • We will define the following events: – J1 is the event that first jar is chosen – J2 is the event that second jar is chosen – J3 is the event that third jar is chosen – R is the event that a red ball is selected
- 26. Example 10 • The events J1 , J2 , and J3 mutually exclusive – Why? • You can’t chose two different jars at the same time • Because of this, our sample space has been divided or partitioned along these three events
- 27. Venn Diagram • Let’s look at the Venn Diagram
- 28. Venn Diagram • All of the red balls are in the first, second, and third jar so their set overlaps all three sets of our partition
- 29. Finding Probabilities • What are the probabilities for each of the events in our sample space? • How do we find them? B P B A P B A P |
- 30. Computing Probabilities • Similar calculations show: 8 1 3 1 8 3 | 1 1 1 J P J R P R J P 27 4 3 1 9 4 | 18 1 3 1 6 1 | 3 3 3 2 2 2 J P J R P R J P J P J R P R J P
- 31. Venn Diagram • Updating our Venn Diagram with these probabilities:
- 32. Where are we going with this? • Our original problem was: – One jar is chosen at random and a ball is selected. If the ball is red, what is the probability that it came from the 2nd jar? • In terms of the events we’ve defined we want: R P R J P R J P 2 2 |
- 33. Finding our Probability R J P R J P R J P R J P R P R J P R J P 3 2 1 2 2 2 |
- 34. Arithmetic! 17 . 0 71 12 27 4 18 1 8 1 18 1 | 3 2 1 2 2 R J P R J P R J P R J P R J P
- 35. Pairwise Independent Mutually or complete Independent Pairwise Independent: let U be a family of events from S, we say that the events U are pairwise independent iff for any pair of distinct event A,B∈ U, i.e: P(A∩B)=P(A).P(B) Mutually or complete Independent: A family of events U is said to be mutually or complete independent family iff for every finite subcollection {𝐴𝑖1, 𝐴𝑖2, … , 𝐴𝑖𝑘} of U, the following relation holds: 𝑃 𝐴𝑖1 ∩ 𝐴𝑖2, … ,∩ 𝐴𝑖𝑘 = 𝑗=1 𝑘 𝑃(𝐴𝑖𝑗)
- 36. Random Variables • A quantitative variable x is a random variable if the value that it assumes, corresponding to the outcome of an experiment is a chance or random event. • Random variables can be discrete or continuous. • Examples: x = SAT score for a randomly selected student x = number of people in a room at a randomly selected time of day x = number on the upper face of a randomly tossed die
- 37. Probability Distributions for Discrete Random Variables The probability distribution for a discrete random variable x resembles the relative frequency distributions. It is a graph, Table or formula that gives the possible values of x and the probability p(x) associated with each value. 1 ) ( and 1 ) ( 0 have must We x p x p
- 38. Example 11 Toss a fair coin three times and define x = number of heads. 1/8 1/8 1/8 1/8 1/8 1/8 1/8 1/8 P(x = 0) = 1/8 P(x = 1) = 3/8 P(x = 2) = 3/8 P(x = 3) = 1/8 HHH HHT HTH THH HTT THT TTH TTT x 3 2 2 2 1 1 1 0 x p(x) 0 1/8 1 3/8 2 3/8 3 1/8 Probability Histogram for x
- 39. Example 12 Toss two dice and define x = sum of two dice. x p(x) 2 1/36 3 2/36 4 3/36 5 4/36 6 5/36 7 6/36 8 5/36 9 4/36 10 3/36 11 2/36 12 1/36
- 40. Probability Distributions Probability distributions can be used to describe the population. –Shape: Symmetric, skewed, mound-shaped… –Outliers: unusual or unlikely measurements –Center and spread: mean and standard deviation. A population mean is called m and a population standard deviation is called s.
- 41. The Mean and Standard Deviation Let x be a discrete random variable with probability distribution p(x). Then the mean, variance and standard deviation of x are given as 2 2 2 : deviation Standard ) ( ) ( : Variance ) ( : Mean s s m s m x p x x xp
- 42. Example 13 Toss a fair coin 3 times and record x the number of heads. x p(x) xp(x) (x-m2p(x) 0 1/8 0 (-1.5)2(1/8) 1 3/8 3/8 (-0.5)2(3/8) 2 3/8 6/8 (0.5)2(3/8) 3 1/8 3/8 (1.5)2(1/8) 5 . 1 8 12 ) ( x xp m ) ( ) ( 2 2 x p x m s 688 . 75 . 75 . 28125 . 09375 . 09375 . 28125 . 2 s s
- 43. Example 14 The probability distribution for x the number of heads in tossing 3 fair coins. • Shape? • Outliers ? • Center? • Spread? Symmetric; mound-shaped None m = 1.5 s = .688 m