Ch05

420 views
344 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
420
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Ch05

  1. 1. Now it’s time to look at… Discrete Probability (Chapter 5)CMSC 203 Discrete Structures 1
  2. 2. Discrete ProbabilityEverything you have learned about countingconstitutes the basis for computing theprobability of events to happen.In the following, we will use the notion experimentfor a procedure that yields one of a given set ofpossible outcomes.This set of possible outcomes is called the samplespace of the experiment.An event is a subset of the sample space. CMSC 203 Discrete Structures 2
  3. 3. Discrete ProbabilityIf all outcomes in the sample space are equallylikely, the following definition of probabilityapplies:The probability of an event E, which is a subset ofa finite sample space S of equally likely outcomes,is given by p(E) = |E|/|S|.Probability values of event range from 0 (for anevent that will never happen) to 1 (for an eventthat will always happen whenever the experimentis carried out). CMSC 203 Discrete Structures 3
  4. 4. Discrete ProbabilityExample I:An urn contains 4 blue balls and 5 red balls. Whatis the probability that a ball chosen at randomfrom the urn is blue?Solution:There are 9 possible outcomes, and the event“blue ball is chosen” comprises four of theseoutcomes. Therefore, the probability of this eventis 4/9 or approximately 44.44%. CMSC 203 Discrete Structures 4
  5. 5. Discrete ProbabilityExample II:What is the probability of winning the lottery6/49, that is, picking the correct set of sixnumbers out of 49?Solution:There are C(49, 6) possible outcomes. Only one ofthese outcomes will actually make us win thelottery.p(E) = 1/C(49, 6) = 1/13,983,816 CMSC 203 Discrete Structures 5
  6. 6. Complimentary EventsLet E be an event in a sample space S. Theprobability of an event –E = S - E, thecomplimentary event of E, is given byp(-E) = 1 – p(E).This can easily be shown:p(-E) = (|S| - |E|)/|S| = 1 - |E|/|S| = 1 – p(E).This rule is useful if it is easier to determine theprobability of the complimentary event than theprobability of the event itself. CMSC 203 Discrete Structures 6
  7. 7. Complimentary EventsExample I: A sequence of 10 bits is randomlygenerated. What is the probability that at leastone of these bits is zero?Solution: There are 210 = 1024 possible outcomesof generating such a sequence. The event –E,“none of the bits is zero”, includes only one ofthese outcomes, namely the sequence 1111111111.Therefore, p(-E) = 1/1024.Now p(E) can easily be computed asp(E) = 1 – p(-E) = 1 – 1/1024 = 1023/1024. CMSC 203 Discrete Structures 7
  8. 8. Complimentary EventsExample II: What is the probability that at leasttwo out of 36 people have the same birthday?Solution: The sample space S encompasses allpossibilities for the birthdays of the 36 people,so |S| = 36536.Let us consider the event –E (“no two people out of36 have the same birthday”). –E includes P(365, 36)outcomes (365 possibilities for the first person’sbirthday, 364 for the second, and so on).Then p(-E) = P(365, 36)/36536 = 0.168,so p(E) = 0.832 or 83.2% CMSC 203 Discrete Structures 8
  9. 9. Discrete ProbabilityLet E1 and E2 be events in the sample space S.Then we have:p(E1 ∪ E2) = p(E1) + p(E2) - p(E1 ∩ E2)Does this remind you of something?Of course, the principle of inclusion-exclusion. CMSC 203 Discrete Structures 9
  10. 10. Discrete ProbabilityExample: What is the probability of a positiveinteger selected at random from the set of allpositive integers not exceeding 100 to be divisibleby 2 or 5?Solution:S = {1, 2, …, 100}, |S| = 100E2: “integer is divisible by 2”E5: “integer is divisible by 5”E2 = {2, 4, 6, …, 100}|E2| = 50p(E2) = 50/100 = 0.5 CMSC 203 Discrete Structures 10
  11. 11. Discrete ProbabilityE5 = {5, 10, 15, …, 100}|E5| = 20p(E5) = 20/100 = 0.2E2 ∩ E5 = {10, 20, 30, …, 100}|E2 ∩ E5| = 10p(E2 ∩ E5) = 0.1p(E2 ∪ E5) = p(E2) + p(E5) – p(E2 ∩ E5 )p(E2 ∪ E5) = 0.5 + 0.2 – 0.1 = 0.6 CMSC 203 Discrete Structures 11
  12. 12. Discrete ProbabilityWhat happens if the outcomes of an experimentare not equally likely?In that case, we assign a probability p(s) to eachoutcome s∈S, where S is the sample space.Two conditions have to be met:(1): 0 ≤ p(s) ≤ 1 for each s∈S, and(2): ∑s∈S p(s) = 1This means that (1) each probability must be avalue between 0 and 1, and (2) the probabilitiesmust add up to 1, because one and only of theoutcomes is guaranteed to occur in an experiment. CMSC 203 Discrete Structures 12
  13. 13. Discrete ProbabilityHow can we obtain these probabilities p(s) ?The probability p(s) assigned to an outcome sequals the limit of the number of times s occursdivided by the number of times the experiment isperformed. m p ( s ) = lim n→∞ where n n is the total number of experiments m is the number of experiments in which s occursWe call function p: S → [0, 1] a probabilitydistribution of S. CMSC 203 Discrete Structures 13
  14. 14. Discrete ProbabilityOnce we know the probability distribution p(s), wecan compute the probability of an event E asfollows:p(E) = ∑s∈E p(s)Example I: A die is biased so that the number 3appears twice as often as each other number.What are the probabilities of all possibleoutcomes? CMSC 203 Discrete Structures 14
  15. 15. Discrete ProbabilitySolution: There are 6 possible outcomes s1, …, s6.p(s1) = p(s2) = p(s4) = p(s5) = p(s6)p(s3) = 2p(s1)Since the probabilities must add up to 1, we have:5p(s1) + 2p(s1) = 17p(s1) = 1p(s1) = p(s2) = p(s4) = p(s5) = p(s6) = 1/7, p(s3) = 2/7 CMSC 203 Discrete Structures 15
  16. 16. Discrete ProbabilityExample II: For the biased die from Example I,what is the probability that an odd numberappears when we roll the die?Solution:Eodd = {s1, s3, s5}Remember the formula p(E) = ∑s∈E p(s).p(Eodd) = ∑s∈E p(s) = p(s1) + p(s3) + p(s5) oddp(Eodd) = 1/7 + 2/7 + 1/7 = 4/7 = 57.14% CMSC 203 Discrete Structures 16
  17. 17. Conditional ProbabilityIf we toss a fair coin three times, what is theprobability that an odd number of tails appears(event E), if the first toss is a tail (event F) ?If the first toss is a tail, the possible sequencesare TTT, TTH, THT, and THH.In two out of these four cases, there is an oddnumber of tails.Therefore, the probability of E, under thecondition that F occurs, is 0.5.We call this conditional probability. CMSC 203 Discrete Structures 17
  18. 18. Conditional ProbabilityIf we want to compute the conditional probabilityof E given F, we use F as the sample space.For any outcome of E to occur under the conditionthat F also occurs, this outcome must also be inE ∩ F.Definition: Let E and F be events with p(F) > 0.The conditional probability of E given F, denotedby p(E | F), is defined asp(E | F) = p(E ∩ F)/p(F)Venn diagram CMSC 203 Discrete Structures 18
  19. 19. Conditional ProbabilityExample: What is the probability of a random bitstring of length four contains at least twoconsecutive 0s, given that its first bit is a 0 ?Solution:E: “bit string contains at least two consecutive 0s”F: “first bit of the string is a 0”We know the formula p(E | F) = p(E ∩ F)/p(F).E ∩ F = {0000, 0001, 0010, 0011, 0100}p(E ∩ F) = 5/16p(F) = 8/16 = 1/2p(E | F) = (5/16)/(1/2) = 10/16 = 5/8 = 0.625 CMSC 203 Discrete Structures 19
  20. 20. Bayes’ TheoremSince p(E | F) = p(E ∩ F)/p(F), we have p(E | F)p(F) = p(E ∩ F)Symmetrically we have p(F | E)p(E) = p(E ∩ F)Therefore, p(E | F)p(F) = p(F | E)p(E)and p(F | E) = p(E | F)p(F)/p(E).This is called Bayes’ theorem, where – p(E | F) is the conditional probability, – p(E) and p(F) prior probabilities, and – P(F | E) posterior probability CMSC 203 Discrete Structures 20
  21. 21. IndependenceLet us return to the example of tossing a cointhree times.Does the probability of event E (odd number oftails) depend on the occurrence of event F (firsttoss is a tail) ?In other words, is it the case thatp(E | F) ≠ p(E) ?We actually find that p(E | F) = 0.5 and p(E) = 0.5,so we say that E and F are independent events. CMSC 203 Discrete Structures 21
  22. 22. IndependenceBecause we have p(E | F) = p(E ∩ F)/p(F),p(E | F) = p(E) if and only if p(E ∩ F) = p(E)p(F).Definition: The events E and F are said to beindependent if and only if p(E ∩ F) = p(E)p(F).Obviously, this definition is symmetrical for E andF. If we have p(E ∩ F) = p(E)p(F), then it is alsotrue that p(F | E) = p(F). CMSC 203 Discrete Structures 22
  23. 23. IndependenceExample: Suppose E is the event that a randomlygenerated bit string of length four begins with a 1,and F is the event that a randomly generated bitstring contains an even number of 0s. Are E and Findependent?Solution: Obviously, p(E) = p(F) = 0.5.E ∩ F = {1111, 1001, 1010, 1100}p(E ∩ F) = 0.25p(E ∩ F) = p(E)p(F)Conclusion: E And F are independent. CMSC 203 Discrete Structures 23
  24. 24. IndependenceIf E and F are independent of each other, thenP(E ∪ F) = p(E) + p(F) - p(E ∩ F) = p(E) + p(F) - p(E)p(F) = 1 – (1 – p(E))(1 – p(F))In general, if E1, E2, …, En are independent of eachother, then p ( E1 ∪ E2 ∪  ∪ En ) = 1 − Π (1 − p( Ei )) n i =1 CMSC 203 Discrete Structures 24
  25. 25. Bernoulli TrialsSuppose an experiment with two possibleoutcomes, such as tossing a coin.Each performance of such an experiment is calleda Bernoulli trial.We will call the two possible outcomes a successor a failure, respectively.If p is the probability of a success and q is theprobability of a failure in a single Bernoulli trial, itis obvious thatp + q = 1. CMSC 203 Discrete Structures 25
  26. 26. Bernoulli TrialsOften we are interested in the probability ofexactly k successes when an experiment consistsof n independent Bernoulli trials.Example:A coin is biased so that the probability of head is2/3. What is the probability of exactly fourheads to come up when the coin is tossed seventimes? CMSC 203 Discrete Structures 26
  27. 27. Bernoulli TrialsSolution:There are 27 = 128 possible outcomes (e.g.,HHHHTTT, HHHTHTT, …, TTTHHHH).The number of possible ways for four headsamong the seven trials is C(7, 4).The seven trials are independent, so theprobability of each of these outcomes is(2/3)4(1/3)3.Consequently, the probability of exactly fourheads to appear isC(7, 4)(2/3)4(1/3)3 = 560/2187 = 25.61% CMSC 203 Discrete Structures 27
  28. 28. Bernoulli TrialsTheorem: The probability of k successes in nindependent Bernoulli trials, with probability ofsuccess p and probability of failure q = 1 – p, isC(n, k)pkqn-k .See the textbook for the proof.We denote by b(k; n, p) the probability of ksuccesses in n independent Bernoulli trials withprobability of success p and probability of failureq = 1 – p.Considered as function of k, we call b the binomialdistribution. CMSC 203 Discrete Structures 28

×