Upcoming SlideShare
×

# Lecture 2-cs648

380 views

Published on

Published in: Technology, Education
0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
380
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
1
0
Likes
0
Embeds 0
No embeds

No notes for slide

### Lecture 2-cs648

1. 1. Randomized Algorithms CS648 Lecture 2 • Randomized Algorithm for Approximate Median • Elementary Probability theory 1
2. 2. Randomized Monte Carlo Algorithm for approximate median This lecture was delivered at slow pace and its flavor was that of a tutorial. Reason: To show that designing and analyzing a randomized algorithm demands right insight and just elementary probability. 2
3. 3. A simple probability exercise • There is a coin which gives HEADS with probability ¼ and TAILS with probability ¾. The coin is tossed times. What is the probability that we get at least HEADS ? [Stirling’s approximation for Factorial: ] 3
4. 4. Probability of getting “at least HEADS in tosses” Probability of getting at least heads: • Using Stirling’s approximation Since , so … Inverse exponential in . 4
5. 5. Approximate median Definition: Given an array A[] storing n numbers and ϵ > 0, compute an element whose rank is in the range [(1- ϵ)n/2, (1+ ϵ)n/2]. Best Deterministic Algorithm: • “Median of Medians” algorithm for finding exact median • Running time: O(n) • No faster algorithm possible for approximate median Can you give a short proof ? 5
6. 6. ½ - Approximate median A Randomized Algorithm Rand-Approx-Median(A) 1. Let k  c log n; 2. S  ∅; 3. For i=1 to k 4. x  an element selected randomly uniformly from A; 5. S  S U {x}; 6. Sort S. 7. Report the median of S. Running time: O(log n loglog n) 6
7. 7. Analyzing the error probability of Rand-approx-median n/4 Left Quarter Elements of A arranged in Increasing order of values 3n/4 Right Quarter When does the algorithm err ? To answer this question, try to characterize what will be a bad sample S ? 7
8. 8. Analyzing the error probability of Rand-approx-median n/4 Elements of A arranged in Increasing order of values Left Quarter Median of S 3n/4 Right Quarter Observation: Algorithm makes an error only if k/2 or more elements sampled from the Right Quarter (or Left Quarter). 8
9. 9. Analyzing the error probability of Rand-approx-median • n/4 Elements of A arranged in Increasing order of values 3n/4 Right Quarter Left Quarter Pr[ An element selected randomly from A is from Right quarter] = ¼ ?? Pr[ Out of k elements sampled from A, at least k/2 are from Right quarter] = ?? for Exactly the same as the coin tossing exercise we did ! 9
10. 10. Main result we discussed • Theorem: The Rand-approx-median algorithm fails to report ½ -approximate median from array A[1.. ] with probability at most. Homework: Design a randomized Monte Carlo algorithm for computing ϵ-approximate median of array A[1.. ] with running time O(log n loglog n) and error probability for any given constants ϵ and . [Do this homework sincerely without any friend’s help.] 10
11. 11. Elementary probability theory (It is so simple that you underestimate its elegance and power) 11
12. 12. Elementary probability theory (Relevant for CS648) • • We shall mainly deal with discrete probability theory in this course. We shall take the set theoretic approach to explain probability theory. Consider any random experiment : o Tossing a coin 5 times. o Throwing a dice 2 times. o Selecting a number randomly uniformly from [1..n]. How to capture the following facts in the theory of probability ? 1. Outcome will always be from a specified set. 2. Likelihood of each possible outcome is non-negative. 3. We may be interested in a collection of outcomes. 12
13. 13. Probability Space Definition: Probability space associated with a random experiment is an ordered pair (Ω,P), where • Ω is the set of all possible outcomes of the random experiment • P : Ω R such that • – P(ω) ≥ 0 for each ωϵ Ω Ω Elements of Ω are called elementary events or sample points. 13
14. 14. Event in a Probability Space Definition: An event A in a probability space (Ω,P) is a subset of Ω. The probability of event A is defined as • A Ω For sake of compact notation, we extend P for events as described above. 14
15. 15. Exercises A randomized algorithm can also be viewed as a random experiment. 1. What is the sample space associated with Randomized Quick sort ? 2. What is the sample space associated with Rand-approx-median algorithm ? 15
16. 16. An Important Advice In the following slides, we shall state well known equations (highlighted in yellow boxes) from probability theory. • You should internalize them fully. • We shall use them crucially in this course. • Make sincere attempts to solve exercises that follow. 16
17. 17. Union of two Events Given two events A and B defined over a probability space (,P), what is P(AUB) ? • A B Ω P(AUB) = P(A) + P(B) P(A∩B) Try to prove it by showing the following: Each ω ϵ AUB contributes exactly P(ω) in the right hand side. 17
18. 18. Union of three Events Given three events A₁, A₂, A₃, defined over a probability space (,P), what is P(A₁ U A₂ U A₃) ? • A B C Ω P(A₁ U A₂UA₃) = P(A₁) + P(A₂) + P( A₃) P(A₁∩A₂) P(A₂∩A₃) P(A₁∩A₃) + P(A₁∩A₂∩A₃) Try to prove this equation as well by showing the following: Each ω ϵ A₁ U A₂UA₃ contributes exactly P(ω) in the right hand side. 18
19. 19. Exercises • • For events ,…, defined over a probability space (,P), prove that P() = … ) • There are letters envelopes. For each letter, there is a unique envelope in which it should be placed. A careless postman places the letters randomly into envelopes (one letter in each envelope). What is the probability that no letter is placed correctly (into the envelope meant for it) ? 19
20. 20. Conditional Probability Happening of some event influences the likelihood of happening of other events. This notion is formally captured by conditional probability as follows. • Probability of event A conditioned on event B, compactly represented as P[A|B], means the following. Given that event B has happened, what is the probability that event A has also happened ? You might have seen and used the following equation for conditional probability. P[A|B] = Can you give suitable reason to justify the validity of the above equation ? In particular, give justification for ] in numerator and ] in denominator in this equation. 20
21. 21. Exercises • A man possesses five coins, two of which are double-headed, one is double-tailed, and two are normal. He shuts his eyes, picks a coin at random, and tosses it. What is the probability that the lower face of the coin is a head ? He opens his eyes and sees that the coin is showing heads; what it the probability that the lower face is a head ? He shuts his eyes again, and tosses the coin again. What is the probability that the lower face is a head ? He opens his eyes and sees that the coin is showing heads; what is the probability that the lower face is a head ? He discards this coin, picks another at random, and tosses it. What is the probability that it shows heads ? 21
22. 22. Partition of sample space and an “important Equation” A set of events ,…, defined over a probability space (,P) is said to induce a partition of if • = • • =∅ for all B Ω Given an event B, how can we express P(B) in terms of a given partition ? P(B) = ) 22
23. 23. Exercises • • There are sticks each of different heights. There are vacant slots arranged along a line and numbered from 1 to as we move from left to right. The sticks are placed into the slots according to a uniformly random permutation. A stick placed at th slot is said to be a dominating stick if its height is largest among all sticks placed in slots 1 to . Find the probability that th slot contains a dominating stick. 23
24. 24. Independent Events Two events A and B defined over a probability space (,P) are said to be independent if happening of one of them has no influence on the probability of the another event. Mathematically, it means that P(A|B)= P(A) and P(B|A)=P(B) • The following equation also compactly captures independence of two events. P(A ∩ B) = P(A) · P(B) Question: Can two independent events ever be disjoint ? 24
25. 25. Exercises • 1. Two fair dice are rolled. Show that the event that their sum is 7 is independent of the score shown by the first die. 2. Let (,P) be a probability space where = {1,2,…,p} for a given prime number p, and each elementary event has probability 1/p. Show that if two events A and B defined over (,P) are independent, then at least one of A and B is either ∅ or . 25