Randomized Algorithms
CS648

Lecture 2
• Randomized Algorithm for Approximate Median
• Elementary Probability theory
1
Randomized Monte Carlo Algorithm for
approximate median
This lecture was delivered at slow pace and its flavor was that of...
A simple probability exercise
•

There is a coin which gives HEADS with probability ¼ and TAILS with
probability ¾. The co...
Probability of getting
“at least HEADS in tosses”
Probability of getting at least heads:

•

Using Stirling’s approximatio...
Approximate median
Definition: Given an array A[] storing n numbers and ϵ > 0, compute an
element whose rank is in the ran...
½ - Approximate median
A Randomized Algorithm

Rand-Approx-Median(A)
1. Let k  c log n;
2. S  ∅;
3. For i=1 to k
4.
x  ...
Analyzing the error probability of
Rand-approx-median
n/4

Left Quarter

Elements of A arranged in
Increasing order of val...
Analyzing the error probability of
Rand-approx-median
n/4

Elements of A arranged in
Increasing order of values

Left Quar...
Analyzing the error probability of
Rand-approx-median
•

n/4

Elements of A arranged in
Increasing order of values

3n/4

...
Main result we discussed

•
Theorem: The Rand-approx-median algorithm fails to report ½
-approximate median from array A[1...
Elementary probability theory
(It is so simple that you underestimate its elegance and power)

11
Elementary probability theory
(Relevant for CS648)
•
•

We shall mainly deal with discrete probability theory in this cour...
Probability Space
Definition: Probability space associated with a random experiment is an
ordered pair (Ω,P), where
• Ω is...
Event in a Probability Space
Definition: An event A in a probability space (Ω,P) is a subset of Ω. The
probability of even...
Exercises

A randomized algorithm can also be viewed as a random experiment.
1. What is the sample space associated with R...
An Important Advice
In the following slides, we shall state well known equations
(highlighted in yellow boxes) from probab...
Union of two Events
Given two events A and B defined over a probability space (,P), what is
P(AUB) ?

•

A

B

Ω

P(AUB) =...
Union of three Events
Given three events A₁, A₂, A₃, defined over a probability space (,P), what is
P(A₁ U A₂ U A₃) ?

•

...
Exercises
•

•

For events ,…, defined over a probability space (,P), prove that P() =

…
)
•

There are letters envelopes...
Conditional Probability
Happening of some event influences the likelihood of happening of other events. This
notion is for...
Exercises
•

A man possesses five coins, two of which are double-headed, one is
double-tailed, and two are normal. He shut...
Partition of sample space and
an “important Equation”
A set of events ,…, defined over a probability space (,P) is said to...
Exercises
•
•

There are sticks each of different heights. There are vacant slots arranged
along a line and numbered from ...
Independent Events
Two events A and B defined over a probability space (,P) are said to be
independent if happening of one...
Exercises
•
1.

Two fair dice are rolled. Show that the event that their sum is 7 is
independent of the score shown by the...
Upcoming SlideShare
Loading in …5
×

Lecture 2-cs648

380 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
380
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Lecture 2-cs648

  1. 1. Randomized Algorithms CS648 Lecture 2 • Randomized Algorithm for Approximate Median • Elementary Probability theory 1
  2. 2. Randomized Monte Carlo Algorithm for approximate median This lecture was delivered at slow pace and its flavor was that of a tutorial. Reason: To show that designing and analyzing a randomized algorithm demands right insight and just elementary probability. 2
  3. 3. A simple probability exercise • There is a coin which gives HEADS with probability ¼ and TAILS with probability ¾. The coin is tossed times. What is the probability that we get at least HEADS ? [Stirling’s approximation for Factorial: ] 3
  4. 4. Probability of getting “at least HEADS in tosses” Probability of getting at least heads: • Using Stirling’s approximation Since , so … Inverse exponential in . 4
  5. 5. Approximate median Definition: Given an array A[] storing n numbers and ϵ > 0, compute an element whose rank is in the range [(1- ϵ)n/2, (1+ ϵ)n/2]. Best Deterministic Algorithm: • “Median of Medians” algorithm for finding exact median • Running time: O(n) • No faster algorithm possible for approximate median Can you give a short proof ? 5
  6. 6. ½ - Approximate median A Randomized Algorithm Rand-Approx-Median(A) 1. Let k  c log n; 2. S  ∅; 3. For i=1 to k 4. x  an element selected randomly uniformly from A; 5. S  S U {x}; 6. Sort S. 7. Report the median of S. Running time: O(log n loglog n) 6
  7. 7. Analyzing the error probability of Rand-approx-median n/4 Left Quarter Elements of A arranged in Increasing order of values 3n/4 Right Quarter When does the algorithm err ? To answer this question, try to characterize what will be a bad sample S ? 7
  8. 8. Analyzing the error probability of Rand-approx-median n/4 Elements of A arranged in Increasing order of values Left Quarter Median of S 3n/4 Right Quarter Observation: Algorithm makes an error only if k/2 or more elements sampled from the Right Quarter (or Left Quarter). 8
  9. 9. Analyzing the error probability of Rand-approx-median • n/4 Elements of A arranged in Increasing order of values 3n/4 Right Quarter Left Quarter Pr[ An element selected randomly from A is from Right quarter] = ¼ ?? Pr[ Out of k elements sampled from A, at least k/2 are from Right quarter] = ?? for Exactly the same as the coin tossing exercise we did ! 9
  10. 10. Main result we discussed • Theorem: The Rand-approx-median algorithm fails to report ½ -approximate median from array A[1.. ] with probability at most. Homework: Design a randomized Monte Carlo algorithm for computing ϵ-approximate median of array A[1.. ] with running time O(log n loglog n) and error probability for any given constants ϵ and . [Do this homework sincerely without any friend’s help.] 10
  11. 11. Elementary probability theory (It is so simple that you underestimate its elegance and power) 11
  12. 12. Elementary probability theory (Relevant for CS648) • • We shall mainly deal with discrete probability theory in this course. We shall take the set theoretic approach to explain probability theory. Consider any random experiment : o Tossing a coin 5 times. o Throwing a dice 2 times. o Selecting a number randomly uniformly from [1..n]. How to capture the following facts in the theory of probability ? 1. Outcome will always be from a specified set. 2. Likelihood of each possible outcome is non-negative. 3. We may be interested in a collection of outcomes. 12
  13. 13. Probability Space Definition: Probability space associated with a random experiment is an ordered pair (Ω,P), where • Ω is the set of all possible outcomes of the random experiment • P : Ω R such that • – P(ω) ≥ 0 for each ωϵ Ω Ω Elements of Ω are called elementary events or sample points. 13
  14. 14. Event in a Probability Space Definition: An event A in a probability space (Ω,P) is a subset of Ω. The probability of event A is defined as • A Ω For sake of compact notation, we extend P for events as described above. 14
  15. 15. Exercises A randomized algorithm can also be viewed as a random experiment. 1. What is the sample space associated with Randomized Quick sort ? 2. What is the sample space associated with Rand-approx-median algorithm ? 15
  16. 16. An Important Advice In the following slides, we shall state well known equations (highlighted in yellow boxes) from probability theory. • You should internalize them fully. • We shall use them crucially in this course. • Make sincere attempts to solve exercises that follow. 16
  17. 17. Union of two Events Given two events A and B defined over a probability space (,P), what is P(AUB) ? • A B Ω P(AUB) = P(A) + P(B) P(A∩B) Try to prove it by showing the following: Each ω ϵ AUB contributes exactly P(ω) in the right hand side. 17
  18. 18. Union of three Events Given three events A₁, A₂, A₃, defined over a probability space (,P), what is P(A₁ U A₂ U A₃) ? • A B C Ω P(A₁ U A₂UA₃) = P(A₁) + P(A₂) + P( A₃) P(A₁∩A₂) P(A₂∩A₃) P(A₁∩A₃) + P(A₁∩A₂∩A₃) Try to prove this equation as well by showing the following: Each ω ϵ A₁ U A₂UA₃ contributes exactly P(ω) in the right hand side. 18
  19. 19. Exercises • • For events ,…, defined over a probability space (,P), prove that P() = … ) • There are letters envelopes. For each letter, there is a unique envelope in which it should be placed. A careless postman places the letters randomly into envelopes (one letter in each envelope). What is the probability that no letter is placed correctly (into the envelope meant for it) ? 19
  20. 20. Conditional Probability Happening of some event influences the likelihood of happening of other events. This notion is formally captured by conditional probability as follows. • Probability of event A conditioned on event B, compactly represented as P[A|B], means the following. Given that event B has happened, what is the probability that event A has also happened ? You might have seen and used the following equation for conditional probability. P[A|B] = Can you give suitable reason to justify the validity of the above equation ? In particular, give justification for ] in numerator and ] in denominator in this equation. 20
  21. 21. Exercises • A man possesses five coins, two of which are double-headed, one is double-tailed, and two are normal. He shuts his eyes, picks a coin at random, and tosses it. What is the probability that the lower face of the coin is a head ? He opens his eyes and sees that the coin is showing heads; what it the probability that the lower face is a head ? He shuts his eyes again, and tosses the coin again. What is the probability that the lower face is a head ? He opens his eyes and sees that the coin is showing heads; what is the probability that the lower face is a head ? He discards this coin, picks another at random, and tosses it. What is the probability that it shows heads ? 21
  22. 22. Partition of sample space and an “important Equation” A set of events ,…, defined over a probability space (,P) is said to induce a partition of if • = • • =∅ for all B Ω Given an event B, how can we express P(B) in terms of a given partition ? P(B) = ) 22
  23. 23. Exercises • • There are sticks each of different heights. There are vacant slots arranged along a line and numbered from 1 to as we move from left to right. The sticks are placed into the slots according to a uniformly random permutation. A stick placed at th slot is said to be a dominating stick if its height is largest among all sticks placed in slots 1 to . Find the probability that th slot contains a dominating stick. 23
  24. 24. Independent Events Two events A and B defined over a probability space (,P) are said to be independent if happening of one of them has no influence on the probability of the another event. Mathematically, it means that P(A|B)= P(A) and P(B|A)=P(B) • The following equation also compactly captures independence of two events. P(A ∩ B) = P(A) · P(B) Question: Can two independent events ever be disjoint ? 24
  25. 25. Exercises • 1. Two fair dice are rolled. Show that the event that their sum is 7 is independent of the score shown by the first die. 2. Let (,P) be a probability space where = {1,2,…,p} for a given prime number p, and each elementary event has probability 1/p. Show that if two events A and B defined over (,P) are independent, then at least one of A and B is either ∅ or . 25

×