Upcoming SlideShare
×

# lecture 15

800 views
721 views

Published on

Published in: Education, Technology
0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
800
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
19
0
Likes
0
Embeds 0
No embeds

No notes for slide

### lecture 15

1. 1. Divide-and-Conquer <ul><li>Matrix multiplication and Strassen’s algorithm </li></ul><ul><li>Median Problem </li></ul><ul><ul><li>In general finding the kth largest element of an unsorted list of numbers using comparisons </li></ul></ul>
2. 2. Matrix Multiplication <ul><li>How many operations are needed to multiply two 2 by 2 matrices? </li></ul>[ [ r s t u [ [ a b c d [ [ e f g h =
3. 3. Traditional Approach <ul><li>r = ae + bg </li></ul><ul><li>s = af + bh </li></ul><ul><li>t = ce + dg </li></ul><ul><li>u = cf + dh </li></ul><ul><li>8 multiplications and 4 additions </li></ul>[ [ r s t u [ [ a b c d [ [ e f g h =
4. 4. Extending to n by n matrices <ul><li>Each letter represents an n/2 by n/2 matrix </li></ul><ul><li>We can use the breakdown to form a divide and conquer algorithm </li></ul><ul><li>R = AE + BG </li></ul><ul><li>S = AF + BH </li></ul><ul><li>T = CE + DG </li></ul><ul><li>U = CF + DH </li></ul><ul><li>8 multiplications of n/2 by n/2 matrices </li></ul><ul><li>T(n) = 8 T(n/2) +  (n 2 ) </li></ul><ul><li>T(n) =  (n 3 ) </li></ul>[ [ R S T U [ [ A B C D [ [ E F G H =
5. 5. Strassen’s Approach <ul><li>p1 = a(f – h) </li></ul><ul><li>p2 = (a+b)h </li></ul><ul><li>p3 = (c+d)e </li></ul><ul><li>p4 = d(g-e) </li></ul><ul><li>p5 = (a+d)(e + h) </li></ul><ul><li>p6 = (b-d)(g+h) </li></ul><ul><li>p7 = (a-c)(e+f) </li></ul><ul><li>r = p5 + p4 - p2 + p6 </li></ul><ul><li>s = p1 + p2 </li></ul><ul><li>t = p3 + p4 </li></ul><ul><li>u = p5 + p1 – p3 – p7 </li></ul><ul><li>7 multiplications </li></ul><ul><li>18 additions </li></ul>[ [ r s t u [ [ a b c d [ [ e f g h =
6. 6. Extending to n by n matrices <ul><li>Each letter represents an n/2 by n/2 matrix </li></ul><ul><li>We can use the breakdown to form a divide and conquer algorithm </li></ul><ul><li>7 multiplications of n/2 by n/2 matrices </li></ul><ul><li>18 additions of n/2 by n/2 matrices </li></ul><ul><li>T(n) = 7 T(n/2) +  (n 2 ) </li></ul><ul><li>T(n) =  (n lg 7 ) </li></ul>[ [ R S T U [ [ A B C D [ [ E F G H =
7. 7. Observations <ul><li>Comparison: n= 70 </li></ul><ul><ul><li>Direct multiplication: 70 3 = 343,000 </li></ul></ul><ul><ul><li>Strassen: 70 lg 7 is approximately 150,000 </li></ul></ul><ul><ul><li>Crossover point typically around n = 20 </li></ul></ul><ul><li>Hopcroft and Kerr have shown 7 multiplications are necessary to multiply 2 by 2 matrices </li></ul><ul><ul><li>But we can do better with larger matrices </li></ul></ul><ul><li>Current best is O(n 2.376 ) by Coppersmith and Winograd, but it is not practical </li></ul><ul><li>Best lower bound is  (n 2 ) (since there are n 2 entries) </li></ul><ul><li>Matrix multiplication can be used in some graph algorithms as a fundamental step, so theoretical improvements in the efficiency of this operation can lead to theoretical improvements in those algorithms </li></ul>
8. 8. Median Problem <ul><li>How quickly can we find the median (or in general the kth largest element) of an unsorted list of numbers? </li></ul><ul><li>Two approaches </li></ul><ul><ul><li>Quicksort partition algorithm expected  (n) time but  (n 2 ) time in the worst-case </li></ul></ul><ul><ul><li>Deterministic  (n) time in the worst-case </li></ul></ul>
9. 9. Quicksort Approach <ul><li>int Select(int A[], k, low, high) </li></ul><ul><ul><li>Choose a pivot item </li></ul></ul><ul><ul><li>Compare all items to this pivot element </li></ul></ul><ul><ul><li>If pivot is kth item, return pivot </li></ul></ul><ul><ul><li>Else update low, high, k and recurse on partition that contains correct item </li></ul></ul>
10. 10. Probabilistic Analysis <ul><li>Assume each of n! permutations is equally likely </li></ul><ul><li>What is probability ith largest item is compared to jth largest item? </li></ul><ul><ul><li>If k is contained in (i..j), then 2/(j-i+1) </li></ul></ul><ul><ul><li>If k <= i, then 2/(j-k+1) </li></ul></ul><ul><ul><li>If k >= j, then 2/(k-i+1) </li></ul></ul>
11. 11. Cases where (i..j) do not contain k <ul><li>k>=j:  (i=1 to k-1)  j = i+1 to k 2/(k-i+1) =  i=1 to k-1 (k-i) 2/(k-i+1) </li></ul><ul><ul><li>=  i=1 to k-1 2i/(i+1) [replace k-i with i] </li></ul></ul><ul><ul><li>= 2  i=1 to k-1 i/(i+1) </li></ul></ul><ul><ul><li><= 2(k-1) </li></ul></ul><ul><li>k<=i:  (j=k+1 to n)  i = k to j-1 2/(j-k+1) =  j=k+1 to n (j-k) 2/(j-k+1) </li></ul><ul><ul><li>=  j = 1 to n-k 2j/(j+1) [replace j-k with j and change bounds] </li></ul></ul><ul><ul><li>= 2  j=1 to n-k j/(j+1) </li></ul></ul><ul><ul><li><= 2(n-k) </li></ul></ul><ul><li>Total for both cases is <= 2n-2 </li></ul>
12. 12. Case where (i..j) contains k <ul><li>At most 1 interval of size 3 contains k </li></ul><ul><ul><li>i=k-1, j=k+1 </li></ul></ul><ul><li>At most 2 intervals of size 4 contain k </li></ul><ul><ul><li>i=k-1, j=k+2 and i=k-2, j= k+1 </li></ul></ul><ul><li>In general, at most q-2 intervals of size q contain k </li></ul><ul><li>Thus we get  (q=3 to n) (q-2)/q <= n-2 </li></ul><ul><li>Summing together all cases we see the expected number of comparisons is less than 3n </li></ul>
13. 13. Best case, Worst-case <ul><li>Best case running time? </li></ul><ul><li>What happens in the worst-case? </li></ul><ul><ul><li>Pivot element chosen is always what? </li></ul></ul><ul><ul><li>This leads to comparing all possible pairs </li></ul></ul><ul><ul><li>This leads to  (n 2 ) comparisons </li></ul></ul>
14. 14. Deterministic O(n) approach <ul><li>Need to guarantee a good pivot element while doing O(n) work to find the pivot element </li></ul><ul><li>int Select(int A[], k, low, high) </li></ul><ul><ul><li>Choosing pivot element </li></ul></ul><ul><ul><ul><li>Divide into groups of 5 </li></ul></ul></ul><ul><ul><ul><li>For each group of 5, find that group’s median </li></ul></ul></ul><ul><ul><ul><li>Find the median of the medians </li></ul></ul></ul><ul><ul><li>Compare remaining items directly to median to identify its current exact placement </li></ul></ul><ul><ul><li>Update low, high, k and recurse on partition that contains correct item </li></ul></ul>
15. 15. Guarantees on the pivot element <ul><li>Median of medians is guaranteed to be smaller than all the red colored items </li></ul><ul><ul><li>Why? </li></ul></ul><ul><ul><li>How many red items are there? </li></ul></ul><ul><ul><li>½ (n/5) * 3 = 3n/10 </li></ul></ul><ul><li>Likewise, median of medians is guaranteed to be larger than the 3n/10 blue colored items </li></ul><ul><li>Thus median of medians is in the range 3n/10 to 7n/10 </li></ul>
16. 16. Analysis of running time <ul><li>int Select(int A[], k, low, high) </li></ul><ul><ul><li>Choosing pivot element </li></ul></ul><ul><ul><ul><li>For each group of 5, find that group’s median </li></ul></ul></ul><ul><ul><ul><li>Find the median of the medians </li></ul></ul></ul><ul><ul><li>Compare remaining items directly to median </li></ul></ul><ul><ul><li>Recurse on correct partition </li></ul></ul><ul><li>Analysis </li></ul><ul><ul><li>Choosing pivot element </li></ul></ul><ul><ul><ul><li>c 1 n/5 </li></ul></ul></ul><ul><ul><ul><ul><li>c 1 for median of 5 </li></ul></ul></ul></ul><ul><ul><ul><li>Recurse on problem of size n/5 </li></ul></ul></ul><ul><ul><li>n comparisons </li></ul></ul><ul><ul><li>Recurse on problem of size 7n/10 </li></ul></ul><ul><li>T(n) = T(7n/10) + T(n/5) + O(n) </li></ul>
17. 17. Solving recurrence relation <ul><li>T(n) = T(7n/10) + T(n/5) + O(n) </li></ul><ul><ul><li>Key observation: 7/10 + 1/5 = 9/10 < 1 </li></ul></ul><ul><li>Prove T(n) <= cn for some constant n by induction on n </li></ul><ul><li>T(n) = 7cn/10 + cn/5 + dn </li></ul><ul><li>= 9cn/10 + dn </li></ul><ul><li>Need 9cn/10 + dn <= cn </li></ul><ul><li>Thus c/10 >= d  c >= 10d </li></ul>