Upcoming SlideShare
×

# L2

647 views
550 views

Published on

0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
647
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
8
0
Likes
0
Embeds 0
No embeds

No notes for slide

### L2

1. 1. Introduction to Algorithms 6.046J Lecture 2 Prof. Shafi Goldwasser
2. 2. Solving recurrences <ul><li>The analysis of integer multiplication from Lecture 1 required us to solve a recurrence. </li></ul><ul><li>Recurrences are a major tool for analysis of algorithms </li></ul><ul><ul><li>Today: Learn a few methods. </li></ul></ul><ul><li>Lecture 3 : Divide and Conquer algorithms which are analyzable by recurrences. </li></ul>
3. 3. Recall: Integer Multiplication <ul><li>Let X = A B and Y = C D where A,B,C and D are n/2 bit integers </li></ul><ul><li>Simple Method : XY = (2 n/2 A+B)(2 n/2 C+D) </li></ul><ul><li>Running Time Recurrence </li></ul><ul><li>T(n) < 4T(n/2) + 100n </li></ul><ul><li>How do we solve it? </li></ul>
4. 4. Substitution method <ul><li>Guess the form of the solution. </li></ul><ul><li>Verify by induction. </li></ul><ul><li>Solve for constants. </li></ul>The most general method: Example: T ( n ) = 4 T ( n /2) + 100 n <ul><li>[Assume that T (1) =  (1) .] </li></ul><ul><li>Guess O ( n 3 ) . (Prove O and  separately.) </li></ul><ul><li>Assume that T ( k )  ck 3 for k < n . </li></ul><ul><li>Prove T ( n )  cn 3 by induction. </li></ul>
5. 5. Example of substitution desired – residual whenever ( c /2) n 3 – 100 n  0 , for example, if c  200 and n  1 . desired residual 3 3 3 3 3 ) ) 2 / (( ) 2 / ( ) 2 / ( 4 ) 2 / ( 4 ) ( cn 100n n c cn 100n n c 100n n c 100n n T n T          
6. 6. Example (continued) <ul><li>We must also handle the initial conditions, that is, ground the induction with base cases. </li></ul><ul><li>Base: T ( n ) =  (1) for all n < n 0 , where n 0 is a suitable constant. </li></ul><ul><li>For 1  n < n 0 , we have “  (1) ”  cn 3 , if we pick c big enough. </li></ul>This bound is not tight!
7. 7. A tighter upper bound? We shall prove that T ( n ) = O ( n 2 ) . Assume that T ( k )  ck 2 for k < n : for no choice of c > 0 . Lose! 2 cn   ) 2 / ( 4 ) ( 2 100n cn 100n n T n T    
8. 8. A tighter upper bound! <ul><li>I DEA : Strengthen the inductive hypothesis. </li></ul><ul><li>Subtract a low-order term. </li></ul>Inductive hypothesis : T ( k )  c 1 k 2 – c 2 k for k < n . if c 2 > 100 . Pick c 1 big enough to handle the initial conditions. ) ( 2 )) 2 / ( ) 2 / ( ( 4 ) 2 / ( 4 ) ( 2 2 1 2 2 2 1 2 2 1 2 2 1 n c n c 100n n c n c n c 100n n c n c 100n n c n c 100n n T n T              
9. 9. Recursion-tree method <ul><li>A recursion tree models the costs (time) of a recursive execution of an algorithm. </li></ul><ul><li>The recursion tree method is good for generating guesses for the substitution method. </li></ul><ul><li>The recursion-tree method can be unreliable, just like any method that uses ellipses (…). </li></ul><ul><li>The recursion-tree method promotes intuition, however. </li></ul>
10. 10. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 :
11. 11. Example of recursion tree T ( n ) Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 :
12. 12. Example of recursion tree n 2 Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : T ( n /4) T ( n /2)
13. 13. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : n 2 ( n /4) 2 ( n /2) 2 T ( n /16) T ( n /8) T ( n /8) T ( n /4)
14. 14. Example of recursion tree ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2 ( n /2) 2  (1) … Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : n 2
15. 15. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2 ( n /2) 2  (1) … n 2
16. 16. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2 ( n /2) 2  (1) … n 2
17. 17. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2  (1) … n 2 ( n /2) 2 …
18. 18. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2  (1) … … Total = =  ( n 2 ) n 2 ( n /2) 2 geometric series
19. 19. Appendix: geometric series Return to last slide viewed. for | x | < 1 for x  1
20. 20. The master method The master method applies to recurrences of the form T ( n ) = a T ( n / b ) + f ( n ) , where a  1 , b > 1 , and f is asymptotically positive.
21. 21. Idea of master theorem f ( n/b ) f ( n/b ) f ( n/b )   (1) … Recursion tree: … f ( n ) a f ( n/b 2 ) f ( n/b 2 ) f ( n/b 2 ) … a h = log b n f ( n ) a f ( n/b ) a 2 f ( n/b 2 ) … #leaves = a h = a log b n = n log b a n log b a   (1)
22. 22. Three common cases Compare f ( n ) with n log b a : <ul><li>f ( n ) = O ( n log b a –  ) for some constant  > 0 . </li></ul><ul><ul><li>f ( n ) grows polynomially slower than n log b a (by an n  factor). </li></ul></ul><ul><li>Solution: T ( n ) =  ( n log b a ) . </li></ul>
23. 23. Idea of master theorem f ( n/b ) f ( n/b ) f ( n/b )   (1) … Recursion tree: … f ( n ) a f ( n/b 2 ) f ( n/b 2 ) f ( n/b 2 ) … a h = log b n f ( n ) a f ( n/b ) a 2 f ( n/b 2 ) … n log b a   (1) C ASE 1 : The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight.  ( n log b a )
24. 24. Three common cases Compare f ( n ) with n log b a : <ul><li>f ( n ) =  ( n log b a lg k n ) for some constant k  0 . </li></ul><ul><ul><li>f ( n ) and n log b a grow at similar rates. </li></ul></ul><ul><ul><li>Solution: T ( n ) =  ( n log b a lg k +1 n ) . </li></ul></ul>
25. 25. Idea of master theorem f ( n/b ) f ( n/b ) f ( n/b )   (1) … Recursion tree: … f ( n ) a f ( n/b 2 ) f ( n/b 2 ) f ( n/b 2 ) … a h = log b n f ( n ) a f ( n/b ) a 2 f ( n/b 2 ) … n log b a   (1) C ASE 2 : ( k = 0 ) The weight is approximately the same on each of the log b n levels.  ( n log b a lg n )
26. 26. Three common cases (cont.) Compare f ( n ) with n log b a : <ul><li>f ( n ) =  ( n log b a +  ) for some constant  > 0 . </li></ul><ul><ul><li>f ( n ) grows polynomially faster than n log b a (by an n  factor), </li></ul></ul><ul><li>and f ( n ) satisfies the regularity condition that a f ( n/b )  c f ( n ) for some constant c < 1 . </li></ul><ul><li>Solution: T ( n ) =  ( f ( n ) ) . </li></ul>
27. 27. Idea of master theorem f ( n/b ) f ( n/b ) f ( n/b )   (1) … Recursion tree: … f ( n ) a f ( n/b 2 ) f ( n/b 2 ) f ( n/b 2 ) … a h = log b n f ( n ) a f ( n/b ) a 2 f ( n/b 2 ) … n log b a   (1) C ASE 3 : The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight.  ( f ( n ))
28. 28. Examples Ex. T ( n ) = 4 T ( n /2) + n a = 4 , b = 2  n log b a = n 2 ; f ( n ) = n . C ASE 1 : f ( n ) = O ( n 2 –  ) for  = 1 .  T ( n ) =  ( n 2 ) . Ex. T ( n ) = 4 T ( n /2) + n 2 a = 4 , b = 2  n log b a = n 2 ; f ( n ) = n 2 . C ASE 2 : f ( n ) =  ( n 2 lg 0 n ) , that is, k = 0 .  T ( n ) =  ( n 2 lg n ) .
29. 29. Examples Ex. T ( n ) = 4 T ( n /2) + n 3 a = 4 , b = 2  n log b a = n 2 ; f ( n ) = n 3 . C ASE 3 : f ( n ) =  ( n 2 +  ) for  = 1 and 4( cn /2) 3  cn 3 (reg. cond.) for c = 1/2 .  T ( n ) =  ( n 3 ) . Ex. T ( n ) = 4 T ( n /2) + n 2 /lg n a = 4 , b = 2  n log b a = n 2 ; f ( n ) = n 2 /lg n . Master method does not apply. In particular, for every constant  > 0 , we have n    (lg n ) .
30. 30. Conclusion <ul><li>Next time: applying the master method. </li></ul><ul><li>For proof of master theorem, goto section </li></ul>