Introduction to Algorithms 6.046J Lecture 2 Prof. Shafi Goldwasser
Solving recurrences <ul><li>The analysis of integer multiplication from  Lecture 1  required us to solve a recurrence. </l...
Recall: Integer Multiplication <ul><li>Let X = A B  and Y = C  D  where A,B,C and D are n/2 bit integers </li></ul><ul><li...
Substitution method <ul><li>Guess  the form of the solution. </li></ul><ul><li>Verify  by induction. </li></ul><ul><li>Sol...
Example of substitution desired   –   residual whenever  ( c /2) n 3  – 100 n     0 , for example, if  c    200  and  n ...
Example (continued) <ul><li>We must also handle the initial conditions, that is, ground the induction with base cases. </l...
A tighter upper bound? We shall prove that  T ( n ) =  O ( n 2 ) . Assume that  T ( k )     ck 2  for  k  <  n : for  no ...
A tighter upper bound! <ul><li>I DEA :  Strengthen the inductive hypothesis. </li></ul><ul><li>Subtract   a low-order term...
Recursion-tree method <ul><li>A recursion tree models the costs (time) of a recursive execution of an algorithm. </li></ul...
Example of recursion tree Solve  T ( n ) =  T ( n /4) +  T ( n/ 2)  + n 2 :
Example of recursion tree T ( n ) Solve  T ( n ) =  T ( n /4) +  T ( n/ 2)  + n 2 :
Example of recursion tree n 2 Solve  T ( n ) =  T ( n /4) +  T ( n/ 2)  + n 2 : T ( n /4) T ( n /2)
Example of recursion tree Solve  T ( n ) =  T ( n /4) +  T ( n/ 2)  + n 2 : n 2 ( n /4) 2 ( n /2) 2 T ( n /16) T ( n /8) T...
Example of recursion tree ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2 ( n /2) 2  (1) … Solve  T ( n ) =  T ( n /4)...
Example of recursion tree Solve  T ( n ) =  T ( n /4) +  T ( n/ 2)  + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /...
Example of recursion tree Solve  T ( n ) =  T ( n /4) +  T ( n/ 2)  + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /...
Example of recursion tree Solve  T ( n ) =  T ( n /4) +  T ( n/ 2)  + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /...
Example of recursion tree Solve  T ( n ) =  T ( n /4) +  T ( n/ 2)  + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /...
Appendix: geometric series Return to last slide viewed. for  | x | < 1 for  x     1
The master method The master method applies to recurrences of the form T ( n ) =  a   T ( n / b ) +  f   ( n )  ,  where  ...
Idea of master theorem f   ( n/b ) f   ( n/b ) f   ( n/b )   (1) … Recursion tree: … f   ( n ) a f   ( n/b 2 ) f   ( n/b...
Three common cases Compare  f   ( n )  with  n log b a : <ul><li>f   ( n ) =  O ( n log b a  –   )  for some constant   ...
Idea of master theorem f   ( n/b ) f   ( n/b ) f   ( n/b )   (1) … Recursion tree: … f   ( n ) a f   ( n/b 2 ) f   ( n/b...
Three common cases Compare  f   ( n )  with  n log b a : <ul><li>f   ( n ) =   ( n log b a  lg k n )  for some constant  ...
Idea of master theorem f   ( n/b ) f   ( n/b ) f   ( n/b )   (1) … Recursion tree: … f   ( n ) a f   ( n/b 2 ) f   ( n/b...
Three common cases (cont.) Compare  f   ( n )  with  n log b a : <ul><li>f   ( n ) =   ( n log b a  +   )  for some cons...
Idea of master theorem f   ( n/b ) f   ( n/b ) f   ( n/b )   (1) … Recursion tree: … f   ( n ) a f   ( n/b 2 ) f   ( n/b...
Examples Ex.  T ( n ) = 4 T ( n /2) +  n a =  4 ,  b  = 2     n log b a  =   n 2 ;  f   ( n ) =  n . C ASE  1 :  f   ( n ...
Examples Ex.  T ( n ) = 4 T ( n /2) +  n 3 a =  4 ,  b  = 2     n log b a  =   n 2 ;  f   ( n ) =  n 3 .   C ASE  3 :  f ...
Conclusion <ul><li>Next time: applying the master method. </li></ul><ul><li>For proof of master theorem, goto section </li...
Upcoming SlideShare
Loading in …5
×

L2

647 views
550 views

Published on

Published in: Education, Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
647
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

L2

  1. 1. Introduction to Algorithms 6.046J Lecture 2 Prof. Shafi Goldwasser
  2. 2. Solving recurrences <ul><li>The analysis of integer multiplication from Lecture 1 required us to solve a recurrence. </li></ul><ul><li>Recurrences are a major tool for analysis of algorithms </li></ul><ul><ul><li>Today: Learn a few methods. </li></ul></ul><ul><li>Lecture 3 : Divide and Conquer algorithms which are analyzable by recurrences. </li></ul>
  3. 3. Recall: Integer Multiplication <ul><li>Let X = A B and Y = C D where A,B,C and D are n/2 bit integers </li></ul><ul><li>Simple Method : XY = (2 n/2 A+B)(2 n/2 C+D) </li></ul><ul><li>Running Time Recurrence </li></ul><ul><li>T(n) < 4T(n/2) + 100n </li></ul><ul><li>How do we solve it? </li></ul>
  4. 4. Substitution method <ul><li>Guess the form of the solution. </li></ul><ul><li>Verify by induction. </li></ul><ul><li>Solve for constants. </li></ul>The most general method: Example: T ( n ) = 4 T ( n /2) + 100 n <ul><li>[Assume that T (1) =  (1) .] </li></ul><ul><li>Guess O ( n 3 ) . (Prove O and  separately.) </li></ul><ul><li>Assume that T ( k )  ck 3 for k < n . </li></ul><ul><li>Prove T ( n )  cn 3 by induction. </li></ul>
  5. 5. Example of substitution desired – residual whenever ( c /2) n 3 – 100 n  0 , for example, if c  200 and n  1 . desired residual 3 3 3 3 3 ) ) 2 / (( ) 2 / ( ) 2 / ( 4 ) 2 / ( 4 ) ( cn 100n n c cn 100n n c 100n n c 100n n T n T          
  6. 6. Example (continued) <ul><li>We must also handle the initial conditions, that is, ground the induction with base cases. </li></ul><ul><li>Base: T ( n ) =  (1) for all n < n 0 , where n 0 is a suitable constant. </li></ul><ul><li>For 1  n < n 0 , we have “  (1) ”  cn 3 , if we pick c big enough. </li></ul>This bound is not tight!
  7. 7. A tighter upper bound? We shall prove that T ( n ) = O ( n 2 ) . Assume that T ( k )  ck 2 for k < n : for no choice of c > 0 . Lose! 2 cn   ) 2 / ( 4 ) ( 2 100n cn 100n n T n T    
  8. 8. A tighter upper bound! <ul><li>I DEA : Strengthen the inductive hypothesis. </li></ul><ul><li>Subtract a low-order term. </li></ul>Inductive hypothesis : T ( k )  c 1 k 2 – c 2 k for k < n . if c 2 > 100 . Pick c 1 big enough to handle the initial conditions. ) ( 2 )) 2 / ( ) 2 / ( ( 4 ) 2 / ( 4 ) ( 2 2 1 2 2 2 1 2 2 1 2 2 1 n c n c 100n n c n c n c 100n n c n c 100n n c n c 100n n T n T              
  9. 9. Recursion-tree method <ul><li>A recursion tree models the costs (time) of a recursive execution of an algorithm. </li></ul><ul><li>The recursion tree method is good for generating guesses for the substitution method. </li></ul><ul><li>The recursion-tree method can be unreliable, just like any method that uses ellipses (…). </li></ul><ul><li>The recursion-tree method promotes intuition, however. </li></ul>
  10. 10. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 :
  11. 11. Example of recursion tree T ( n ) Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 :
  12. 12. Example of recursion tree n 2 Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : T ( n /4) T ( n /2)
  13. 13. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : n 2 ( n /4) 2 ( n /2) 2 T ( n /16) T ( n /8) T ( n /8) T ( n /4)
  14. 14. Example of recursion tree ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2 ( n /2) 2  (1) … Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : n 2
  15. 15. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2 ( n /2) 2  (1) … n 2
  16. 16. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2 ( n /2) 2  (1) … n 2
  17. 17. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2  (1) … n 2 ( n /2) 2 …
  18. 18. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2  (1) … … Total = =  ( n 2 ) n 2 ( n /2) 2 geometric series
  19. 19. Appendix: geometric series Return to last slide viewed. for | x | < 1 for x  1
  20. 20. The master method The master method applies to recurrences of the form T ( n ) = a T ( n / b ) + f ( n ) , where a  1 , b > 1 , and f is asymptotically positive.
  21. 21. Idea of master theorem f ( n/b ) f ( n/b ) f ( n/b )   (1) … Recursion tree: … f ( n ) a f ( n/b 2 ) f ( n/b 2 ) f ( n/b 2 ) … a h = log b n f ( n ) a f ( n/b ) a 2 f ( n/b 2 ) … #leaves = a h = a log b n = n log b a n log b a   (1)
  22. 22. Three common cases Compare f ( n ) with n log b a : <ul><li>f ( n ) = O ( n log b a –  ) for some constant  > 0 . </li></ul><ul><ul><li>f ( n ) grows polynomially slower than n log b a (by an n  factor). </li></ul></ul><ul><li>Solution: T ( n ) =  ( n log b a ) . </li></ul>
  23. 23. Idea of master theorem f ( n/b ) f ( n/b ) f ( n/b )   (1) … Recursion tree: … f ( n ) a f ( n/b 2 ) f ( n/b 2 ) f ( n/b 2 ) … a h = log b n f ( n ) a f ( n/b ) a 2 f ( n/b 2 ) … n log b a   (1) C ASE 1 : The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight.  ( n log b a )
  24. 24. Three common cases Compare f ( n ) with n log b a : <ul><li>f ( n ) =  ( n log b a lg k n ) for some constant k  0 . </li></ul><ul><ul><li>f ( n ) and n log b a grow at similar rates. </li></ul></ul><ul><ul><li>Solution: T ( n ) =  ( n log b a lg k +1 n ) . </li></ul></ul>
  25. 25. Idea of master theorem f ( n/b ) f ( n/b ) f ( n/b )   (1) … Recursion tree: … f ( n ) a f ( n/b 2 ) f ( n/b 2 ) f ( n/b 2 ) … a h = log b n f ( n ) a f ( n/b ) a 2 f ( n/b 2 ) … n log b a   (1) C ASE 2 : ( k = 0 ) The weight is approximately the same on each of the log b n levels.  ( n log b a lg n )
  26. 26. Three common cases (cont.) Compare f ( n ) with n log b a : <ul><li>f ( n ) =  ( n log b a +  ) for some constant  > 0 . </li></ul><ul><ul><li>f ( n ) grows polynomially faster than n log b a (by an n  factor), </li></ul></ul><ul><li>and f ( n ) satisfies the regularity condition that a f ( n/b )  c f ( n ) for some constant c < 1 . </li></ul><ul><li>Solution: T ( n ) =  ( f ( n ) ) . </li></ul>
  27. 27. Idea of master theorem f ( n/b ) f ( n/b ) f ( n/b )   (1) … Recursion tree: … f ( n ) a f ( n/b 2 ) f ( n/b 2 ) f ( n/b 2 ) … a h = log b n f ( n ) a f ( n/b ) a 2 f ( n/b 2 ) … n log b a   (1) C ASE 3 : The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight.  ( f ( n ))
  28. 28. Examples Ex. T ( n ) = 4 T ( n /2) + n a = 4 , b = 2  n log b a = n 2 ; f ( n ) = n . C ASE 1 : f ( n ) = O ( n 2 –  ) for  = 1 .  T ( n ) =  ( n 2 ) . Ex. T ( n ) = 4 T ( n /2) + n 2 a = 4 , b = 2  n log b a = n 2 ; f ( n ) = n 2 . C ASE 2 : f ( n ) =  ( n 2 lg 0 n ) , that is, k = 0 .  T ( n ) =  ( n 2 lg n ) .
  29. 29. Examples Ex. T ( n ) = 4 T ( n /2) + n 3 a = 4 , b = 2  n log b a = n 2 ; f ( n ) = n 3 . C ASE 3 : f ( n ) =  ( n 2 +  ) for  = 1 and 4( cn /2) 3  cn 3 (reg. cond.) for c = 1/2 .  T ( n ) =  ( n 3 ) . Ex. T ( n ) = 4 T ( n /2) + n 2 /lg n a = 4 , b = 2  n log b a = n 2 ; f ( n ) = n 2 /lg n . Master method does not apply. In particular, for every constant  > 0 , we have n    (lg n ) .
  30. 30. Conclusion <ul><li>Next time: applying the master method. </li></ul><ul><li>For proof of master theorem, goto section </li></ul>

×