Your SlideShare is downloading. ×
L2
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

L2

461
views

Published on

Published in: Education, Business, Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
461
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Introduction to Algorithms 6.046J Lecture 2 Prof. Shafi Goldwasser
  • 2. Solving recurrences
    • The analysis of integer multiplication from Lecture 1 required us to solve a recurrence.
    • Recurrences are a major tool for analysis of algorithms
      • Today: Learn a few methods.
    • Lecture 3 : Divide and Conquer algorithms which are analyzable by recurrences.
  • 3. Recall: Integer Multiplication
    • Let X = A B and Y = C D where A,B,C and D are n/2 bit integers
    • Simple Method : XY = (2 n/2 A+B)(2 n/2 C+D)
    • Running Time Recurrence
    • T(n) < 4T(n/2) + 100n
    • How do we solve it?
  • 4. Substitution method
    • Guess the form of the solution.
    • Verify by induction.
    • Solve for constants.
    The most general method: Example: T ( n ) = 4 T ( n /2) + 100 n
    • [Assume that T (1) =  (1) .]
    • Guess O ( n 3 ) . (Prove O and  separately.)
    • Assume that T ( k )  ck 3 for k < n .
    • Prove T ( n )  cn 3 by induction.
  • 5. Example of substitution desired – residual whenever ( c /2) n 3 – 100 n  0 , for example, if c  200 and n  1 . desired residual 3 3 3 3 3 ) ) 2 / (( ) 2 / ( ) 2 / ( 4 ) 2 / ( 4 ) ( cn 100n n c cn 100n n c 100n n c 100n n T n T          
  • 6. Example (continued)
    • We must also handle the initial conditions, that is, ground the induction with base cases.
    • Base: T ( n ) =  (1) for all n < n 0 , where n 0 is a suitable constant.
    • For 1  n < n 0 , we have “  (1) ”  cn 3 , if we pick c big enough.
    This bound is not tight!
  • 7. A tighter upper bound? We shall prove that T ( n ) = O ( n 2 ) . Assume that T ( k )  ck 2 for k < n : for no choice of c > 0 . Lose! 2 cn   ) 2 / ( 4 ) ( 2 100n cn 100n n T n T    
  • 8. A tighter upper bound!
    • I DEA : Strengthen the inductive hypothesis.
    • Subtract a low-order term.
    Inductive hypothesis : T ( k )  c 1 k 2 – c 2 k for k < n . if c 2 > 100 . Pick c 1 big enough to handle the initial conditions. ) ( 2 )) 2 / ( ) 2 / ( ( 4 ) 2 / ( 4 ) ( 2 2 1 2 2 2 1 2 2 1 2 2 1 n c n c 100n n c n c n c 100n n c n c 100n n c n c 100n n T n T              
  • 9. Recursion-tree method
    • A recursion tree models the costs (time) of a recursive execution of an algorithm.
    • The recursion tree method is good for generating guesses for the substitution method.
    • The recursion-tree method can be unreliable, just like any method that uses ellipses (…).
    • The recursion-tree method promotes intuition, however.
  • 10. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 :
  • 11. Example of recursion tree T ( n ) Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 :
  • 12. Example of recursion tree n 2 Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : T ( n /4) T ( n /2)
  • 13. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : n 2 ( n /4) 2 ( n /2) 2 T ( n /16) T ( n /8) T ( n /8) T ( n /4)
  • 14. Example of recursion tree ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2 ( n /2) 2  (1) … Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : n 2
  • 15. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2 ( n /2) 2  (1) … n 2
  • 16. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2 ( n /2) 2  (1) … n 2
  • 17. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2  (1) … n 2 ( n /2) 2 …
  • 18. Example of recursion tree Solve T ( n ) = T ( n /4) + T ( n/ 2) + n 2 : ( n /16) 2 ( n /8) 2 ( n /8) 2 ( n /4) 2 ( n /4) 2  (1) … … Total = =  ( n 2 ) n 2 ( n /2) 2 geometric series
  • 19. Appendix: geometric series Return to last slide viewed. for | x | < 1 for x  1
  • 20. The master method The master method applies to recurrences of the form T ( n ) = a T ( n / b ) + f ( n ) , where a  1 , b > 1 , and f is asymptotically positive.
  • 21. Idea of master theorem f ( n/b ) f ( n/b ) f ( n/b )   (1) … Recursion tree: … f ( n ) a f ( n/b 2 ) f ( n/b 2 ) f ( n/b 2 ) … a h = log b n f ( n ) a f ( n/b ) a 2 f ( n/b 2 ) … #leaves = a h = a log b n = n log b a n log b a   (1)
  • 22. Three common cases Compare f ( n ) with n log b a :
    • f ( n ) = O ( n log b a –  ) for some constant  > 0 .
      • f ( n ) grows polynomially slower than n log b a (by an n  factor).
    • Solution: T ( n ) =  ( n log b a ) .
  • 23. Idea of master theorem f ( n/b ) f ( n/b ) f ( n/b )   (1) … Recursion tree: … f ( n ) a f ( n/b 2 ) f ( n/b 2 ) f ( n/b 2 ) … a h = log b n f ( n ) a f ( n/b ) a 2 f ( n/b 2 ) … n log b a   (1) C ASE 1 : The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight.  ( n log b a )
  • 24. Three common cases Compare f ( n ) with n log b a :
    • f ( n ) =  ( n log b a lg k n ) for some constant k  0 .
      • f ( n ) and n log b a grow at similar rates.
      • Solution: T ( n ) =  ( n log b a lg k +1 n ) .
  • 25. Idea of master theorem f ( n/b ) f ( n/b ) f ( n/b )   (1) … Recursion tree: … f ( n ) a f ( n/b 2 ) f ( n/b 2 ) f ( n/b 2 ) … a h = log b n f ( n ) a f ( n/b ) a 2 f ( n/b 2 ) … n log b a   (1) C ASE 2 : ( k = 0 ) The weight is approximately the same on each of the log b n levels.  ( n log b a lg n )
  • 26. Three common cases (cont.) Compare f ( n ) with n log b a :
    • f ( n ) =  ( n log b a +  ) for some constant  > 0 .
      • f ( n ) grows polynomially faster than n log b a (by an n  factor),
    • and f ( n ) satisfies the regularity condition that a f ( n/b )  c f ( n ) for some constant c < 1 .
    • Solution: T ( n ) =  ( f ( n ) ) .
  • 27. Idea of master theorem f ( n/b ) f ( n/b ) f ( n/b )   (1) … Recursion tree: … f ( n ) a f ( n/b 2 ) f ( n/b 2 ) f ( n/b 2 ) … a h = log b n f ( n ) a f ( n/b ) a 2 f ( n/b 2 ) … n log b a   (1) C ASE 3 : The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight.  ( f ( n ))
  • 28. Examples Ex. T ( n ) = 4 T ( n /2) + n a = 4 , b = 2  n log b a = n 2 ; f ( n ) = n . C ASE 1 : f ( n ) = O ( n 2 –  ) for  = 1 .  T ( n ) =  ( n 2 ) . Ex. T ( n ) = 4 T ( n /2) + n 2 a = 4 , b = 2  n log b a = n 2 ; f ( n ) = n 2 . C ASE 2 : f ( n ) =  ( n 2 lg 0 n ) , that is, k = 0 .  T ( n ) =  ( n 2 lg n ) .
  • 29. Examples Ex. T ( n ) = 4 T ( n /2) + n 3 a = 4 , b = 2  n log b a = n 2 ; f ( n ) = n 3 . C ASE 3 : f ( n ) =  ( n 2 +  ) for  = 1 and 4( cn /2) 3  cn 3 (reg. cond.) for c = 1/2 .  T ( n ) =  ( n 3 ) . Ex. T ( n ) = 4 T ( n /2) + n 2 /lg n a = 4 , b = 2  n log b a = n 2 ; f ( n ) = n 2 /lg n . Master method does not apply. In particular, for every constant  > 0 , we have n    (lg n ) .
  • 30. Conclusion
    • Next time: applying the master method.
    • For proof of master theorem, goto section