Your SlideShare is downloading. ×
0
Analysis of Algorithms II The Recursive Case Except as otherwise noted, the content of this presentation is licensed under...
Key Topics * Recurrence Relations * Solving Recurrence Relations * The Towers of Hanoi * Analyzing Recursive Subprograms
Recurrence Relations <ul><li>Recall that a recursive or inductive definition has the following parts: </li></ul><ul><li>Ba...
Example 1 <ul><li>a) One way to solve this problem is enumeration: </li></ul><ul><li>length 0: empty string 1 </li></ul><u...
  A Recurrence Relation: 1) s 0  = 1   s 1  = 2 2) s  k  = s k-1  + s k-2 If we calculate this recurrence to s 10 , we get...
Example 2 <ul><li>Consider the following recurrence relation: </li></ul><ul><li>a 0  = 1 </li></ul><ul><li>a  k  = a  k-1 ...
Proof: Let P(n) denote the following: if a 1 , a 2 , a 3 , ..., a n  is the sequence defined by:  A  0  = 1 A  k  = a k-1 ...
Example 3 Enumerating the first few values of this sequence yields: K2 = K1 + 1 = 0 + 1 = 1 K3 = K2 + 2 = 1 + 2 = 3 K4 = K...
Example 4 The object of the  Towers of Hanoi  is to move all the disks from the first peg to the third peg while always fo...
The n – 1 disks on the auxiliary peg are moved to the ending peg, requiring an additional H n-1  moves.  This gives us the...
Example 5 Find a closed form solution for the following recurrence relations.  For brevity, we omit the inductive proofs. ...
b) Recurrence relation : a 0  = 1 a n  = 2a n-1  – 1 Solution : a n  = 2(2a n-2  – 1) – 1 = 2 2  a n-2  – 2 – 1 a n  = 2 2...
A  recurrence relation  expresses the running time of a recursive algorithm.  This includes how many recursive calls are g...
Example 6 Consider the following function to compute factorials: int factorial(int n) { 1) if (n <= 1) 2)   return(1); els...
Proof: Let P(n) denote that the running time of  factorial  is T(n) = a + (n-1)b.  And, recall that the recurrence relatio...
Example 7 Consider the following recursive selection sort algorithm: void SelectionSort(int A[], int i, int n) {   int j, ...
The inductive definition of recursive SelectionSort is:   T(1) = a T(m) = T(m-1) + O(m)   To solve this recurrence relatio...
To get a closed form formula we let j = m – 1 T(m)  = T(1) + (m-1)Cm - ((m-1)(m-2)/2)C = a + m 2 C - Cm - (m 2 C - 3Cm + 2...
Example 8 MergeSort(L) 1)  if (length of L > 1) { 2) Split list into first half and second half 3) MergeSort(first half) 4...
To analyze the complexity of  MergeSort , we need to define a recurrence relation.  The base case is when we have a list o...
There is obviously a pattern but it is not as easy to represent as the others have been.  Note the following relationships...
Theorem 1 (Master Theorem) :  Let  f  be an increasing function that satisfies the recurrence relation: f (n) = a  f (n/b)...
Example 9 Problem : Use Theorem 1 above to find the big-Oh running time of MergeSort (from Example 9). Solution : In Examp...
Consider the three cases of the  Master Theorem , which are based on the relationship between a and bd.  Here are some exa...
Example 10 <ul><li>Problem :  </li></ul><ul><li>Use repeated substitution to find the time complexity of the function  rec...
We use these two points to perform the recurrence relation analysis below: From the base case, we have: T 0  = a From the ...
We need to compute the closed form for  (2 n-2  + 2 n-4  + … + 2 0 ) = (2 n  – 1)/3 . Here is a brief digression on how we...
Base case:  Show P(0).T 0  =  2 0  a + ((2 0  – 1)/3) b = a + ((1 – 1)/3) b = a.   This is the initial condition. Inductiv...
Upcoming SlideShare
Loading in...5
×

Analysis Of Algorithms Ii

907

Published on

Published in: Education, Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
907
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
60
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Analysis Of Algorithms Ii"

  1. 1. Analysis of Algorithms II The Recursive Case Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution 2.5 License.
  2. 2. Key Topics * Recurrence Relations * Solving Recurrence Relations * The Towers of Hanoi * Analyzing Recursive Subprograms
  3. 3. Recurrence Relations <ul><li>Recall that a recursive or inductive definition has the following parts: </li></ul><ul><li>Base Case : the initial condition or basis which defines the first (or first few) elements of the sequence </li></ul><ul><li>2. Inductive (Recursive) Case : an inductive step in which later terms in the sequence are defined in terms of earlier terms. </li></ul>A recurrence relation for a sequence a 1 , a 2 , a 3 , ... is a formula that relates each term a k to certain of its predecessors a k-1 , a k-2 , ..., a k-i , where i is a fixed integer and k is any integer greater than or equal to i . The initial conditions for such a recurrence relation specify the values of a 1 , a 2 , a 3 , ..., a i-1 .
  4. 4. Example 1 <ul><li>a) One way to solve this problem is enumeration: </li></ul><ul><li>length 0: empty string 1 </li></ul><ul><li>length 1: 0, 1 2 </li></ul><ul><li>length 2: 00, 01, 10, 11 3 </li></ul><ul><li>length 3: 000, 001, 010, 011, 100, 101, 110, 111 5 </li></ul><ul><li>b) Suppose the number of bit strings of length less than some integer k that do not contain the 11 pattern is known (it just so happens that we do know it for 0, 1, 2, 3). To find the number of strings of length k that do not contain the bit pattern 11, we can try to express strings that have a certain property in terms of shorter strings with the same property. This is called recursive thinking. </li></ul>Set of all bit strings of form 0 ----- K-1 bits without 11 Set of all bit strings of form 10 --- k-2 bits without 11
  5. 5. A Recurrence Relation: 1) s 0 = 1 s 1 = 2 2) s k = s k-1 + s k-2 If we calculate this recurrence to s 10 , we get 144
  6. 6. Example 2 <ul><li>Consider the following recurrence relation: </li></ul><ul><li>a 0 = 1 </li></ul><ul><li>a k = a k-1 + 2 </li></ul><ul><li>Problem : What is the formula for this recurrence relation? </li></ul><ul><li>We start by substituting k-1 into the recursive definition above to obtain: </li></ul><ul><li>a k-1 = a k-2 +2 </li></ul><ul><li>What we are looking for is a lot of different representations of the formula for ak based on </li></ul><ul><li>previous elements in the sequence: a k = a k-1 + 2 = (a k-2 + 2) + 2 = a k-2 + 4 </li></ul><ul><li>we substitute k-2 for k in the original equation to get a k-2 = a k-3 + 2 and then substitute this value for a k-2 in the previous expression: </li></ul><ul><li>a k = a k-2 + 4 = (a k-3 + 2) + 4 = a k-3 + 6 </li></ul><ul><li>a k = a k-3 + 6 = (a k-4 + 2) + 6 = a k-4 + 8 </li></ul><ul><li>a k = a k-4 + 8.... </li></ul>
  7. 7. Proof: Let P(n) denote the following: if a 1 , a 2 , a 3 , ..., a n is the sequence defined by: A 0 = 1 A k = a k-1 + 2 then, a n = 1 + 2n for all n >= 1 Base Case : prove that P(1) is true. Using the definition of the recurrence relation, we have: a 1 = a 0 + 2 = 1 + 2 = 3. Now, we show that we get the same answer using the formula: a n = 1 + 2n for n = 1. a n = 1 + 2n = 1 + 2 = 3. Therefore, the base case holds. Inductive Case: The induction hypothesis is to assume P(k): a k = 1 + 2k. We then need to show that P(k+1) = a k+1 = 1 + 2(k + 1). a k+1 = a ((k+1)–1) + 2 from the recurrence relation (this is a &quot;given&quot;) a k+1 = a k + 2 algebra a k+1 = 1 + 2k + 2 substitution from inductive hypothesis (a k = 1 + 2k) a k+1 = 1 + 2(k + 1) algebra
  8. 8. Example 3 Enumerating the first few values of this sequence yields: K2 = K1 + 1 = 0 + 1 = 1 K3 = K2 + 2 = 1 + 2 = 3 K4 = K3 + 3 = 3 + 3 = 6 K5 = K4 + 4 = 6 + 4 = 10 From the recurrence relation, we can see that this sequence may be given by the formula (for all n >= 1): n-1 K n = 0 + 1 + 2 + ... + (n – 1) =  i = n(n – 1)/2 i=0
  9. 9. Example 4 The object of the Towers of Hanoi is to move all the disks from the first peg to the third peg while always following these rules: 1. Only one disk may be moved at a time (specifically, the top disk on any peg). 2. At no time can a larger disk be placed on a smaller one. Tower of Hanoi
  10. 10. The n – 1 disks on the auxiliary peg are moved to the ending peg, requiring an additional H n-1 moves. This gives us the recurrence relation : H n = 2H n-1 + 1 The base case for this relations is H 0 = 0, since if there are no disks, there are no moves required to have “all” the disks on the third peg. To determine a closed form formula where there is no H n on the right side of the equation, we will try repeated substitution: H n = 2H n-1 + 1 = 2(2H n-2 + 1) + 1 = 2 2 * H n-2 + 2 + 1 = 2 2 (2H n-3 + 1) + 2 + 1 = 2 3 * H n-3 + 2 2 + 2 + 1 The pattern to this recurrence relation is: H n = (2 i * H n-i ) + 2 i-1 + 2 i-2 + ... + 2 2 + 2 1 + 2 0 We can set n = i (this will require an inductive proof): H n = (2 n * H 0 ) + 2 n-1 + 2 n-2 + ... + 2 2 + 2 1 + 1 Substituting H0 = 0, yields the formula: H n = 2 n-1 + 2 n-2 + ... + 2 2 + 2 1 + 1
  11. 11. Example 5 Find a closed form solution for the following recurrence relations. For brevity, we omit the inductive proofs. a) Recurrence relation : a 0 = 2 a n = 3a n-1 Solution : a n = 3a n-1 a n = 3 (3a n-2 ) = 3 2 a n-2 a n = 3 2 (3a n-3 ) = 3 3 a n-3 a n = 3 i-1 (3a n-i ) = 3 i a n-I Set i = n, yielding: a n = 3 n a 0 a n = 2 * 3 n
  12. 12. b) Recurrence relation : a 0 = 1 a n = 2a n-1 – 1 Solution : a n = 2(2a n-2 – 1) – 1 = 2 2 a n-2 – 2 – 1 a n = 2 2 (2a n-3 – 1) – 2 – 1 = 2 3 a n-3 – 2 2 – 2 – 1 a n = 2 i (a n-i ) – 2 i-1 – 2 i-2 – … – 2 0 Set i = n, yielding: a n = 2 n (a 0 ) – 2 n-1 – 2 n-2 – … – 2 0 = 2 n – 2 n-1 – 2 n-2 – … – 2 0 a n = 2 n – (2 n – 1) (Note: 2 n-1 – 2 n-2 – … – 1 = 2 n – 1) a n = 1 c) Recurrence relation: a 0 = 5 a n = na n-1 Solution : a n = n(n-1)a n-2 a n = n(n-1)(n-2)a n-3 a n = n(n-1)(n-2)…(n-i+1)a n-I Set i = n, yielding: a n = n(n-1)(n-2)…(1) a 0 = a 0 n! a n = 5n!
  13. 13. A recurrence relation expresses the running time of a recursive algorithm. This includes how many recursive calls are generated at each level of the recursion, how much of the problem is solved by each recursive call, and how much work is done at each level.
  14. 14. Example 6 Consider the following function to compute factorials: int factorial(int n) { 1) if (n <= 1) 2) return(1); else 3) return(n * factorial(n-)); } Recurrence relation: base case: T(1) = a induction: T(n) = b + T(n-1), for n > 1 Closed form formula: T(1) = a T(2) = b + T(1) = b + a T(3) = b + T(2) = b + (b + a) = a + 2b T(4) = b + T(3) = b + (a + 2b) = a + 3b T(n) = a + (n-1)b for all n >= 1
  15. 15. Proof: Let P(n) denote that the running time of factorial is T(n) = a + (n-1)b. And, recall that the recurrence relation definition give us: T(1) = a, and T(n) = b + T(n-1) for n > 1. Base case : prove that P(1) is true T(1) = a + ((1 – 1) * b) = a + (0 * b) = a. This equation holds since the base case of our inductive definition states that T(1) = a. Inductive case : inductive hypothesis is to assume P(k): T(k) = a + (k-1)b, and show P(k+1): T(k+1) = a + kb. We know from the recurrence relation definition that T(k+1) = b + T(k). We use the inductive hypothesis to substitute for T(k). This gives us: T(k+1) = b + T(k) = b + a + (k-1)b = b + a + kb - b = a + kb P(k+1) holds when P(k) holds, and P(1) holds, therefore P(n) is true for all n >= 1.
  16. 16. Example 7 Consider the following recursive selection sort algorithm: void SelectionSort(int A[], int i, int n) { int j, small, temp; 1) if (i < n) { 2) small = i; 3) for (j = i + 1, j <= n, j++) { 4) if (A[j] < A[small]) small = j; } 6) temp = A[small]; 7) A[small] = A[i]; 8) A[i] = temp; 9) SelectionSort(A, i + 1, n); } }
  17. 17. The inductive definition of recursive SelectionSort is: T(1) = a T(m) = T(m-1) + O(m) To solve this recurrence relation, we first get rid of the big-Oh expression by substituting the definition of big-Oh: (f(n) = O(g(n)) if f(n) <= C * g(n)), so we can substitute C*m for O(m): T(1) = a T(m) = T(m-1) + C*m Now, we can either try repeated substitutions or just enumerate a few cases to look for a pattern. Let’s try repeated substitution: T(m) = T(m-1) + C*m = T(m-2) + 2Cm - C because T(m-1) = T(m-2) + C(m-1) = T(m-3) + 3Cm - 3C because T(m-2) = T(m-3) + C(m-2) = T(m-4) + 4Cm - 6C because T(m-3) = T(m-4) + C(m-3) = T(m-5) + 5Cm - 10C because T(m-4) = T(m-5) + C(m-4) ... = T(m-j) + jCm - (j(j-1)/2)C
  18. 18. To get a closed form formula we let j = m – 1 T(m) = T(1) + (m-1)Cm - ((m-1)(m-2)/2)C = a + m 2 C - Cm - (m 2 C - 3Cm + 2C)/2 = a + (2m 2 C - 2Cm - m 2 C + 3Cm - 2C)/2 = a + (m 2 C + Cm - 2C)/2 Closed form formula T(m) = a + (m 2 C + Cm - 2C)/2
  19. 19. Example 8 MergeSort(L) 1) if (length of L > 1) { 2) Split list into first half and second half 3) MergeSort(first half) 4) MergeSort(second half) 5) Merge first half and second half into sorted list }
  20. 20. To analyze the complexity of MergeSort , we need to define a recurrence relation. The base case is when we have a list of 1 element and only line 1 of the function is executed. Thus, the base case is constant time, O(1). If the test of line 1 fails, we must execute lines 2-5. The time spent in this function when the length of the list > 1 is the sum of the following: 1) O(1) for the test on line 1 2) O(n) for the split function call on line 2 3) T(n/2) for recursive call on line 3 4) T(n/2) for recursive call on line 4 5) O(n) for the merge function call on line 5 If we drop the O(1)'s and apply the summation rule, we get 2T(n/2) + O(n). If we substitute constants in place of the big-Oh notation we obtain: T(1) = a T(n) = 2T(n/2) + bn To solve this recurrence relation, we will enumerate a few values. We will stick to n's that are powers of two so things divide evenly: T(2) = 2T(1) + 2b = 2a + 2b T(4) = 2T(2) + 4b = 2(2a + 2b) + 4b = 4a + 8b T(8) = 2T(4) + 8b = 2(4a + 8b) + 8b = 8a + 24b T(16) = 2T(8) + 16b = 2(8a + 24b) + 16b = 16a + 64b
  21. 21. There is obviously a pattern but it is not as easy to represent as the others have been. Note the following relationships: value of n: 2 4 8 16 coefficient of b: 2 8 24 64 ratio: 1 2 3 4 So, it appears that the coefficient of b is n times another factor that grows by 1 each time n doubles. The ratio is log 2 n because log 2 2 = 1, log 2 4 = 2, log 2 8 = 3 , etc. Our &quot;guess&quot; for the solution of this recurrence relation is T(n) = an + bn log 2 n . We could have used repeated substitution in which case we would have the following formula: T(n) = 2 i T(n/2 i ) + ibn Now, if we let i = log 2 n, we end up with: n*T(1) + bn log 2 n = an + bn log 2 n (because 2 log2n = n).
  22. 22. Theorem 1 (Master Theorem) : Let f be an increasing function that satisfies the recurrence relation: f (n) = a f (n/b) + cn d whenever n = b k , where k is a positive integer, a >= 1, b is an integer greater than 1, c is a positive real number, and d is a non-negative real number. Then: O(n d ) if a < b d f (n) = O(n d log n) if a = b d O(n log a ) if a > b d
  23. 23. Example 9 Problem : Use Theorem 1 above to find the big-Oh running time of MergeSort (from Example 9). Solution : In Example 9, we are given the recurrence relation for MergeSort as: T(n) = 2T(n/2) + xn where we have simply replaced the positive constant b in the original recurrence relation with the constant x, so that we do not confuse variable names below. Applying Theorem 1, we choose these constants: a = 2, b = 2, c = x, and d = 1 and it becomes: f (n) = 2 f (n/2) + xn 1
  24. 24. Consider the three cases of the Master Theorem , which are based on the relationship between a and bd. Here are some examples to consider: 1. a < b d : f(n) = 2 f(n/2) + xn 2 . In this example, 2 < 2 2 , and thus f(n) = O(n 2 ). The xn 2 term is growing faster than the 2 f(n/2) and thus dominates. 2. a = b d : f(n) = 2 f(n/2) + xn 1 . In this example, 2 = 2, and thus f(n) = O(n log n). The two terms grow together. 3. a > b d : f(n) = 8 f(n/2) + xn 2 . In this example, 8 < 2 2 , and thus f(n) = O(n log 8 ) = O(n 3 ). The 8 f(n/2) term is growing faster than the xn 2 and thus dominates.
  25. 25. Example 10 <ul><li>Problem : </li></ul><ul><li>Use repeated substitution to find the time complexity of the function recurse . Verify your result using induction. </li></ul><ul><li>/* Assume only non-negative even values of n are passed in */ </li></ul><ul><li>void recurse(int n) { </li></ul><ul><li> int i, total = 0; </li></ul><ul><li>1) if (n == 0) return 1; </li></ul><ul><li>2) for (i = 0; i < 4; i++) { </li></ul><ul><li>3) total += recurse(n-2); </li></ul><ul><li>} </li></ul><ul><li>4) return total; </li></ul><ul><li>} </li></ul><ul><li>Solution: </li></ul><ul><li>When n = 0 (the base case), this function only executes line 1, doing O(1) work. We will note this constant amount of work as a . </li></ul><ul><li>When n >= 2 (the recursive case), this function does O(1) work in executing lines 1, 2, and 4, which we will denote by b, and also makes four recursive calls with parameter values n-2 in the body loop (line 3). </li></ul>
  26. 26. We use these two points to perform the recurrence relation analysis below: From the base case, we have: T 0 = a From the recursive case, we have: T n = 4T n-2 + b We apply repeated substitution: T n = 4T n-2 + b T n = 4(4T n-4 + b) + b = 4 2 T n-4 + 4b + b T n = 4 2 (4T n-6 + b) + 4b + b = 4 3 T n-6 + 4 2 b + 4b + b T n = 4 i T n-2i + 4 i-1 b + 4 i-2 b + … + 4 0 b Note that 4 k = 2 2k , yielding: T n = 2 2i T n-2i + 2 2 ( i-1 ) b + 2 2(i-2) b + … + 2 0 b Set i = n/2, yielding: T n = 2 n T 0 + 2 n-2 b + 2 n-4 b + … + 2 0 b = 2 n a + (2 n-2 + 2 n-4 + … + 2 0 ) b
  27. 27. We need to compute the closed form for (2 n-2 + 2 n-4 + … + 2 0 ) = (2 n – 1)/3 . Here is a brief digression on how we compute this closed form formula: Let x = (2 n-2 + 2 n-4 + … + 2 0 ) So, 2 2 x = 4x = 2 2 (2 n-2 + 2 n-4 + … + 2 0 ) = (2 n + 2 n-2 + … + 2 2 ) 2 2 x – x = 3x = (2 n + 2 n-2 + … + 2 2 ) – (2 n-2 + 2 n-4 + … + 2 0 ) 3x = 2 n – 2 0 = 2 n – 1 x = (2 n – 1)/3 = (2 n-2 + 2 n-4 + … + 2 0 )
  28. 28. Base case: Show P(0).T 0 = 2 0 a + ((2 0 – 1)/3) b = a + ((1 – 1)/3) b = a. This is the initial condition. Inductive case: Assume inductive hypothesis P(n): T n = 2 n a + ((2 n – 1)/3) b Show P(n+2): T n+2 = 2 n+2 a + ((2 n+2 – 1)/3) b Note: only consider even values of n T n+2 = 4T n + b by definition of the recurrence T n+2 = 4(2 n a + ((2 n – 1)/3) b) + b substituting T n with the inductive hypothesis T n+2 = 2 2 (2 n a + ((2 n – 1)/3) b) + b algebra: 4 = 2 2 T n+2 = 2 n+2 a + ((2 n+2 – 2 2 )/3) b + b algebra: multiply in the 2 2 T n+2 = 2 n+2 a + ((2 n+2 – 3 – 1)/3) b + b algebra: -2 2 = -3 - 1 T n+2 = 2 n+2 a + ((2 n+2 – 1)/3) b – b + b algebra: (-3/3)b = -b T n+2 = 2 n+2 a + ((2 n+2 – 1)/3) b this is the desired result QED. We calculate the big-Oh complexity for the function, as follows: From the recurrence relation T n = 2 n a + ((2 n – 1)/3) b , we see that the algorithm has time complexity: O(2 n a) + O(2 n b) = O(2 n ) + O(2 n ) = O(2 n ).
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×