Introduction to Algorithms

Slide 1: Algorithm Analysis & Data Structures Jaideep Srivastava

Slide 2: Schedule of topics s Lecture 1: Algorithm analysis  Concept – what is it?  Importance – why do it?  Examples – lots of it  Formalism s Lecture 2: Recursion  Concept  Examples s Lecture 3: Trees  Concept & properties  Tree algorithms 2

Slide 3: Introduction A famous quote: Program = Algorithm + Data Structure. s s All of you have programmed; thus have already been exposed to algorithms and data structure. s Perhaps you didn't see them as separate entities; s Perhaps you saw data structures as simple programming constructs (provided by STL--standard template library). s However, data structures are quite distinct from algorithms, and very important in their own right. 3

Slide 4: Lecture 1 – Algorithm Analysis & Complexity

Slide 5: Objectives The main focus of is to introduce you to a systematic study s of algorithms and data structures. s The two guiding principles of the course are: abstraction and formal analysis. s Abstraction: We focus on topics that are broadly applicable to a variety of problems. s Analysis: We want a formal way to compare two objects (data structures or algorithms). s In particular, we will worry about "always correct"-ness, and worst-case bounds on time and memory (space). 5

Slide 6: What is Algorithm Analysis For Foundations of Algorithm Analysis and Data Structures. s Analysis: s  How to predict an algorithm’s performance  How well an algorithm scales up  How to compare different algorithms for a problem Data Structures s  How to efficiently store, access, manage data  Data structures effect algorithm’s performance 6

Slide 7: Example Algorithms Two algorithms for computing the Factorial s s Which one is better? int factorial (int n) { s if (n <= 1) return 1; else return n * factorial(n-1); } int factorial (int n) { s if (n<=1) return 1; else { fact = 1; for (k=2; k<=n; k++) fact *= k; return fact; } } 7

Slide 8: Examples of famous algorithms Constructions of Euclid s s Newton's root finding s Fast Fourier Transform (signal processing) s Compression (Huffman, Lempel-Ziv, GIF, MPEG) s DES, RSA encryption (network security) s Simplex algorithm for linear programming (optimization) s Shortest Path Algorithms (Dijkstra, Bellman-Ford) s Error correcting codes (CDs, DVDs) s TCP congestion control, IP routing (computer networks) s Pattern matching (Genomics) s Search Engines (www) 8

Slide 9: Role of Algorithms in Modern World Enormous amount of data s  Network traffic (telecom billing, monitoring)  Database transactions (Sales, inventory)  Scientific measurements (astrophysics, geology)  Sensor networks. RFID tags  Radio frequency identification (RFID) is a method of remotely storing and retrieving data using devices called RFID tags.  Bioinformatics (genome, protein bank) 9

Slide 10: A real-world Problem Communication in the Internet s s Message (email, ftp) broken down into IP packets. s Sender/receiver identified by IP address. s The packets are routed through the Internet by special computers called Routers. s Each packet is stamped with its destination address, but not the route. s Because the Internet topology and network load is constantly changing, routers must discover routes dynamically. s What should the Routing Table look like? 10

Slide 11: IP Prefixes and Routing Each router is really a switch: it receives packets at several s input ports, and appropriately sends them out to output ports. s Thus, for each packet, the router needs to transfer the packet to that output port that gets it closer to its destination. s Should each router keep a table: IP address x Output Port? s How big is this table? s When a link or router fails, how much information would need to be modified? s A router typically forwards several million packets/sec! 11

Slide 12: Data Structures The IP packet forwarding is a Data Structure problem! s s Efficiency, scalability is very important. Similarly, how does Google find the documents matching your s query so fast? s Uses sophisticated algorithms to create index structures, which are just data structures. s Algorithms and data structures are ubiquitous. s With the data glut created by the new technologies, the need to organize, search, and update MASSIVE amounts of information FAST is more severe than ever before. 12

Slide 13: Algorithms to Process these Data Which are the top K sellers? s Correlation between time spent at a web site and purchase s amount? Which flows at a router account for > 1% traffic? s Did source S send a packet in last s seconds? s Send an alarm if any international arrival matches a profile s in the database Similarity matches against genome databases s Etc. s 13

Slide 14: Max Subsequence Problem Given a sequence of integers A1, A2, …, An, find the maximum possible s value of a subsequence Ai, …, Aj. Numbers can be negative. s You want a contiguous chunk with largest sum. s Example: -2, 11, -4, 13, -5, -2 s The answer is 20 (subseq. A2 through A4). s We will discuss 4 different algorithms, with time complexities O(n3), s O(n2), O(n log n), and O(n). With n = 106, algorithm 1 may take > 10 years; algorithm 4 will take a s fraction of a second! 14

Slide 15: Algorithm 1 for Max Subsequence Sum Given A1,…,An , find the maximum value of Ai+Ai+1+···+Aj s 0 if the max value is negative int maxSum = 0; O (1) for( int i = 0; i < a.size( ); i++ ) for( int j = i; j < a.size( ); j++ ) { O (1) n 1 n 1 n 1 int thisSum = 0; O( ( j  i )) O( ( j  i )) for( int k = i; k <= j; k++ ) O( j  i ) j i i  0 j i O (1) thisSum += a[ k ]; if( thisSum > maxSum ) O (1) maxSum = thisSum; } return maxSum; complexity: On3 s Time 15

Slide 16: Algorithm 2 s Idea: Given sum from i to j-1, we can compute the sum from i to j in constant time. s This eliminates one nested loop, and reduces the running time to O(n2). into maxSum = 0; for( int i = 0; i < a.size( ); i++ ) int thisSum = 0; for( int j = i; j < a.size( ); j++ ) { thisSum += a[ j ]; if( thisSum > maxSum ) maxSum = thisSum; } return maxSum; 16

Slide 17: Algorithm 3 s This algorithm uses divide-and-conquer paradigm. s Suppose we split the input sequence at midpoint. s The max subsequence is entirely in the left half, entirely in the right half, or it straddles the midpoint. s Example: left half | right half 4 -3 5 -2 | -1 2 6 -2 Max in left is 6 (A1 through A3); max in right is 8 (A6 s through A7). But straddling max is 11 (A1 thru A7). 17

Slide 18: Algorithm 3 (cont.) Example: s left half | right half 4 -3 5 -2 | -1 2 6 -2 s Max subsequences in each half found by recursion. s How do we find the straddling max subsequence? s Key Observation:  Left half of the straddling sequence is the max subsequence ending with -2.  Right half is the max subsequence beginning with -1. A linear scan lets us compute these in O(n) time. s 18

Slide 19: Algorithm 3: Analysis s The divide and conquer is best analyzed through recurrence: T(1) = 1 T(n) = 2T(n/2) + O(n) s This recurrence solves to T(n) = O(n log n). 19

Slide 20: Algorithm 4 2, 3, -2, 1, -5, 4, 1, -3, 4, -1, 2 int maxSum = 0, thisSum = 0; for( int j = 0; j < a.size( ); j++ ) { thisSum += a[ j ]; if ( thisSum > maxSum ) maxSum = thisSum; else if ( thisSum < 0 ) thisSum = 0; } return maxSum; } complexity clearly O(n) s Time s But why does it work? I.e. proof of correctness. 20

Slide 21: Proof of Correctness Max subsequence cannot start or end at a negative Ai. s s More generally, the max subsequence cannot have a prefix with a negative sum. Ex: -2 11 -4 13 -5 -2 s Thus, if we ever find that Ai through Aj sums to < 0, then we can advance i to j+1  Proof. Suppose j is the first index after i when the sum becomes < 0  The max subsequence cannot start at any p between i and j. Because Ai through Ap-1 is positive, so starting at i would have been even better. 21

Slide 22: Algorithm 4 int maxSum = 0, thisSum = 0; for( int j = 0; j < a.size( ); j++ ) { thisSum += a[ j ]; if ( thisSum > maxSum ) maxSum = thisSum; else if ( thisSum < 0 ) thisSum = 0; } return maxSum • The algorithm resets whenever prefix is < 0. Otherwise, it forms new sums and updates maxSum in one pass. 22

Slide 23: Why Efficient Algorithms Matter Suppose N = 106 s s A PC can read/process N records in 1 sec. s But if some algorithm does N*N computation, then it takes 1M seconds = 11 days!!! 100 City Traveling Salesman Problem. s  A supercomputer checking 100 billion tours/sec still requires 10100 years! Fast factoring algorithms can break encryption schemes. s Algorithms research determines what is safe code length. (> 100 digits) 23

Slide 24: How to Measure Algorithm Performance What metric should be used to judge algorithms? s  Length of the program (lines of code)  Ease of programming (bugs, maintenance)  Memory required  Running time s Running time is the dominant standard.  Quantifiable and easy to compare  Often the critical bottleneck 24

Slide 25: Abstraction An algorithm may run differently depending on: s  the hardware platform (PC, Cray, Sun)  the programming language (C, Java, C++)  the programmer (you, me, Bill Joy) While different in detail, all hardware and programming s models are equivalent in some sense: Turing machines. It suffices to count basic operations. s Crude but valuable measure of algorithm’s performance as a s function of input size. 25

Slide 26: Average, Best, and Worst-Case On which input instances should the algorithm’s performance s be judged? Average case: s  Real world distributions difficult to predict s Best case:  Seems unrealistic s Worst case:  Gives an absolute guarantee  We will use the worst-case measure. 26

Slide 27: Examples s Vector addition Z = A+B for (int i=0; i<n; i++) Z[i] = A[i] + B[i]; T(n) = c n s Vector (inner) multiplication z =A*B z = 0; for (int i=0; i<n; i++) z = z + A[i]*B[i]; T(n) = c’ + c1 n 27

Slide 28: Examples s Vector (outer) multiplication Z = A*BT for (int i=0; i<n; i++) for (int j=0; j<n; j++) Z[i,j] = A[i] * B[j]; T(n) = c2 n2; sA program does all the above T(n) = c0 + c1 n + c2 n2; 28

Slide 29: Simplifying the Bound s T(n) = ck nk + ck-1 nk-1 + ck-2 nk-2 + … + c1 n + co too complicated   too many terms  Difficult to compare two expressions, each with 10 or 20 terms s Do we really need that many terms? 29

Slide 30: Simplifications Keep just one term! s  the fastest growing term (dominates the runtime) s No constant coefficients are kept  Constant coefficients affected by machines, languages, etc. Asymtotic behavior (as n gets large) is determined entirely s by the leading term. Example. T(n) = 10 n3 + n2 + 40n + 800  If n = 1,000, then T(n) = 10,001,040,800  error is 0.01% if we drop all but the n3 term  In an assembly line the slowest worker determines the  throughput rate 30

Slide 31: Simplification s Drop the constant coefficient  Does not effect the relative order 31

Slide 32: Simplification faster growing term (such as 2n) eventually will s The outgrow the slower growing terms (e.g., 1000 n) no matter what their coefficients! Put another way, given a certain increase in s allocated time, a higher order algorithm will not reap the benefit by solving much larger problem 32

Slide 33: Complexity and Tractability T(n) n2 n3 n4 n10 2n n n n log n 10 .01s .03s .1s 1s 10s 10s 1s 20 .02s .09s .4s 8s 160s 2.84h 1ms s 30 .03s .15s .9s 810s 6.83d 1s s 40 .04s .21s 1.6s 2.56ms 121d 18m s s 50 .05s .28s 6.25ms 3.1y 13d 41013y 100 .1s .66s 10s 1ms 100ms 3171y 103 3.171013y 3210283y 1s 9.96s 1ms 1s 16.67m 104 3.171023y s 130s 100ms 16.67m 115.7d 105 3.171033y s 1.66ms 10s 11.57d 3171y 106 31.71y 3.17107y 3.171043y ms 19.92ms 16.67m Assume the computer does 1 billion ops per sec. 33

Slide 34: n2 n3 2n log n n n log n 0 1 0 1 1 2 1 2 2 4 8 4 2 4 8 16 64 16 3 8 24 64 512 256 4 16 64 256 4096 65,536 5 32 160 1,024 32,768 4,294,967,296 2n n2 70000 2n 100000 n2 n3 60000 n log n 10000 50000 n 40000 1000 n3 30000 100 20000 n log n log n 10 10000 n 0 1 log n n n 34

Slide 35: Another View More resources (time and/or processing power) translate into s large problems solved if complexity is low T(n) Problem size Problem size Increase in solved in 103 solved in 104 Problem size sec sec 100n 10 100 10 1000n 1 10 10 5n2 14 45 3.2 N3 10 22 2.2 2n 10 13 1.3 35

Slide 36: Asymptotics T(n) keep one drop coef 3n2+4n+1 3 n2 n2 101 n2+102 101 n2 n2 15 n2+6n 15 n2 n2 a n2+bn+c a n2 n2 s They all have the same “growth” rate 36

Slide 37: Caveats s Followthe spirit, not the letter  a 100n algorithm is more expensive than n2 algorithm when n < 100 s Other considerations:  a program used only a few times  a program run on small data sets  ease of coding, porting, maintenance  memory requirements 37

Slide 38: Asymptotic Notations Big-O, “bounded above by”: T(n) = O(f(n)) s  For some c and N, T(n)  c·f(n) whenever n > N. Big-Omega, “bounded below by”: T(n) = (f(n)) s  For some c>0 and N, T(n)  c·f(n) whenever n > N.  Same as f(n) = O(T(n)). Big-Theta, “bounded above and below”: T(n) = (f(n)) s  T(n) = O(f(n)) and also T(n) = (f(n)) Little-o, “strictly bounded above”: T(n) = o(f(n)) s  T(n)/f(n)  0 as n   38

Slide 39: By Pictures s Big-Oh (most commonly used)  bounded above s Big-Omega  bounded below N0 s Big-Theta  exactly s Small-o N0  not as expensive as ... N0 39

Slide 40: Example T ( n )  n  2n 3 2 O (?) (?)  0 10 n n 5 2 n n 3 3 n n 40

Slide 41: Examples f ( n) Asymptomic (1) c  i 1 ci n ( n ) k i k  in1 i ( n 2 )  in1 i 2 ( n 3 )  i 1i ( n ) k 1 nk  in0 r i ( r n ) (n(n /e) n ) n!  i 11 / i (log n) n 41

Slide 42: Summary (Why O(n)?) s T(n) = ck nk + ck-1 nk-1 + ck-2 nk-2 + … + c1 n + co s Too complicated s O(nk )  a single term with constant coefficient dropped s Much simpler, extra terms and coefficients do not matter asymptotically s Other criteria hard to quantify 42

Slide 43: Runtime Analysis Useful rules s  simple statements (read, write, assign) O(1) (constant)  simple operations (+ - * / == > >= < <=  O(1)  sequence of simple statements/operations  rule of sums  for, do, while loops  rules of products  43

Slide 44: Runtime Analysis (cont.) s Two important rules  Rule of sums if you do a number of operations in sequence, the  runtime is dominated by the most expensive operation Rule of products  if you repeat an operation a number of times, the total  runtime is the runtime of the operation multiplied by the iteration count 44

Slide 45: Runtime Analysis (cont.) O(1) if (cond) then T1(n) body1 else T2(n) body2 endif T(n) = O(max (T1(n), T2(n)) 45

Slide 46: Runtime Analysis (cont.) s Method calls  A calls B  B calls C  etc. s A sequence of operations when call sequences are flattened T(n) = max(TA(n), TB(n), TC(n)) 46

Slide 47: Example for (i=1; i<n; i++) if A(i) > maxVal then maxVal= A(i); maxPos= i; Asymptotic Complexity: O(n) 47

Slide 48: Example for (i=1; i<n-1; i++) for (j=n; j>= i+1; j--) if (A(j-1) > A(j)) then temp = A(j-1); A(j-1) = A(j); A(j) = tmp; endif endfor endfor Asymptotic Complexity is O(n2) s 48

Slide 49: Run Time for Recursive Programs s T(n) is defined recursively in terms of T(k), k<n s The recurrence relations allow T(n) to be “unwound” recursively into some base cases (e.g., T(0) or T(1)). s Examples:  Factorial  Hanoi towers 49

Slide 50: Example: Factorial T ( n) int factorial (int n) {  T (n  1)  d if (n<=1) return 1;  T (n  2)  d  d else return n * factorial(n-1);  T (n  3)  d  d  d }  ....  T (1)  (n  1) * d factorial (n) = n*n-1*n-2* … *1  c  (n  1) * d  O ( n) n * factorial(n-1) T(n) n-1 * factorial(n-2) T(n-1) n-2 * factorial(n-3) T(n-2) … 2 *factorial(1) T(1) 50

Slide 51: Example: Factorial (cont.) int factorial1(int n) { if (n<=1) return 1; else { fact = 1; O (1) for (k=2;k<=n;k++) fact *= k; O(n) O (1) return fact; } } s Both algorithms are O(n). 51

Slide 52: Example: Hanoi Towers s Hanoi(n,A,B,C) = s Hanoi(n-1,A,C,B)+Hanoi(1,A,B,C)+Hanoi(n-1,C,B,A) T ( n)  2T (n  1)  c  2 2 T (n  2)  2c  c  23 T (n  3)  2 2 c  2c  c  ....  2 n 1T (1)  (2 n  2  ...  2  1)c  (2 n 1  2 n  2  ...  2  1)c  O(2 n ) 52

Slide 53: Worst Case, Best Case, and Average Case template<class T> void SelectionSort(T a[], int n) { // Early-terminating version of selection sort bool sorted = false; for (int size=n; !sorted && (size>1); size--) { int pos = 0; sorted = true; // find largest for (int i = 1; i < size; i++) if (a[pos] <= a[i]) pos = i; else sorted = false; // out of order Swap(a[pos], a[size - 1]); } } s Worst Case s Best Case 53

Slide 54: c f(N) T(N) n0 f(N) T(N)=O(f(N)) s T(N)=6N+4 : n0=4 and N2 = O(N log N)? c=7, f(N)=N N10 = O(2N)? s T(N)=6N+4 <= c f(N) = 6N + 4 = W(N) ? 7N? 7N for N>=4 N+4 ? N2? N log N? s 7N+4 = O(N) N log N = W(N2)? s 15N+20 = O(N) 3 = O(1) s N2=O(N)? 1000000=O(1) Sum i = O(N)? s N log N = O(N)? s N log N = O(N2)? 54

Slide 55: An Analogy: Cooking Recipes Algorithms are detailed and precise instructions. s Example: bake a chocolate mousse cake. s  Convert raw ingredients into processed output.  Hardware (PC, supercomputer vs. oven, stove)  Pots, pans, pantry are data structures. Interplay of hardware and algorithms s  Different recipes for oven, stove, microwave etc. New advances. s  New models: clusters, Internet, workstations  Microwave cooking, 5-minute recipes, refrigeration 55

Slide 56: Lecture 2 - Recursion

Slide 57: What is Recursion? Recursion is when a function either directly or s indirectly makes a call to itself. Recursion is a powerful problem solving tool. s Many mathematical functions and algorithms s are most easily expressed using recursion. 57

Slide 58: How does it work? Functions are implemented using an internal stack of s activation records. Each time a function is called a new activation record s is pushed on the stack. When a function returns, the stack is popped and the s activation record of the calling method is on top of the stack. 58

Slide 59: How does it work? (cont.) The function being called, and whose activation record is s being pushed on the stack, can be different from the calling function (e.g., when main calls a function). The function being called can be a different instance of s the calling subprogram. Each instance of the function has its own parameters and s local variables. 59

Slide 60: Example Many mathematical functions are defined s recursively. For example, the factorial function: N! = N * (N-1)! for N>0 0! =1 We have defined factorial in terms of a smaller (or q simpler) instance of itself. We must also define a base case or stopping condition. q 60

Slide 61: Recursive Function Call A recursive call is a function call in which the s called function is the same as the one making the call. We must avoid making an infinite sequence of s function calls (infinite recursion).

Slide 62: Finding a Recursive Solution Each successive recursive call should bring you closer s to a situation in which the answer is known. general (recursive) case A case for which the answer is known (and can be s expressed without recursion) is called a base case.

Slide 63: General format for many recursive functions if (some conditions for which answer is known) // base case solution statement else // general case recursive function call

Slide 64: Tail Recursion Use only one recursive call at the end of a function s void tail (int i) { if (i > 0) { void iterativetail(int i) { System.out.print( i+ “ ”); for ( ; i > 0; i--) tail(i – 1); } System.out.print( i+ “ ”); } }

Slide 65: NonTail Recursion void nonTail (int i) { if (i > 0) { nonTail(i – 1); System.out.print( i+ “ ”); nonTail(i – 1); } } 65

Slide 66: Indirect recursion Receive( buffer ) while buffer is not filled up If information is still incoming get a char and store it in buffer; else exit(); decode(buffer); Decode(buffer) decode information in buffer; store(buffer); Store(buffer) transfer information from buffer to file; receive(buffer); 66

Slide 67: Nested recursion h(n) { 0 if n=0 h(n) ={ n if n>4 h(n) { h(2+h(2n)) if n <=4 67

Slide 68: Writing a recursive function to find n factorial DISCUSSION The function call Factorial(4) should have value 24, because that is 4 * 3 * 2 * 1 . For a situation in which the answer is known, the value of 0! is 1. So our base case could be along the lines of if ( number == 0 ) return 1;

Slide 69: Writing a recursive function to find Factorial(n) Now for the general case . . . The value of Factorial(n) can be written as n * the product of the numbers from (n - 1) to 1, that is, n * (n - 1) * . . . * 1 or, n* Factorial(n - 1) And notice that the recursive call Factorial(n - 1) gets us “closer” to the base case of Factorial(0).

Slide 70: Recursive Function Example: Factorial (n factorial) Problem: calculate n! s n! = 1 if n = 0 n! = 1 * 2 * 3 *...* n if n > 0 Recursively: s if n = 0, then n! = 1 if n > 0, then n! = n * (n-1)! 70

Slide 71: Factorial Function int RecFactorial( /* in */ int n) // Calculates n factorial, n! // Precondition: n is a non-negative // integer { if (n <= 0) then return 1 else return n * RecFactorial(n-1) } 71

Slide 72: 1int fact(int n) 2{ 3 if (n <= 0) then 4 return 1; 5 else 6 return n*fact(n-1); 7} main(...) { ... 20 System.out.print( (fact(3)); ... returns 6 to main() 72

Slide 73: Exponentiation base exponent e.g. 53 Could be written as a function Power(base, exp) 73

Slide 74: Can we write it recursively? be = b * b(e-1) What’s the limiting case? When e = 0 we have b0 which always equals? 1 74

Slide 75: Another Recursive Function Power xN = x * xN-1 for N>0 x0 = 1 1 function Power returnsa Num(base, exp) 2 // Computes the value of BaseExp 3 // Pre: exp is a non-negative integer 4 if (exp = 0) then 5 returns 1 6 else 7 returns base * Power(base, exp-1) 8 endif 9 endfunction main(...) { ... 20 cout << (Power(2,3)); ... 75

Slide 76: The Puzzle A man has an infant male-female pair of rabbits in a hutch s entirely surrounded by a wall. We wish to know how many rabbits can be bred from this pair in one year, if the nature of the rabbits is such that every month they breed one other male- female pair which begins to breed in the second month after their birth. Assume that no rabbits die during the year. 76

Slide 77: A Tree Diagram for Fibonacci’s Puzzle 77

Slide 78: Observations The number of rabbits at the beginning of any month s equals the number of rabbits of the previous month plus the number of new pairs. The number of new pairs at the beginning of a month s equals the number of pairs two months ago. One gets the sequence: s 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, …233 78

Slide 79: Fibonacci sequence Recursive definition s f(1) = 1 f(2) = 1 f(n) = f(n-1) + f(n-2) 79

Slide 80: A More Complex Recursive Function Fibonacci Number Sequence s if n = 1, then Fib(n) = 1 if n = 2, then Fib(n) = 1 if n > 2, then Fib(n) = Fib(n-2) + Fib(n-1) Numbers in the series: s 1, 1, 2, 3, 5, 8, 13, 21, 34, ... 80

Slide 81: Fibonacci Sequence Function function Fib returnsaNum (n) // Calculates the nth Fibonacci number // Precondition: N is a positive integer if ((n = 1) OR (n = 2)) then returns 1 else returns Fib(n-2) + Fib(n-1) endif endfunction //Fibonacci 81

Slide 82: Tracing with Multiple Recursive Calls Main Algorithm: answer <- Fib(5) 82

Slide 83: Tracing with Multiple Recursive Calls Fib(5): Fib returns Fib(3) + Fib(4) Main Algorithm: answer <- Fib(5) 83

Slide 84: Tracing with Multiple Recursive Calls Fib(3): Fib returns Fib(1) + Fib(2) Fib(5): Fib returns Fib(3) + Fib(4) Main Algorithm: answer <- Fib(5) 84

Slide 85: Tracing with Multiple Recursive Calls Fib(1): Fib returns 1 Fib(3): Fib returns Fib(1) + Fib(2) Fib(5): Fib returns Fib(3) + Fib(4) Main Algorithm: answer <- Fib(5) 85

Slide 86: Tracing with Multiple Recursive Calls Fib(3): Fib returns 1 + Fib(2) Fib(5): Fib returns Fib(3) + Fib(4) Main Algorithm: answer <- Fib(5) 86

Slide 87: Tracing with Multiple Recursive Calls Fib(2): Fib returns 1 Fib(3): Fib returns 1 + Fib(2) Fib(5): Fib returns Fib(3) + Fib(4) Main Algorithm: answer <- Fib(5) 87

Slide 88: Tracing with Multiple Recursive Calls Fib(3): Fib returns 1 + 1 Fib(5): Fib returns Fib(3) + Fib(4) Main Algorithm: answer <- Fib(5) 88

Slide 89: Tracing with Multiple Recursive Calls Fib(5): Fib returns 2 + Fib(4) Main Algorithm: answer <- Fib(5) 89

Slide 90: Tracing with Multiple Recursive Calls Fib(4): Fib returns Fib(2) + Fib(3) Fib(5): Fib returns 2 + Fib(4) Main Algorithm: answer <- Fib(5) 90

Slide 91: Tracing with Multiple Recursive Calls Fib(2): Fib returns 1 Fib(4): Fib returns Fib(2) + Fib(3) Fib(5): Fib returns 2 + Fib(4) Main Algorithm: answer <- Fib(5) 91

Slide 92: Tracing with Multiple Recursive Calls Fib(4): Fib returns 1 + Fib(3) Fib(5): Fib returns 2 + Fib(4) Main Algorithm: answer <- Fib(5) 92

Slide 93: Tracing with Multiple Recursive Calls Fib(3): Fib returns Fib(1) + Fib(2) Fib(4): Fib returns 1 + Fib(3) Fib(5): Fib returns 2 + Fib(4) Main Algorithm: answer <- Fib(5) 93

Slide 94: Tracing with Multiple Recursive Calls Fib(1): Fib returns 1 Fib(3): Fib returns Fib(1) + Fib(2) Fib(4): Fib returns 1 + Fib(3) Fib(5): Fib returns 2 + Fib(4) Main Algorithm: answer <- Fib(5) 94

Slide 95: Tracing with Multiple Recursive Calls Fib(3): Fib returns 1 + Fib(2) Fib(4): Fib returns 1 + Fib(3) Fib(5): Fib returns 2 + Fib(4) Main Algorithm: answer <- Fib(5) 95

Slide 96: Tracing with Multiple Recursive Calls Fib(2): Fib returns 1 Fib(3): Fib returns 1 + Fib(2) Fib(4): Fib returns 1 + Fib(3) Fib(5): Fib returns 2 + Fib(4) Main Algorithm: answer <- Fib(5) 96

Slide 97: Tracing with Multiple Recursive Calls Fib(3): Fib returns 1 + 1 Fib(4): Fib returns 1 + Fib(3) Fib(5): Fib returns 2 + Fib(4) Main Algorithm: answer <- Fib(5) 97

Slide 98: Tracing with Multiple Recursive Calls Fib(4): Fib returns 1 + 2 Fib(5): Fib returns 2 + Fib(4) Main Algorithm: answer <- Fib(5) 98

Slide 99: Tracing with Multiple Recursive Calls Fib(5): Fib returns 2 + 3 Main Algorithm: answer <- Fib(5) 99

Slide 100: Tracing with Multiple Recursive Calls Main Algorithm: answer <- 5 100

Slide 101: Fib(5) Fib(4) Fib(3) Fib(3) Fib(2) Fib(2) Fib(1) Fib(1) Fib(0) Fib(1) Fib(0) Fib(2) Fib(1) Fib(1) Fib(0) 15 calls to Fib to find the 5th Fibonacci number!!! 101

Slide 102: Excessive Recursion-Fibonacci recursion Int iterativeFib ( int n ) if ( n < 2) return n; else { int i=2, tmp,current = 1, last=0; for ( ; i<=n; ++i) { tmp=current; current+=last; last=tmp; } return current; } } 102

Slide 103: 103

Slide 104: 104

Slide 105: Rules of Recursion First two fundamental rules of recursion: Base cases: Always have at least one case that can be solved without using recursion. Make progress: Any recursive call must make progress towards the base case. 105

Slide 106: Rules of Recursion Third fundamental rule of recursion: s “You gotta believe”: Always assume that the recursive call works. 106

Slide 107: Rules of Recursion Fourth fundamental rule of recursion: s Compound interest rule: Never duplicate work by solving the same instance of a problem in separate recursive calls. 107

Slide 108: Towers of Hanoi The puzzle consisted of N disks and three poles: s A (the source), B (the destination), and C (the spare) 108

Slide 109: Towers of Hanoi A B C 109

Slide 117: Towers of Hanoi A pseudocode description of the solution is: Towers(Count, Source, Dest, Spare) if (Count is 1) Move the disk directly from Source to Dest else { Solve Towers(Count-1, Source, Spare, Dest) Solve Towers(1, Source, Dest, Spare) Solve Towers(Count-1, Spare, Dest, Source) } 117

Slide 118: Towers of Hanoi void solveTowers( int count, char source, char dest, char spare){ if (count == 1) cout<<“Move disk from pole “ << source << " to pole " << destination <<endl; else { towers(count-1, source, spare, destination); towers(1, source, destination, spare); towers(count-1, spare, destination, source); }//end if }//end solveTowers 118

Slide 119: Recall that . . . Recursion occurs when a function calls itself s (directly or indirectly). Recursion can be used in place of iteration s (looping). Some functions can be written more easily using s recursion.

Slide 120: Pascal Triangle (Is this recursive?) 120

Slide 121: Pascal Triangle The combinations of n items taken r at a time. For example: s three items: a b c taken 2 at a time: ab ac bc s Thus there are three combinations of 3 items taken 2 at a time. s In General: C(n,r) = n!/(r!(n-r)!) Obviously you can calculate s C(n,r) using factorials. 121

Slide 122: Pascal Triangle This leads to Pascal Triangle: s n 0 1 1 1 1 2 1 2 1 3 1 3 3 1 4 1 4 6 41 5 1 5 10 10 5 1 This can also be written: r 012 3 4 5 n0 1 1 11 2 121 3 133 1 4 146 41 5 1 5 10 10 5 1 122

Slide 123: Pascal Triangle Note from Pascal's Triangle that: s C(n,r) = C(n-1, r-1) + C(n-1,r) This leads to the recurrence for nonnegative r and n, s C(n,r) = 1 if r = 0 or r = n, 0 if r > n, C(n-1, r-1) + C(n-1,r) otherwise. 123

Slide 124: Pascal Triangle This immediately leads to the recursive s function for combinations: int C(int n, int r) { if((r == 0) || (r == n)) return 1; else if(r > n) return 0; else return C(n-1, r-1) + C(n-1, r); } 124

Slide 125: What is the value of rose(25)? int rose (int n) { // base case if ( n == 1 ) return 0; // general case else return ( 1 + rose ( n / 2 ) ); }

Slide 126: Finding the value of rose(25) the original call rose(25) first recursive call = 1 + rose(12) second recursive call = 1 + ( 1 + rose(6) ) third recursive call = 1 + ( 1 + ( 1 + rose(3) ) ) fourth recursive call = 1 + ( 1 + ( 1 + (1 + rose(1) ) ) ) = 1+ 1+ 1 + 1+0 = 4

Slide 127: Writing recursive functions There must be at least one base case, and at least one general s (recursive) case. The general case should bring you “closer” to the base case. The parameter(s) in the recursive call cannot all be the same as the s formal parameters in the heading. Otherwise, infinite recursion would occur. In function rose( ), the base case occurred when (n == 1) s was true. The general case brought us a step closer to the base case, because in the general case the call was to rose(n/2), and the argument n/2 was closer to 1 (than n was).

Slide 128: Three-Question Method of verifying recursive functions Base-Case Question: Is there a nonrecursive s way out of the function? Smaller-Caller Question: Does each recursive s function call involve a smaller case of the original problem leading to the base case? General-Case Question: Assuming each s recursive call works correctly, does the whole function work correctly?

Slide 129: Guidelines for writing recursive functions 1. Get an exact definition of the problem to be solved. 2. Determine the size of the problem to be solved on this call to the function. On the initial call, the size of the whole problem is expressed by the actual parameter(s). 3. Identify and solve the base case(s) which have non-recursive solutions. 4. Identify and solve the general case(s) in terms of smaller (recursive) cases of the same problem.

Slide 130: struct ListType struct ListType { int length ; // number of elements in the list int info[ MAX_ITEMS ] ; }; ListType list ;

Slide 131: Recursive function to determine if value is in list PROTOTYPE bool ValueInList( ListType list , int value , int startIndex ) 74 36 ... 95 75 29 47 ... list[0] [1] [startIndex] [length -1] Already searched Needs to be searched index of current element to examine

Slide 132: bool ValueInList ( ListType list , int value , int startIndex ) // Searches list for value between positions startIndex // and list.length-1 { // one base case if ( list.info[startIndex] == value ) return true ; // another base case else if (startIndex == list.length -1 ) return false ; // general case else return ValueInList( list, value, startIndex + 1 ) ; } 132

Slide 133: “Why use recursion?” Many solutions could have been written without recursion, by using iteration instead. The iterative solution uses a loop, and the recursive solution uses an if statement. However, for certain problems the recursive solution is the most natural solution. This often occurs when pointer variables are used.

Slide 134: When to Use Recursion If the problem is recursive in nature therefore it is likely  the a recursive algorithm will be preferable and will be less complex  If running times of recursive and non-recursive algorithms are hardly perceivable, recursive version is better  If both recursive and non-recursive algorithms have the same development complexity, a non-recursive version should be preferred  A third alternative in some problems is to use table-driven techniques Sometimes we know that we will not use the only a few values of a particular function If this is the case an implementation using a table would probably suffice and the performance will be better int factorial[8] = {1, 1, 2, 6, 24, 120, 720, 5040}; 134

Slide 135: Use a recursive solution when: The depth of recursive calls is relatively “shallow” compared s to the size of the problem. The recursive version does about the same amount of work s as the nonrecursive version. The recursive version is shorter and simpler than the s nonrecursive solution. SHALLOW DEPTH EFFICIENCY CLARITY

Slide 136: Lecture 3 – Binary Trees

Slide 137: Why Trees? Limitations of s  Arrays  Linked lists  Stacks  Queues 137

Slide 138: Trees: Recursive Definition A tree is a collection of nodes. s The collection can be empty, or consist of a s “root” node R. There is a “directed edge” from R to the root of s each subtree. The root of each subtree is a “child” of R. R is the “parent” of each subtree root. 138

Slide 139: Trees: Recursive Definition (cont.) ROOT OF TREE T T2 T3 T4 T1 T5 SUBTREES 139

Slide 140: Trees: An Example A B C D E H G F I 140

Slide 141: Trees: More Definitions Nodes with no children are leaves: (C,E,F,H,I). s Nodes with the same parents are siblings: s (B,C,D,E) and (G,H). A path from node n to node m is the sequence of s directed edges from n to m. A length of a path is the number of edges in the s path 141

Slide 142: Trees: More Definitions (cont.) The level/depth of node n is the length of the path from s the root to n. The level of the root is 0. The height/depth of a tree is equal to the maximum level s of a node in the tree. The height of a node n is the length of the longest path s from n to a leaf. The height of a leaf node is 0. The height of a tree is equal to the height of the root s node. 142

Slide 143: Binary Trees – A Informal Definition A binary tree is a tree in which no node can s have more than two children. Each node has 0

Slideshare.net (beta)