Cis435 week01

255 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
255
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Cis435 week01

  1. 1. Advanced Data Structures & Algorithm Design Introduction to Algorithms and Algorithm Analysis
  2. 2. Introduction to Algorithms & Algorithm Analysis 2 What is an Algorithm?  Any well-defined computational procedure that takes some value or set of values as input, and produces some value or values as output; or  A sequence of computational steps that transforms the input into the output; or  Tool for solving a well-specified computational problem  The problem specifies the relationship between input and output  The algorithm describes a procedure for achieving that relationship
  3. 3. Introduction to Algorithms & Algorithm Analysis 3 Algorithms: An Example  The Sorting Problem  The need to sort data arises frequently in practice  Formally, the sorting problem is specified as:  Input: A sequence of n numbers {a1, a2, …, an}  Output: A permutation (reordering) {a1’, a2’, …, an’} of the input sequence such that a1’ <= a2’ <= … <= an’  That is, given the input sequence { 5, 7, 3, 2, 9 }, a sorting algorithm returns as output the sequence { 2, 3, 5, 7, 9}  This is one instance of the sorting problem
  4. 4. Introduction to Algorithms & Algorithm Analysis 4 Correctness of Algorithms  An algorithm is correct if for every input instance, it halts with the correct output  i.e., it solves the given computational problem  Correctness isn't everything  Incorrectness can be useful, if the error rate can be controlled
  5. 5. Introduction to Algorithms & Algorithm Analysis 5 Analyzing Algorithms  To analyze an algorithm means to predict the resources that an algorithm will require  Why do we care about an algorithm’s resource requirements?  Resources are bounded – there isn’t an infinite amount of time or space for an algorithm to execute in  What resources do we care about?  Time (how long does it take)  Space (how much memory does it use)
  6. 6. Introduction to Algorithms & Algorithm Analysis 6 Analyzing Algorithms  Analysis usually measures two forms of complexity:  Spatial Complexity: memory, communications bandwidth  Temporal Complexity: computational time
  7. 7. Introduction to Algorithms & Algorithm Analysis 7 Analyzing Algorithms: An Example  Two algorithms for solving the same problem often differ dramatically in their efficiency  Consider two sorting algorithms:  Insertion sort takes time roughly equal to c1n2  n is the number of items being sorted;  c1 is a constant that does not depend on n  Merge sort takes time roughly equal to c2nlog2n
  8. 8. Introduction to Algorithms & Algorithm Analysis 8 Analyzing Algorithms: An Example  What does this mean? Consider sorting 1,000,000 numbers on two different computers  Computer A uses insertion sort, and executes 1,000,000,000 instructions per second; we’ll assume c1 = 2  Computer B uses merge sort, and executes 10,000,000 instructions per second; we’ll assume c2 = 50  How long does it take?
  9. 9. Introduction to Algorithms & Algorithm Analysis 9 Analyzing Algorithms: An Example  Computer A:  Computer B: seconds2000 ns/secinstructio10 nsinstructio102 9 26 seconds100 ns/secinstructio10 nsinstructio10log1050 7 6 2 6
  10. 10. Introduction to Algorithms & Algorithm Analysis 10 Insertion Sort  Insertion sort is one method of solving the sorting problem  Conceptually, insertion sort works the way many people sort playing cards:  Start with one hand empty and the cards face down  Remove one card at a time from the table and insert it into the correct position in the left hand
  11. 11. Introduction to Algorithms & Algorithm Analysis 11 Insertion Sort: Algorithm void InsertionSort(ArrayType A[], unsigned size) { for ( unsigned j = 1 ; j < size ; ++j ) { ArrayType key = A[j]; int i = j-1; while ( i >= 0 && A[i] > key ) { A[i+1] = A[i]; --i; } A[i+1] = key; } }
  12. 12. Introduction to Algorithms & Algorithm Analysis 12 Operation of Insertion Sort 5 2 4 6 1 3 5 2 4 6 1 3 52 4 6 1 3
  13. 13. Introduction to Algorithms & Algorithm Analysis 13 Operation of Insertion Sort 52 4 6 1 3 52 4 6 1 3 52 4 61 3 52 4 61 3 52 4 61 3
  14. 14. Introduction to Algorithms & Algorithm Analysis 14 Analysis of Insertion Sort  How long does insertion sort take?  Time varies based on:  The size of the input  How well sorted the input is to begin with  What is “the size of the input”?  May represent the number of items in the input, or the number of bits being calculated  May be represented by more than one number
  15. 15. Introduction to Algorithms & Algorithm Analysis 15 Analysis of Insertion Sort  What is “running time”?  Running time is based on the number of steps performed by the processor  Different processors run at different speeds, so time per step will vary between processors  We will express times in terms of a constant cost per step, ci
  16. 16. Introduction to Algorithms & Algorithm Analysis 16 Analysis of Insertion Sort  What is the “cost” of Insertion Sort?  Total cost = the sum of the cost of each statement * number of times statement is executed Statem ent Cost Tim es for ( unsigned j = 1 ; j < size ; ++j ) c1 n ArrayType key = A[j]; c2 n-1 int i = j-1; c3 n-1 while ( i >= 0 && A[i] > key ) c4 n j j t 1 A[i+1] = A[i]; c5 n j t j1 )1( --i; c6 n j t j1 )1( A[i+1] = key; c7 n-1
  17. 17. Introduction to Algorithms & Algorithm Analysis 17 Analysis of Insertion Sort  In the best case, all tj = 1  The inner loop only executes once, because the array is already sorted  In this case, T(n)=(c1+c2+c3+c4+c7)n-(c2+c3+c4+c7)  Since all constants are unknown, we can as easily say that T(n)=an+b, where a and b depend on the cost of each statement  So in the best case, the running time is a linear function of n )1()1()()1()1()( 716514321 nctcctcncncncnT n j j n j j  Total cost for insertion sort is:
  18. 18. Introduction to Algorithms & Algorithm Analysis 18 Analysis of Insertion Sort  In the worst case, tj=j, since every element must be compared  This occurs when the array is reverse sorted  To solve this, we need the solve the summations: )1()1()()1()1()( 716514321 nctcctcncncncnT n j j n j j n j nn j 1 1 2 )1( n j nn j 1 2 )1( )1(
  19. 19. Introduction to Algorithms & Algorithm Analysis 19 Analysis of Insertion Sort  So, in the worst case: )1( 2 )1( )(1 2 )1( )1()1()( 7654321 nc nn cc nn cncncncnT )()( 2 1 2 )()( 74326547321 2 654 ccccnccccccc n cccnT  We can say that T(n)=ax2+bx+c  The running time in the worst case is therefore a quadratic function of n  In general, we are most interested in worst case running time
  20. 20. Introduction to Algorithms & Algorithm Analysis 20 Analysis of Insertion Sort  Order of Growth  Actual running time is not nearly as important as the “order of growth” of the running time  Order of growth of the running time is how the running time changes as the size of the input changes  Provides a concrete method of comparing alternative algorithms
  21. 21. Introduction to Algorithms & Algorithm Analysis 21 Analysis of Insertion Sort  Order of Growth  Order of growth is typically easier to compute than exact running time  In general, we are most interested in the highest order terms of the running time  Lower order terms become insignificant as the input size grows larger  We can also ignore leading coefficients  constant factors are not as significant as growth rate in determining computational efficiency
  22. 22. Introduction to Algorithms & Algorithm Analysis 22 Analysis of Insertion Sort  Example of Order of Growth:  Best case:  Linear running time: T(n)=an+b  The order of growth is O(n)  This representation (O(n)) is called asymptotic notation  Worst case:  Quadratic running time: T(n)=an2+bn+c  The order of growth is (O(n2))  Order of growth provides us a means of comparing the efficiency of algorithms  Algorithms with lower worst-case order of growth are usually considered to be more efficient
  23. 23. Introduction to Algorithms & Algorithm Analysis 23 Comparison of Order Of Growth n lgn sqrt(n) n nlgn n*n n*n*n 2^n n! 1 0 1 1 0 1 1 2 1 4 2 2 4 8 16 64 16 24 16 4 4 16 64 256 4096 65536 2.0923E+13 64 6 8 64 384 4096 262144 1.8447E+19 1.2689E+89 256 8 16 256 2048 65536 16777216 1.1579E+77 #NUM! 1024 10 32 1024 10240 1048576 1073741824 #NUM! #NUM! 4096 12 64 4096 49152 16777216 6.8719E+10 #NUM! #NUM!
  24. 24. Introduction to Algorithms & Algorithm Analysis 24 Designing Algorithms  Insertion sort is typical of an incremental approach to algorithm design  The input is processed in equally sized increments  An alternative approach is divide-and- conquer  The input is recursively divided and processed in smaller, similar chunks
  25. 25. Introduction to Algorithms & Algorithm Analysis 25 Divide and Conquer  Divide and Conquer is a recursive approach to algorithm design  Consists of three steps at each level of recursion:  Divide the problem into smaller, similar subproblems  Conquer the subproblems by solving them recursively  Combine the solutions to the subproblems into the solution to the original problem
  26. 26. Introduction to Algorithms & Algorithm Analysis 26 The Merge Sort 9 7 4 5 2 1 6 3 8 0 9 7 4 5 2 1 6 3 8 0 974 52 1 63 80 0 1 2 3 4 5 6 7 8 9
  27. 27. Introduction to Algorithms & Algorithm Analysis 27 The Merge Sort void MergeSort(ArrayType A[], int p, int r) { if ( p < r ) { int q = (p+r)/2; // Divide MergeSort(A, p, q); // Conquer left MergeSort(A, q+1, r); // Conquer right Merge(A, p, q, r); // Combine } }
  28. 28. Introduction to Algorithms & Algorithm Analysis 28 Analysis of Merge Sort  How do we describe the running time of Merge Sort?  We must derive a recurrence  A recurrence describes the overall running time of an algorithm on a problem of size n in terms of the running time on smaller inputs
  29. 29. Introduction to Algorithms & Algorithm Analysis 29 Analysis of Merge Sort  What is the recurrence for Merge Sort?  Dividing always takes the same time; it is constant (represented as O(1))  Conquering occurs twice, on an input half the original size  Conquering therefore takes 2T(n/2)  Merge() is not presented, but should be linear (O(n))  This leads to a total running time of T(n) = 2T(n/2)+O(n)+O(1)  O(1) is constant, and can be dropped  This is an infinite recurrence - when does it stop?  When the input size reaches 1
  30. 30. Introduction to Algorithms & Algorithm Analysis 30 Analysis of Merge Sort  So, the total running time is shown as the following recurrence:  We shall show later in the course that the order of growth for this recurrence is O(nlog2n) 1if)()2/(2 1if)1( )( nnOnT nO nT
  31. 31. Introduction to Algorithms & Algorithm Analysis 31 Asymptotic Notation  Asymptotic notation provides a means of bounding the running time of an algorithm  Upper bounds signify worst-case running times  E.g., if T(n) has an upper bound of g(n), this means that the running time of T(n) is at most cg(n)  Lower bounds signify best-case running times
  32. 32. Introduction to Algorithms & Algorithm Analysis 32 Asymptotic Notation  There are three basic asymptotic notations:  O-notation: denotes the upper bound of an expression  -notation: denotes the lower bound of an expression  -notation: denotes both upper and lower bounds of an expression
  33. 33. Introduction to Algorithms & Algorithm Analysis 33 Asymptotic Notation f(n) ag(n) bg(n) f(n) bg(n) f(n) ag(n) T(n)= (g(n)) T(n)= (g(n)) T(n)= (g(n))
  34. 34. Introduction to Algorithms & Algorithm Analysis 34 Asymptotic Notation  Why do we use asymptotic notation?  Reduces clutter by eliminating constant factors and lower-order terms  E.g., 2n2 + 3n + 1 = 2n2 + O(n) = O(n2)  We are interested in comparing algorithms  Actual running times are very hard to compute, and compare  Bounds are easier to compute and provide a more realistic basis for comparison  We can always go back and compute actual running times if we need them
  35. 35. Introduction to Algorithms & Algorithm Analysis 35 Asymptotic Notation  Asymptotic notation of running times exhibits some important mathematical properties:  Transitivity: If f(n) = O(g(n)) and g(n) = O(h(n)) then f(n) = O(h(n))  Reflexivity: f(n) = O(f(n))
  36. 36. Introduction to Algorithms & Algorithm Analysis 36 Homework  Reading: Chapters 1 & 2  Exercises:  1.1: 1, 2, 3  1.2: 2, 5  1.3: 1, 2, 4, 5, 6  2.1: 2  Problems:  1-1, 2-1, 2-4

×