Successfully reported this slideshow.
Upcoming SlideShare
×

# Lect07

2,194 views

Published on

• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

### Lect07

1. 1. Algorithm Analysis Searching Dr.Haitham A. El-Ghareeb Information Systems Department Faculty of Computers and Information Sciences Mansoura University helghareeb@gmail.com November 25, 2012Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 1 / 55
2. 2. Algorithms l  Algorithms are designed to solve problems. l  A problem can have multiple solutions. How do we determine which solution is the most efficient? Chapter 4: Algorithm Analysis –2 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 2 / 55
3. 3. Execution Time l  Measure execution time: l  construct a program for a given solution. l  execute the program. l  time it using a “wall clock”. l  Dependent on: l  amount of data l  type of hardware and time of day l  programming language and compiler Chapter 4: Algorithm Analysis –3 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 3 / 55
4. 4. Complexity Analysis l  What if we examine the solution itself and measure critical operations: l  logical comparisons l  assignments l  arithmetic operations Chapter 4: Algorithm Analysis –4 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 4 / 55
5. 5. Example Algorithm l  Given a matrix of size n x n, compute the: l  sum of each row of a matrix. l  overall sum of the entire matrix. Chapter 4: Algorithm Analysis –5 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 5 / 55
6. 6. Example Algorithm (v.1) l  How many additions are performed? = 2n (n) = 2n2 rowSum = Array(n) totalSum = 0 for i in range( n ) : rowSum[i] = 0 n for j in range( n ) : n rowSum[i] = rowSum[i] + matrix[i,j] 2 totalSum = totalSum + matrix[i,j] Chapter 4: Algorithm Analysis –6 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 6 / 55
7. 7. Example Algorithm (v.2) l  How many additions are performed? =(n + 1) n = n2 + n rowSum = Array(n) totalSum = 0 for i in range( n ) : rowSum[i] = 0 n for j in range( n ) : n 1 rowSum[i] = rowSum[i] + matrix[i,j] totalSum = totalSum + matrix[i,j] 1 Chapter 4: Algorithm Analysis –7 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 7 / 55
8. 8. Compare the Results l  Number of additions: v1: 2n2 v2: n2 + n l  Second version has fewer additions (n > 1) l  Will execute faster than the first. l  Difference will not be significant. Both algorithms execute on the same order of magnitude, n2 Chapter 4: Algorithm Analysis –8 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 8 / 55
9. 9. Growth Rates l  As n increases, both algorithms increase at approx the same rate: n 2n2 n2 + n 10 200 110 100 20,000 10,100 1000 2,000,000 1,001,000 10,000 200,000,000 100,010,000 100,000 20,000,000,000 10,000,100,000 Chapter 4: Algorithm Analysis –9 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 9 / 55
10. 10. Growth Rates Chapter 4: Algorithm Analysis –10 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 10 / 55
11. 11. Big-O Notation l  No need to count precise number of steps. l  Classify algorithms by order of magnitude. l  execution time l  space requirements Approximates actual number of steps or actual storage in terms of variable-sized data sets. Chapter 4: Algorithm Analysis –11 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 11 / 55
12. 12. Big-O Definition l  Given a function T(n) l  # of steps required for an input of size n. l  Ex: T2(n) = n2 + n l  Suppose there exist a function f(n) for all integers n > 0 such that T(n) < c f(n) for some constant c and for all large values of n > m (a constant). Chapter 4: Algorithm Analysis –12 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 12 / 55
13. 13. Big-O Definition l  Then, the algorithm has a time-complexity of or executes “on the order of” f(n) l  We use the notation: O( f(n) ) l  Big-O is intended for large values of n. f(n) indicates the rate of growth at which the run time increases as the input size increases. Chapter 4: Algorithm Analysis –13 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 13 / 55
14. 14. Big-O Example (v.1) l  Consider the previous sample algorithms. l  Version 1: T1(n) = 2n2 T1(n) < c f(n) Let c = 2 2n2 < 2n2 O( n2 ) Chapter 4: Algorithm Analysis –14 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 14 / 55
15. 15. Big-O Example (v.2) l  Consider the previous sample algorithms. l  Version 2: T2(n) = n2 + n T2(n) < c f(n) Let c = 2 n 2 + n < n2 + n2 n2 + n < 2n2 O( n2 ) Chapter 4: Algorithm Analysis –15 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 15 / 55
16. 16. Upper Bound l  There is more than one f(n) for an algorithm. n2 + n < c f(n) l  n2 is not the only choice l  f(n) could be n2, n3, n4 Objective: find an f(n) that provides the tightest (lowest) upper bound. Chapter 4: Algorithm Analysis –16 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 16 / 55
17. 17. Constant of Proportionality l  Is it important? l  Consider two algorithms: l  O(n2), with c = 1 l  O(2n), with c = 2 n n2 2n 10 100 20 100 10,000 200 1000 1,000,000 2,000 10,000 100,000,000 20,000 100,000 10,000,000,000 200,000 Chapter 4: Algorithm Analysis –17 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 17 / 55
18. 18. Constant of Proportionality Chapter 4: Algorithm Analysis –18 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 18 / 55
19. 19. Constructing T(n) l  We dont count total number of specific instructions (math operations, comparisons, etc) l  Assume each basic statement takes the same time, constant time. l  Total number of steps required: T(n) = f1(n) + f2(n) + ... + fk(n) Chapter 4: Algorithm Analysis –19 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 19 / 55
20. 20. Constructing T(n) Example 1 rowSum = Array(n) 1 totalSum = 0 1 for i in range( n ) : 1 rowSum[i] = 0 n 1 for j in range( n ) : n 1 rowSum[i] = rowSum[i] + matrix[i,j] 1 totalSum = totalSum + matrix[i,j] Chapter 4: Algorithm Analysis –20 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 20 / 55
21. 21. Constructing T(n) Example l  Constant time steps are generally omitted. rowSum = Array(n) totalSum = 0 ... for i in range( n ) : rowSum[i] = 0 ... for j in range( n ) : n n rowSum[i] = rowSum[i] + matrix[i,j] totalSum = totalSum + matrix[i,j] ... Chapter 4: Algorithm Analysis –21 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 21 / 55
22. 22. Choosing the Function l  Given T(n), choose the dominant term. T(n) = n2 + log2n + 3n n2 dominates the other terms (for n > 3) n2 + log2n + 3n < n2 + n2 + n2 n2 + log2n + 3n < 3n2 Chapter 4: Algorithm Analysis –22 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 22 / 55
23. 23. Choosing the Function l  What is the dominant term for the following expression? T(n) = 2n2 + 15n + 500 When n < 16, 500 dominates. When n > 16, n2 is the dominate term. Chapter 4: Algorithm Analysis –23 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 23 / 55
24. 24. Classes of Algorithms l  Many algorithms have a time-complexity selected from a common set of functions. f() Common Name 1 constant log n logarithmic n linear n log n log linear n2 quadratic n3 cubic an exponential Chapter 4: Algorithm Analysis –24 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 24 / 55
25. 25. Classes of Algorithms Chapter 4: Algorithm Analysis –25 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 25 / 55
26. 26. Evaluating Python Code l  Basic operations only require constant time: l  x = 5 l  z = x + y * 6 l  if x > 0 and x < 100 l  What about function calls? y = ex1(n) Chapter 4: Algorithm Analysis –26 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 26 / 55
27. 27. Code Evaluation #1 def ex1( n ): count = 0 for i in range( n ): count += i return count Chapter 4: Algorithm Analysis –27 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 27 / 55
28. 28. Code Evaluation #2 def ex2( n ): count = 0 for i in range( n ): count += 1 for j in range( n ): count += 1 return count Chapter 4: Algorithm Analysis –28 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 28 / 55
29. 29. Code Evaluation #3 def ex3( n ): count = 0 for i in range( n ): for j in range( n ): count += 1 return count Chapter 4: Algorithm Analysis –29 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 29 / 55
30. 30. Code Evaluation #3b def ex3b( n ): count = 0 for i in range( n ): count += ex2( n ) return count Chapter 4: Algorithm Analysis –30 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 30 / 55
31. 31. Code Evaluation #4 def ex4( n ): count = 0 for i in range( n ): for j in range( 25 ): count += 1 return count Chapter 4: Algorithm Analysis –31 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 31 / 55
32. 32. Code Evaluation #5 def ex5( n ): count = 0 for i in range( n ): for j in range( i+1 ): count += 1 return count Chapter 4: Algorithm Analysis –32 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 32 / 55
33. 33. Code Evaluation #6 def ex6( n ): count = 0 i = n while i >= 1 : count += 1 i = i // 2 return count Chapter 4: Algorithm Analysis –33 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 33 / 55
34. 34. Code Evaluation #7 def ex7( n ): count = 0 for i in range( n ) : count += ex6( n ) return count Chapter 4: Algorithm Analysis –34 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 34 / 55
35. 35. Different Cases l  Some algorithms have different run times for different sets of inputs of the same size. l  best case l  worst case l  average case Chapter 4: Algorithm Analysis –35 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 35 / 55
36. 36. Different Cases def findNeg( intSeq ): n = len( intSeq ) for i in range( n ) : if intSeq[i] < 0 : return i return None L = [ 72, 4, 90, 56, 12, 67, 43, 17, 2, 86, 33 ] p = findNeg( L ) L = [ -12, 50, 4, 67, 39, 22, 43, 2, 17, 28 ] p = findNeg( L ) Chapter 4: Algorithm Analysis –36 © 2011 John Wiley & Sons, Data Structures and Algorithms Using Python, by Rance D. Necaise.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 36 / 55
37. 37. Searching SearchingDr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 37 / 55
38. 38. SearchingThere are two fundamental ways to search for data in a list: the sequentialsearch and the binary search. Sequential search is used when the items in the list are in random order. Binary search is used when the items are sorted in the list. Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 38 / 55
39. 39. Sequential Search The most obvious type of search begin at the beginning of a set of records and move through each record until you ﬁnd the record you are looking for or you come to the end of the records. A sequential search (also called a linear search) is very easy to implement. Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 39 / 55
40. 40. Sequential Searchbool SeqSearch ( int [ ] arr , int sValue ) {for ( int index = 0 ; index < arr . Length −1; index++) i f ( arr [ index ] == sValue ) return true ; return false ;} Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 40 / 55
41. 41. Modiﬁed Sequential SearchYou can write the sequential search function so that the function returnsthe position in the array where the searched-for value is found or a 1 if thevalue cannot be found. First, lets look at the new function: Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 41 / 55
42. 42. Modiﬁed Sequential Search Codestatic int SeqSearch ( int [ ] arr , int sValue ) {for ( int index = 0 ; index < arr . Length −1; index++) i f ( arr [ index ] == sValue ) return index ;return −1;} Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 42 / 55
43. 43. Searching for Minimum and Maximum Values Computer programs are often asked to search a data structure for minimum and maximum values. In an ordered array, searching for these values is a trivial task. Searching an unordered array, however, is a little more challenging. Lets look at how to ﬁnd the minimum value in an array. The algorithm is: 1 Assign the ﬁrst element of the array to a variable as the minimum value. 2 Begin looping through the array, comparing each successive array element with the minimum value variable. 3 If the currently accessed array element is less than the minimum value, assign this element to the minimum value variable. 4 Continue until the last array element is accessed. 5 The minimum value is stored in the variable.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 43 / 55
44. 44. FindMin Functionstatic int FindMin ( int [ ] arr ) {int min = arr [ 0 ] ;for ( int index = 1 ; index < arr . Length −1; i++) i f ( arr [ index ] < min ) min = arr [ index ] ;return min ;} Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 44 / 55
45. 45. FindMax Functionstatic int FindMax ( int [ ] arr ) {int max = arr [ 0 ] ;for ( int i = 0 ; i < arr . Length −1; i++) i f ( arr [ index ] > max ) max = arr [ index ] ;return max ;} Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 45 / 55
46. 46. AssignmentAn alternative version of these two functions could return the position ofthe maximum or minimum value in the array rather than the actual value. Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 46 / 55
47. 47. Self-Organizing Data The fastest successful sequential searches occur when the data element being searched for is at the beginning of the data set. You can ensure that a successfully located data item is at the beginning of the data set by moving it there after it has been found. The concept behind this strategy is that we can minimize search times by putting frequently searched-for items at the beginning of the data set. Eventually, all the most frequently searched-for data items will be located at the beginning of the data set. This is an example of self-organization, in that the data set is organized not by the programmer before the program runs, but by the program while the program is running.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 47 / 55
48. 48. Binary Search When the records you are searching through are sorted into order, you can perform a more eﬃcient search than the sequential search to ﬁnd a value. This search is called a binary search.Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 48 / 55
49. 49. Binary Search in Action To understand how a binary search works, imagine you are trying to guess a number between 1 and 100 chosen by a friend. For every guess you make, the friend tells you if you guessed the correct number, or if your guess is too high, or if your guess is too low. The best strategy then is to choose 50 as the ﬁrst guess. If that guess is too high, you should then guess 25. If 50 is to low, you should guess 75. Each time you guess, you select a new midpoint by adjusting the lower range or the upper range of the numbers (depending on if your guess is too high or too low), which becomes your next guess. As long as you follow that strategy, you will eventually guess the correct number. Next slide demonstrates how this works if the number to be chosen is 82. Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 49 / 55
50. 50. Binary Search in Action Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 50 / 55
51. 51. Binary Search in Action Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 51 / 55
52. 52. Binary Search in C#static int binSearch ( int value ) {int upperBound , lowerBound , mid ;upperBound = arr . Length −1;lowerBound = 0 ;w h i l e ( lowerBound <= upperBound ) { mid = ( upperBound + lowerBound ) / 2 ; i f ( arr [ mid ] == value ) return mid ; else i f ( value < arr [ mid ] ) upperBound = mid − 1 ; else lowerBound = mid + 1 ;}return −1;} Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 52 / 55
53. 53. Recursive Binary Searchpublic int RbinSearch ( int value , int lower , int upper ) {i f ( lower > upper )return −1;else { int mid ; mid = ( int ) ( upper+lower ) / 2;i f ( value < arr [ mid ] ) RbinSearch ( value , lower , mid −1) ;e l s e i f ( value = arr [ mid ] ) return mid ;else RbinSearch ( value , mid +1 , upper )} } Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 53 / 55
54. 54. Iterative vs. Recursive Binary Search The main problem with the recursive binary search algorithm, as compared to the iterative algorithm, is its eﬃciency. When a 1,000-element array is sorted using both algorithms, the recursive algorithm is consistently 10 times slower than the iterative algorithm Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 54 / 55
55. 55. Excercises Just a Reminder Dr.Haitham A. El-Ghareeb (CIS) Data Structures and Algorithms - 2012 November 25, 2012 55 / 55