Successfully reported this slideshow.
Upcoming SlideShare
×

# Algorithms

4,107 views

Published on

• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

### Algorithms

1. 1. Sorting and Searching Algorithms Cutajar & Cutajar
2. 2. Sorting ID Card Surname Sorting means the arrangement of 712364 Cutajar records in a particular order, according to a specified sorting 345222 Zammit key. 778879 Sammut For simplicity we will consider 453211 Abela sorting of an array of records, all present in main memory. Sort key Sorting Generally sorting involves a ID Card Surname complexity of N2 where N is the number of records. This means 345222 Zammit that in a list of N records, sorting 453211 Abela normally involves scanning all the 712364 Cutajar N records for N times. 778879 Sammut Algorithms 2
3. 3. Considerations For Choosing A Sorting Method Number of items to be sorted. Initial arrangement of data Complexity of algorithm (hence its implementation) The relative speed of a given method Size of data items to be sorted. Algorithms 3
4. 4. Selection Sort This is a simple technique similar to the one performed by pencil and paper. It consists of repeatedly looking through a data array to find the lowest key (for sorting in ascending order). This element is than written to another array. The data element is than cancelled from the original array. ( sometimes a rogue value is just written) This procedure is repeated until all the records are sorted out of the original data array. n=3 data new pass 1 pass 2 pass 3 array array data new data new data new 234 234 156 156 156 320 320 320 234 234 n=3 156 320 Algorithms 4
5. 5. Selection Sort Algorithm In this algorithm, we use the same array instead of two arrays. We scan the data array and find the smallest item and place it in the first place, The smallest element is swapped with the considered element. We repeat with the second item and place it in the second place … etc. Consider the following data array: pass 1 pass 2 pass 3 pass 4 pass 5123 123 100 100 100 100213 213 213 123 123 123456 456 456 456 145 145145 145 145 456 456 213431 431 431 431 431 431100 100 123 123 213 145 213 213 456 431smallest element Algorithms 5
6. 6. Pseudo-CodeProgram Selection_Sortuse variables: numbers array[n] of type integer temp, count, pass, lowest of type integer for pass := 1 to n do lowest := pass for count := pass to n-1 {find smallest} if numbers[count+1] < numbers[lowest] lowest := count+1 end for temp := numbers[pass] {swap current with smallest} numbers[pass] := numbers[lowest] numbers[lowest] := temp end forend program Algorithms 6
7. 7. Insertion Sort In this algorithm the first two elements of the array are compared and arranged in order. The third element is compared with the first two and inserted in its correct position. This process is repeated until every element in the list has been inserted in its correct position. This method is similar to when we sort cards by hand. Initial array pass 1 pass 2 pass 3 pass 4 pass 5 21 16 16 16 16 12 16 21 21 21 19 16 35 35 35 21 19 47 47 35 21 19 47 35 12 47 Algorithms 7
8. 8. Insertion Sort Flowchart Start  Flowchart Notation:  CP(item) : item in location pointed at by CP  PP(item) : item in location pointed at by PP CP:=1  CP : value of pointer pointing to the number currently being inserted  PP : value of pointer pointing to the numbers (in CP:=CP+1 ordered list) that are being compared to CP(item) PP:=1 Y N PP:=PP+1 PP<=CP PP(item)>CP(item) ? ? NN Y CP=max? Exchange Temp:=CP(item) consecutive PP(item):=temp Y items down to PP+1 End Algorithms 8
9. 9. Insertion Sort Algorithm initially pass 1 pass 2 pass 3In this 123 123 123 123 123 123 123algorithm weuse the same 213 213 213 213 213 145array and shift 456 456 456 456 456 213 213the datadownwards to 145 145 145 145 145 456 456make space for 431 431 431 431 431 431 431the insertion 100 100 100 100 100 100 100 pass 4 pass 5 123 123 123 123 123 123 100 145 145 145 145 145 145 123 123 213 213 213 213 213 213 145 145 456 456 431 431 431 213 213 431 431 456 456 456 456 431 431 100 100 100 100 100 100 456 456 Algorithms 9
10. 10. Pseudo CodeProgram Insertion_Sortuse variables: numbers array[n] of type integer current, pass, position of type integer for pass := 1 to n-1 do current := numbers[pass] position := pass while (position > 0 AND numbers[position-1] > current ) numbers[position]:=numbers[position-1] {shift down} position := position-1 end while numbers[position] := current {insert current element in space} end forend program Algorithms 10
11. 11. Bubble Sort This is so called because the smallest element rises to the top of the array and then the next smallest „bubbles‟ up to the next position and so on. On the first pass the last two elements (n) and (n-1) of the array are compared and exchanged if necessary. This process is repeated with the (n-1) and (n-2) elements and then with the (n-2) and (n-3) and so on until the smallest element arrives at the top of the array. The next pass repeats the same procedure with the whole array except the already sorted elements. The passes end when the array is completely sorted Algorithms 11
12. 12. Bubble Sort Algorithminitially pass 1 pass 2 pass 3 pass 4 final 3 3 3 3 0 0 0 0 0 0 0 0 2 2 2 0 3 3 3 1 1 1 1 1 0 0 0 2 2 2 1 3 3 2 2 2 4 1 1 1 1 1 2 2 2 3 3 3 1 4 4 4 4 4 4 4 4 4 4 4 Algorithms 12
13. 13. Pseudo-CodeProgram bubble_sortuse variables numbers array[n] of type integer temp, pass, current, of type integer for pass := 1 to n-2 do for current := n downto pass+1 do if numbers[current] < numbers[current-1] then swap(numbers[current],numbers[current-1]) end for end forend programProcedure swap (parameters: variable a,b of type integer)use variables temp of type integer temp := a a := b b := tempend procedure Algorithms 13
14. 14. Computational Complexity Computational Complexity or simply Complexity is the number of steps or arithmetic operations required to solve the posed problem. The interesting aspect is usually how complexity scales with the size of the input (the "scalability"), where the size of the input is described by some number N. Thus an algorithm may have computational complexity O(N2) (of the order of the square of the size of the input), in which case if the input doubles in size, the computation will take four times as many steps. The ideal is a constant time algorithm (O(1)) or failing that, O(log2N) or O(N). O(N 2) Algorithms 14
15. 15. The Big-O Analysis In complexity O denotes “in the order of” It gives us an idea of the proportion of the efficiency of an algorithm with N and not an equality. Thus if the execution steps of an algorithm is given by:  a.N + b we say it is of the order of N or O(N)  a.(log2N) + b we say if is of the order of log2N or O(log2N)  a.N.(log2N) + b we say if is of the order of N.log2N or O(N.log2N)  a.N2 + b we say it is of the order of N2 or O(N2)  Or simply a we say it is constant or O(1).  If the complexity has an order higher then polynomial, say in exponential relation with N, it is considered impossible due to the large amount of time required to perform the required task. Algorithms 15
16. 16. Complexity Example Consider the following codes:Procedure Example1 Procedure Example2… b steps … b steps… … For j := 1 to 2N do For i : = 1 to 2N do … a1 steps For j := 1 to N do End for … For i := 1 to N do … a steps … a2 steps End for End for End forEnd Procedure End ProcedureThe execution steps would be b + a.N , The execution steps would be b + 2.a.N2where a = 2a1+a2 , and so the and so the complexity is still O(N2)complexity is still O(N) Algorithms 16
17. 17. Complexity Classes Most algorithms fall into one of the following types:Type Complexity CommentLogarithmic O(log2N) Very GoodLinear O(N) GoodLinear-Logarithmic O(N.Log2N) Fairly GoodQuadratic O(N2) OkPolynomial O(Nk) k ≥ 1 PoorExponential O(aN) a > 1 Awful Algorithms 17
18. 18. Complexity of Simple Sorts To analyse the complexity of the sorting algorithms seen so far, i.e. Insertion Sort, Selection Sort and Bubble Sort, let us analyse the number of comparisons made for a general list of N elements. In all these algorithms the sum of comparisons is:  N-1 for the pass 1 Sum = N-1 + N-2 + N-3 + … + 2 + 1  N-2 for the pass 2 Or 1 + 2 + 3 + … + N-2 + N-1  N-3 for the pass 3 If we add the two lines above we get 2*Sum  … 2*Sum = N + N + N + … + N + N  2 for pass N-2 i.e. N for (N-1) times  1 for pass N-1 Thus 2*Sum = N*(N-1) Therefore Sum = N*(N-1) 2 Thus Which could have been obtained by the sum of an arithmetic progression.Cplx = O(N2) Algorithms 18
19. 19. Quick Sort Complex type routine used when list of items to be sorted is large. Although Quicksort is faster than other methods when sorting large amounts of data, it is often slower (depending on both the implementation and the starting order) with less than about a dozen items. Hence quicksort programs sometimes include a switch to another method whenever the number remaining to be sorted drops below some arbitrary figure. Additionally if the unsorted list is already somewhat ordered the quicksort method becomes somewhat inefficient – the worst case for quicksort being an input list which is already in order! Algorithms 19
20. 20. Algorithm Select an item, usually the first item, of the unsorted list. This is called the Pivot Partition the remaining items into TWO sublists.  A LEFT SUBLIST, with data items LESS than the selected item  A RIGHT SUBLIST, with data items GREATER than the selected item. Place the pivot between these two sublists.  If left sublist contains more than one item Then Quicksort the left sublist  If right sublist contains more than one item Then Quicksort the right sublist. Algorithms 20
21. 21. How to PartitionPivot Value:= Table[First]Up := FirstDown = LastRepeat Increment Up until Table[Up] > Pivot Value Decrement Down until Table[Down] <= Pivot Value If Up < Down exchange their valuesUntil Up >= DownExchange Table[First] and Table[Down]Define Pivot Index as Down Algorithms 21
22. 22. Partitioning ExamplePivot Value:= Table[First] ; Up := First ; Down = Last Table 44 76 23 43 55 12 64 77 33 Pivot = 44 First Up Down LastIncrement Up until Table[Up] > Pivot ValueDecrement Down until Table[Down] <= Pivot Value Table 44 76 23 43 55 12 64 77 33 Pivot = 44 First Up Down LastIf Up < Down Exchange their values Table 44 33 23 43 55 12 64 77 76 Pivot = 44 First Up Down Last Algorithms 22
23. 23. Partitioning (cont…)Up is < Down so ContinuePivot = 44 44 33 23 43 55 12 64 77 76 First Up Down LastIncrement Up until Table[Up] > Pivot ValueDecrement Down until Table[Down] <= Pivot ValuePivot = 44 44 33 23 43 55 12 64 77 76 First Up Down LastIf Up < Down Exchange their valuesPivot = 44 44 33 23 43 12 55 64 77 76 First Up Down Last Algorithms 23
24. 24. Partitioning (cont…)Up is < Down so Continue Table 44 33 23 43 12 55 64 77 76 Pivot = 44 First Up Down LastIncrement Up until Table[Up] > Pivot ValueDecrement Down until Table[Down] <= Pivot Value Table 44 33 23 43 12 55 64 77 76 Pivot = 44 First Down Up LastNow Down < Up so exchange pivot value with Table[Down] Table 12 33 23 43 44 55 64 77 76 Pivot = 44 First Down Up Last Algorithms 24
25. 25. Partitioning (cont…) Note that all values under the Pivot Index are smaller than the Pivot Value and all values above the Pivot Index are Larger than the Pivot Value Partition 1 Partition 2 Table 12 33 23 43 44 55 64 77 76 Pivot = 44 First 1 Last 1 First 2 Last 2 Pivot Index This gives us two sub-arrays to re-partition Algorithms 25
26. 26. Quick Sort AlgorithmProcedure QuickSort( use variables First, Last : integer)Use Varianbles PivIndex: integer;If (First < Last) then PivIndex = Partition(First,Last); QuickSort(First, Pivindex-1); QuickSort(Pivindex+1, Last);EndifEnd Procedure; Algorithms 26
27. 27. Quick Sort Example Consider the following list 12 33 23 43 44 55 64 77 76 12 33 23 43 55 64 77 76 23 33 43 64 77 76 23 43 76 77 77 12 23 33 43 44 55 64 76 77 Algorithms 27
28. 28. Complexity of Quick Sort Let us analyse the number of comparisons that are made in this algorithm:  N for the first pass where all the elements are compared with the pivot  2*N/2 for the next pair of passes where N/2 elements in each “half” of the original array are compared to their own pivot values.  4*N/4 for the next four passes where N/4 elements in each “quarter” of the original array are compared to their own pivot values. How Many Partitions Occur ?? Algorithms 28
29. 29. How Many PartitionsIt depends on the order of the original array elements:  If each partition divides the sub-array approximately in half, there will be only log2N partitions made, and so Quicksort is O(Nlog2N).  But, if the original array was sorted to begin with, the recursive calls will partition the array into parts of unequal length, with one part empty, and the other part containing all the rest of the array except for the pivot value itself. In this case, there can be as many as N-1 partitions made, and QuickSort will have O(N2). Best Case: 3 (log28) comparisons of 8 elements Worst Case: 7 comparisons of 8 elements Algorithms 29
30. 30. Comparison Of Sorting Routines Relative speeds for sorting random integers, using different methods.  Bubble Sort  Insertion Sort  Selection Sort  Quick Sort Algorithms 30
31. 31. Merge Sort In MergeSort, the list to be sorted is successively subdivided in two until the number of elements in the sub-list remain one or two. Subsequently they are merged together in order in such a way that after successive merges the whole list is recomposed in the desired sorting order. This algorithm lends itself well to a recursive method of programming. Algorithms 31
32. 32. Merge Sort Algorithm1. If the input sequence has fewer than two elements, return.2. Partition the input sequence into two halves.3. Sort the two subsequences using the same algorithm.4. Merge the two sorted subsequences to form the output sequence.MergeSort(list, first, last)if (first < last) middle = (first + last) div 2 MergeSort(list, first, middle) MergeSort(list, middle+1, last) Merge(list, first,middle, last)endif list 44 76 23 43 55 12 64 77 33 First Last Algorithms 32
33. 33. The Merge Algorithm (Part I)void Merge(use variables A[] array of integer; f, m, l : integer)first1 := f; last1 := m; first2 := m+1; last2 := last;index = first1;B[SIZE] : array of integer;while((first1 <= last1) && (first2 <= last2)) if(A[first1] < A[first2]) then A 44 76 23 43 B[index] = A[first1]; first1:= first1+1; First1 Last1 First2 Last2 else B[index] = A[first2]; first2:=first2+1; B 23 43 44 76 Endif index := index+1; f lEnd While Algorithms 33
34. 34. The Merge Algorithm (Part II)while(first1 <= last1) finish off first sub-array if necessary B[index] = A[first1]; first1:=first1+1; index:=index+1;Endwhilewhile(first2 <= last2) B[index] = A[first2]; finish off second sub-array if necessary first2:= first2+1; index:=index+1;EndwhileFor index := f to l Copy Temporary array back to original array A[index] = B[index]; Algorithms 34
35. 35. Merge Sort Complexity The entire array can be subdivided into halves only log2N times. Each time it is subdivided, function Merge is called to re-combine the halves. Function Merge uses a temporary array to store the merged elements. Merging is O(N) because it compares each element in the sub-arrays. Copying elements back from the temporary array to the values in the array is also O(N) Thus Merge Sort is O(N.log2N) Algorithms 35
36. 36. Tree Sort (Tournament Sort)Algorithm: Transform the unsorted list into a binary search tree. In a binary search tree, every node has the following property:  All of its left descendants are smaller in value than the value of the node itself, and all of its right descendants are larger than its value. Traverse the resultant binary search tree in order. Algorithms 36
37. 37. Tree Sort Example 2 7 Consider the 1 4 following unsorted 3 8 list of numbers: 1 3 5  27 48 13 50 9 9 0 39 77 82 91 7 65 19 70 66 7 Creating the 8 binary search tree 6 5 2 for this given list: 7 9 0 1 Traversing this resultant tree in order we 6get the sorted list: 613 19 27 39 48 50 65 66 70 77 82 91. Algorithms 37
38. 38. Tree Sort Complexity Building of the tree has a complexity of O(N). This is done just one time in the algorithm. To read back the tree into a sorted array the visit of the tree must be performed N times with a complexity depending on the distribution of the tree.  In the best case the tree is perfectly balanced, i.e. the maximum difference between the lowest leaf and the highest leaf is 1 level. Then it would require log2N for each element thus leading to a total complexity of O(N.log2N)  In the worst case the tree is totally unbalanced, which is similar to a simple linked list. This would lead an average time of N/2 to read each of the N elements thus giving an overall complexity of O(N2). This occurs when the original list is already ordered and thus all children are placed on the right of the parent node  Note that this algorithm requires N extra memory space to build the tree. Algorithms 38
39. 39. Complexity SummarySorting Algorithm Best Case Average Case Worst CaseSelection Sort O(N2) O(N2) O(N2)Insertion Sort O(N2) O(N2) O(N2)Bubble Sort O(N2) O(N2) O(N2)Merge Sort O(N.log2N) O(N.log2N) O(N.log2N)Quick Sort O(N.log2N) O(N.log2N) O(N2)Tree Sort O(N.log2N) O(N.log2N) O(N2) Algorithms 39
40. 40. Comments on Sorting Algorithms Bubble, Insertion and Selection sort routines are preferable when list of items to be sorted consists of a few elements. Bubble sort is the slowest in execution but the easiest method (and simplest implementation). Insertion and Selection sorts have approximately the same speeds and both are usually marginally faster than a bubble sort, implementation (programs) are short and simple (advantage) Quick Sort and Tree Sort are far faster than the above methods when large quantities of items are to be sorted. They both suffer from the initial arrangement of the list problem. Merge Sort doesn‟t suffer from the initial arrangement problem. However Merge Sort and Tree Sort require an extra memory space for the temporary array or the binary tree. However the algorithms for the later three (and implementation) are much more complex. Algorithms 40
41. 41. Linear Search This can be a matter of looking through the array, element by element sequentially until 342 456 the required key is found. 234 Since the particular key may be absent from 123 the unsorted array, searching the array may require the search through the entire array. 675 Thus the complexity of this type of 455 searching is of the order of N, O(N). N 664 This is clearly an inefficient method, called direct or linear search, and is only used for 344 small arrays. 888 Sorting the array on the search key can improve the efficiency by stopping when the 645 the particular key is found or a greater one 456 456 is found instead. Algorithms 41
42. 42. Pseudo-CodeProgram direct_search use variables: numbers array[N] of type integer key, item of type integer found of type boolean found := FALSE repeat if key = numbers[item] then found := TRUE else item := item + 1 until found = TRUE OR item = Nend program Algorithms 42
43. 43. Binary Search This algorithm, although performing more computation is more efficient since it performs a less number of comparisons. The method involves finding the center of the array and comparing the search key with that element.  If the search key is equal then the element is found.  If the search key is less than the element then the key must be in the lower half of the array so the same procedure is repeated on the lower half.  If the search key is greater than the element then the key must be in the upper half of the array so the same procedure is repeated on the upper half.  An allowance must be taken for the case where the key doesn‟t exist in the array The complexity of this algorithm is thus log2(N) NOTE: This algorithm works only on sorted arrays ! Algorithms 43
44. 44. Binary Search Flowchart Start Consider the entire input list l S NOT FOUND Compare S to S FOUND middle item of considered list Output YES Is S < NO Considered item? „found‟ Take the Take theconsidered list as considered list asthe lower half of the upper half of the list the listNO Is Considered list YES Output „not empty? found‟ End Algorithms 44
45. 45. Binary Search Algorithm pass 1 pass 2 pass 3123 123 123234 234 234342 342 342344 344 344455 456 455 455456 456 456 456645 645 645664 664 456 664675 675 675888 888 888 Algorithms 45
46. 46. Pseudo-CodeProgram Binary_Search use variables numbers array[N] of type integer start, middle, end, key of type integer found of type boolean start := 0 end := N-1 found := false repeat middle := ( end + start ) div 2 if key = numbers[middle] then found := TRUE else if key > numbers[middle] then start := middle + 1 else if key < numbers[middle] then end := middle – 1 until found = TRUE OR start > endend program Algorithms 46