1.
Algorithms Sorting Vicente García Díaz – garciavicente@uniovi.es University of Oviedo, 2013
2.
2Table of contents Sorting1. Basic concepts2. Sorting algorithms ▫ Simple algorithms Sorting by direct exchange Sorting by insertion Sorting by selection ▫ Sophisticated algorithms Sorting by direct exchange Sorting by insertion Sorting by selection ▫ Constrained algorithms Radix ▫ External algorithms
3.
4Basic concepts Introduction • Given a set of n elements a1, a2, a3, …, an and an order relation (≤) the problem of sorting is to sort these elements increasingly • What can influence the sorting? ▫ The type ▫ The size ▫ The device on which they are • Integers stored in a vector (simplification)
4.
5Basic concepts Sorting integers • In practice, the problems tend to be much more complex ▫ However they can be reduced to the same problem that the integers (using registry keys, indexes, etc.) • In essence it is the same to sort numbers or listed streets, houses, cars or any numerically quantifiable property
5.
6Basic concepts Classification criteria for algorithms 1. Total number of steps ▫ Complexity 2. Number of comparisons performed 3. Number of interchanges performed ▫ Much more expensive than the comparison 4. Stability 5. Type of memory used ▫ Internal algorithms ▫ External algorithms
6.
7Basic concepts Preliminary considerations (I) • We will sort arrays of integers using the Java programming language • We will make use of utility methods to facilitate the work and make the code more readable
7.
8Basic concepts Preliminary considerations (II)
8.
10Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Bubble External algorithms • Description ▫ Based on the successive 1 comparison and exchange of 2 adjacent elements ▫ At each step, each item is 3 compared with the previous 4 one and in case of being 6 5 disordered, they are exchanged 7 ▫ In the first step, the smallest 8 element will be placed in the leftmost position, and so on…
10.
12Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Bubble External algorithms Best case: O(n2) • High number of exchanges • High number of comparisons Worst case: O(n2) Average case: O(n2)
11.
13Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Bubble with sentinel External algorithms • Description ▫ Observing the previous result 1 you can see that the last steps 2 are repeated unnecessarily ▫ To avoid this you can enter a 3 stop condition when the vector 4 is sorted 6 5 If you make a pass and you do 7 not make any exchange it 8 means that it is already sorted
13.
15Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Bubble with sentinel External algorithms Best case: O(n)• Complexity whether the input would be 8 1 2 3 4 5 6 7? Worst case: O(n2) Average case: O(n2)
15.
17Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Bidirectional bubble External algorithms • Description ▫ Used to prevent problems when 1 the vector is almost sorted 2 ▫ It operates in both directions at the same time 3 4 6 5 7 8
17.
19Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Bidirectional bubble External algorithms Best case: O(n) Worst case: O(n2) • It still has a high number of comparisons and exchanges, just as the other methods of bubble Average case: O(n2)
18.
20Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Direct insertion External algorithms • Description ▫ The vector is divided into two “virtual” parts: an ordered and Unordered sequence an unordered sequence 6 8 2 5 ▫ At each step we take the first element of the unordered sequence and insert it into the corresponding place of the ordered sequence 1 3 4 7 Ordered sequence
21.
23Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Direct insertion External algorithms Best case: O(n) • High number of exchanges Worst case: O(n2) • Less comparisons than the bubble method Average case: O(n2)
22.
24Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Binary insertion External algorithms • Description ▫ The direct insertion can be improved taking into account Unordered sequence that the sequence in which the 6 8 2 5 elements are inserted is already sorted ▫ We use binary search to quickly locate the suitable position for the insertion 1 3 4 7 ▫ *** The binary search was discussed Ordered sequence in the introduction to Algorithms
23.
25Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Binary insertion External algorithms • Initial vector: 4 5 6 1 3 2 7 8 1 4 5 6 1 3 2 7 8 2 4 5 6 1 3 2 7 8 3 1 4 5 6 3 2 7 8 4 1 3 4 5 6 2 7 8 5 1 2 3 4 5 6 7 8 6 1 2 3 4 5 6 7 8 7 1 2 3 4 5 6 7 8 The result is exactly the same as with the direct insertion
24.
26Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Binary insertion External algorithms • Less comparisons than the direct insertion Best case: O(n logn) • High number of exchanges Worst case: O(n2) • THE INSERTION MOVING ELEMENTS ONE POSITION Average case: O(n2) IS NOT ECONOMIC
25.
27Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Direct selection External algorithms • Description ▫ It consists on selecting the smallest element and exchange it with the first element ▫ Repeat the process with the 1 2 3 4 7 5 3 2 remaining elements until there is only one element (the greatest of all)
27.
29Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection Direct selection External algorithms • The number of exchanges is minimum Best case: O(n2) • It is predictable: the number of exchanges and Worst case: O(n2) comparisons depends on n Average case: O(n2) • The number of comparisons is very high
28.
30Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection ShellSort External algorithms • Description ▫ The idea is to divide the vector of elements in smaller virtual Unordered sequence vectors 6 8 2 5 ▫ Each subvector is sorted using an insertion sort algorithm ▫ Each element of the subvectors is positioned at a fixed distance which decreases in each 1 3 4 7 iteration (increments sequence) Ordered sequence
29.
31Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection ShellSort External algorithms 59 20 17 13 28 14 23 83 36 98 11 70 65 41 42 15 • Increments sequence: {1, 2, 4, 8} (example)Iteration 18 sublists of size 2 36 20 11 13 28 14 23 15 59 98 17 70 65 41 42 83(increment of 8)Iteration 24 sublists of size 4 28 14 11 13 36 20 17 15 59 41 23 70 65 98 42 83(increment of 4)Iteration 32 sublists of size 8 11 13 17 14 23 15 28 20 36 41 42 70 59 83 65 98(increment of 2)Iteration 41 sublists of size 16 11 13 14 15 17 20 23 28 36 41 42 59 65 70 83 98(increment of 1) **The elements of the sequence of increments should not be multiples of each other
31.
33Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection ShellSort External algorithms The complexity depends on the sequence of increments We do not know the sequence that works best Knuth recommends: hk = 3hk-1 + 1 1, 4, 13, 40, …h1 = 1 hk = 2hk-1 + 1 1, 3, 7, 15, 31, … O(n1.2) • Can be optimized by distributing the processing of each subvector across multiple processors
32.
34Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection ShellSort External algorithms • Principle on which relies ▫ THEOREM: If a sequence is sorted in n by n and subsequently ordered at intervals of j by j, it will also be ordered in n by n • The intervals should be decreasing • The last interval must be 1 • The interval values should not be multiples of each other (for efficiency)
33.
35Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection ShellSort External algorithms ShellSort Direct insertion VS
34.
36Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection HeapSort External algorithms Unsorted area of the vector • Description ▫ The idea is to create a heap of 5 4 maximums in the vector ▫ Subsequently, the first element is 4 3 3 2 chosen (maximum) and exchanged with the final element of the 1 2 vector (sorted area) 1 ▫ The heap is reorganized and we again choose the first element Sorted area of the vector (maximum) to place it in the last unsorted position of the vector The process is repeated until all the 4 5 6 7 9 elements of the unsorted vector are placed in the sorted vector
35.
37Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection HeapSort External algorithms • Idea of the algorithm CREATE HEAP OF MAXIMUMS IN THE FIRST PART OF VECTOR O(n) First part GET THE ROOT ELEMENT AND EXCHANGE IT WITH THE LAST ELEMENT OF THE VECTOR (LAST POSITION) O(1)Second part REPEAT UNTIL THE HEAP IS EMPTY O(n) REORGANIZE THE HEAP OF MAXIMUMS O(log n) GET THE ROOT ELEMENT AND EXCHANGE IT WITH THE NEXT UNSORTED ELEMENT OF THE VECTOR O(1)
43.
45Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection HeapSort External algorithms • Initial vector: 2 1 3 4 5 6 7 8 2. Exchange and restructure the heap 1 sinks 1 1 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
44.
46Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection HeapSort External algorithms Best case: O(n logn) • Not recommended for small vectors Worst case: O(n logn) • Fast for large vectors Average case: O(n logn)
45.
47Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection HeapSort External algorithms • Peculiarities ▫ First all big elements are moved to the left ▫ Then they are moved to the right ▫ The best case does not have to be the ordered vector ▫ The worst case does not have to be the unordered vector
46.
48Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection QuickSort External algorithms Partitioned element • Description ▫ It is based on partitioning ▫ When you partition an item, 2 3 1 4 7 6 that item will be in its corresponding position Then, the idea is to partitionate 1 2 3 6 7 all the elements ▫ You have to choose a good element to partitionate (pivot) 1 3 7 so that it is in the center and it creates a tree (if would be on one corner it will create a list) ▫ The implementation is recursive
47.
49Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection QuickSort External algorithms • Criteria for choosing a good pivot 1 3 8 7 2 4 6 5 ▫ The median It is the ideal solution but it is very expensive to compute ▫ The first element It is "cheap" but it is a bad choice ▫ The last element Occurs as with the first element ▫ A random element Statistically it is not the worst choice, but has computational cost ▫ The central element Statistically it is not a bad choice ▫ A compromise solution (median-of-3) We obtain a sample of 3 elements and calculate the median It does not guarantee anything but it can be a good indicator We chose the first element, the last element and the center element We order the elements, and we assume that the median is the element which is in the center
48.
50Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection QuickSort External algorithms • Idea of the algorithm REPEAT UNTIL ALL THE ELEMENTS ARE SORTED O(log n) …O(n) First CHOOSE A PIVOT Using median-of-3 is O(1) part PARTITIONING THE PIVOT THROUGH A PARTITIONING STRATEGY Typical case O(n) Second part
49.
51Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection QuickSort External algorithms • Initial vector: 4 5 6 1 3 2 7 8 Choice of pivot 1 4 5 6 1 3 2 7 8 1 1 5 6 8 3 2 7 4 Exchange 1 i j 1 1 1 2 6 8 3 5 7 4 1 5 6 4 3 2 7 8 Partitioning strategy Pivot 1 1 2 6 8 3 5 7 4 Exchange 2 1 1 5 6 8 3 2 7 4 i j i j 1 1 2 3 8 6 5 7 4Search for an element[i] > pivot and anelement[j] < pivot and interchange themi is moved to the right and j is moved to the left 1 1 2 3 8 6 5 7 4displacing elements element[j] < pivot to the left j i Positioningand elements element[i] > pivot to the right(Ends when j and i se are crossed, determining i 1 1 2 3 4 6 5 7 8the final position)
50.
52Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection QuickSort External algorithms • Initial vector: 4 5 6 1 3 2 7 8 1 2 3 4 6 5 7 8 Choice of pivot Choice of pivot 2 1 2 3 2 6 5 7 8 2 1 2 3 2 5 6 7 8Partitioning strategy Partitioning strategy Pivot 2 5 8 7 6 2 1 2 3 Positioning 2 5 8 7 6 j i i jBeing 3 items or less, the 2 5 6 7 8elements are already sortedwhen searching the median of 3(ends the recursion in thatbranch)
51.
53Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection QuickSort External algorithms • Initial vector: 4 5 6 1 3 2 7 8 5 6 7 8 Choice of pivot 3 7 8 3 5 3 7 8 There is only one element, then there is no need to do the Choice of pivot median of 3 because the element is already sorted (ends the recursion in that branch) 3 7 8 Being 3 items or less, the elements are already sorted when searching the median of 3 (ends the recursion in that branch)
53.
55Sorting algorithms Simple algorithms Sorting by direct exchange Sophisticated algorithms Sorting by insertion Constrained algorithms Sorting by selection QuickSort External algorithms Best case: O(n logn) • Very fast for values of n > 20 Worst case: O(n2) • Optimizable distributing the processing of each Average case: O(n logn) partition in different threads in parallel
54.
56Sorting algorithms Comparison of algorithms (I) • Data consists of only one key n = 256 Sorted Random Inverse n = 512 Sorted Random Inverse Bubble 540 1026 1492 Bubble 2165 4054 5931 Bubble with sent. 5 1104 1645 Bubble with sent. 8 4270 6542 Bidirect. bubble 5 961 1619 Bidirect. bubble 9 3642 6520 Direct insertion 12 366 704 Direct insertion 23 1444 2836 Binary insertion 56 373 662 Binary insertion 125 1327 2490 Direct selection 489 509 695 Direct selection 1907 1956 2675 ShellSort 58 127 157 ShellSort 116 349 492 HeapSort 116 110 104 HeapSort 253 241 226 QuickSort 31 60 37 QuickSort 69 146 79
55.
57Sorting algorithms Comparison of algorithms (II) • Data consists of only one key VS data with a size 7 times the key size n = 256 Sorted Random Inverse n = 256 Sorted Random Inverse Bubble 540 1026 1492 Bubble 610 3212 5599 Bubble with sent. 5 1104 1645 Bubble with sent. 5 3237 5762 Bidirect. bubble 5 961 1619 Bidirect. bubble 5 3071 5757 Direct insertion 12 366 704 Direct insertion 46 1129 2150 Binary insertion 56 373 662 Binary insertion 76 1105 2070 Direct selection 489 509 695 Direct selection 547 607 1430 ShellSort 58 127 157 ShellSort 186 373 435 HeapSort 116 110 104 HeapSort 264 246 227 QuickSort 31 60 37 QuickSort 55 137 75
56.
58Sorting algorithms Simple algorithms Sophisticated algorithms Constrained algorithms Radix External algorithms • Description 8 0 5 ▫ Its actually a external sorting 3 1 2 6 4 method ▫ It is not COMPARATIVE ▫ It can be used to sort sequences of digits that support a lexicographic order (words, numbers, dates, ...) ▫ It consists of defining K queues[0, k-1], being k the possible Q0 Q1 … Q9 values that can take each of the digits that make up the sequence ▫ Then the elements are distributed in the queues carrying different passes (ordered from the least significant digit to the most significant)
58.
60Sorting algorithms Simple algorithms Sophisticated algorithms Constrained algorithms Radix External algorithms Best case: O(k * n) = O(n) • Needs external data structures Worst case: O(k * n) = O(n) • Not very good if many digits in each element Average case: O(k * n) = O(n) • It is very efficient **K is the number of digits of each element
59.
61Sorting algorithms Simple algorithms Sophisticated algorithms Constrained algorithms External algorithms External algorithms • They are based on the principle of DIVIDE AND CONQUER • They use external memory ▫ The slowest of them is like Quicksort • They use the same basic principles that internal sorting algorithms • Different algorithms: ▫ Direct mixture ▫ Natural mixture ▫ Balanced mixture ▫ Polyphasic mixture
60.
62BibliographyJUAN RAMÓN PÉREZ PÉREZ; (2008) Introducción al diseño y análisis de algoritmos enJava. Issue 50. ISBN: 8469105957, 9788469105955 (Spanish)
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.
Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.