This document discusses various sorting algorithms and their time complexities. It covers common sorting algorithms like bubble sort, selection sort, insertion sort, which have O(N^2) time complexity and are slow for large data sets. More efficient algorithms like merge sort, quicksort, heapsort with O(N log N) time complexity are also discussed. Implementation details and examples are provided for selection sort, insertion sort, merge sort and quicksort algorithms.

1. Sorting
Algorithms

2. Motivation of Sorting
• The term list here is a collection of records.
• Each record has one or more fields.
• Each record has a key to distinguish one record with
another.
• For example, the phone directory is a list. Name,
phone number, and even address can be the key,
depending on the application or need.

3. Two Common Categories
Sorting Algorithms of
O(N^2)
• Bubble Sort
• Selection Sort
• Insertion Sort
Sorting Algorithms of
O(N log N)
• Heap Sort
• Merge Sort
• Quick Sort

4. For small values of N
• It is important to note that all algorithms
appear to run equally as fast for small values
of N.
• For values of N from the thousands to the
millions, The differences between O(N^2)
and O(N log N) become dramatically
apparent

5. O(N^2) Sorts
• Easy to program
• Simple to understand
• Very slow, especially for large values of N
• Almost never used in professional software

6. Selection Sort
• More efficient than Bubble Sort.
• Works by finding the largest element in the
list and swapping it with the last element,
effectively reducing the size of the list by 1.

7. Selection Sort Algorithm
void SelectionSort()
{
for (int i = 0; i < n-1; i++)
{
int indx=i, small = a[i];
for (int j = i+1; j < n; j++ )
{
if (a[j] < small)
{
small = a[j];
indx = j;
}
}
a[indx] = a[i];
a[i] = small;
}
}

8. Insertion Sort
• while some elements unsorted:
– Using linear search, find the location in the sorted portion
where the 1st element of the unsorted portion should be
inserted
– Move all the elements after the insertion location up one
position to make space for the new element
13 21
45 79 47 22
38 74 36
66 94 29
57 81
60 16
45
66
60
45
the fourth iteration of this loop is shown here

9. An insertion sort partitions the array into two regions

10. An insertion sort of an array of five integers

11. Insertion Sort Algorithm
void insertionSort()
{
for (int i = 1; i < n; i++)
{
int pos;
int y = a[i];
// Shuffle up all sorted items > arr[i]
for (pos = i-1; pos >=0 && y < a[pos]; pos--)
a[pos+1] = a[pos];
// Insert the current item
a[pos+1] = y;
}
}

12. O(N log N) Sorts
• Fast
• Efficient
• Complicated, not easy to understand
• Most make extensive use of recursion and
complex data structures

13. 13
Overview
• Divide and Conquer
• Merge Sort
• Quick Sort

14. 14
Divide and Conquer
1. Base Case, solve the problem directly if it is
small enough
2. Divide the problem into two or more similar
and smaller subproblems
3. Recursively solve the subproblems
4. Combine solutions to the subproblems

15. 15
Divide and Conquer - Sort
Problem:
• Input: A[left..right] – unsorted array of integers
• Output: A[left..right] – sorted in non-decreasing order

16. 16
Divide and Conquer - Sort
1. Base case
at most one element (left ≥ right), return
2. Divide A into two subarrays: FirstPart, SecondPart
Two Subproblems:
sort the FirstPart
sort the SecondPart
3. Recursively
sort FirstPart
sort SecondPart
4. Combine sorted FirstPart and sorted SecondPart

17. 17
Merge Sort: Idea
Merge
Recursively sort
Divide into
two halves
FirstPart SecondPart
FirstPart SecondPart
A
A is sorted!

18. 18
Merge Sort: Algorithm
Merge-Sort (A, left, right)
if left ≥ right return
else
middle =(left+right)/2
Merge-Sort(A, left, middle)
Merge-Sort(A, middle+1, right)
Merge(A, left, middle, right)
Recursive Call

19. 19
A[middle]
A[left]
Sorted
FirstPart
Sorted
SecondPart
Merge-Sort: Merge
A[right]
merge
A:
A:
Sorted

20. 20
6 10 14 22
3 5 15 28
L: R:
Temporary Arrays
5 15 28 30 6 10 14
5
Merge-Sort: Merge Example
2 3 7 8 1 4 5 6
A:

21. 21
Merge-Sort: Merge Example
3 5 15 28 30 6 10 14
L:
A:
3 15 28 30 6 10 14 22
R:
i=0 j=0
k=0
2 3 7 8 1 4 5 6
1

22. 22
Merge-Sort: Merge Example
1 5 15 28 30 6 10 14
L:
A:
3 5 15 28 6 10 14 22
R:
k=1
2 3 7 8 1 4 5 6
2
i=0 j=1

23. 23
Merge-Sort: Merge Example
1 2 15 28 30 6 10 14
L:
A:
6 10 14 22
R:
i=1
k=2
2 3 7 8 1 4 5 6
3
j=1

24. 24
Merge-Sort: Merge Example
1 2 3 6 10 14
L:
A:
6 10 14 22
R:
i=2 j=1
k=3
2 3 7 8 1 4 5 6
4

25. 25
Merge-Sort: Merge Example
1 2 3 4 6 10 14
L:
A:
6 10 14 22
R:
j=2
k=4
2 3 7 8 1 4 5 6
i=2
5

26. 26
Merge-Sort: Merge Example
1 2 3 4 5 6 10 14
L:
A:
6 10 14 22
R:
i=2 j=3
k=5
2 3 7 8 1 4 5 6
6

27. 27
Merge-Sort: Merge Example
1 2 3 4 5 6 14
L:
A:
6 10 14 22
R:
k=6
2 3 7 8 1 4 5 6
7
i=2 j=4

28. 28
Merge-Sort: Merge Example
1 2 3 4 5 6 7 14
L:
A:
3 5 15 28 6 10 14 22
R:
2 3 7 8 1 4 5 6
8
i=3 j=4
k=7

29. 29
Merge-Sort: Merge Example
1 2 3 4 5 6 7 8
L:
A:
3 5 15 28 6 10 14 22
R:
2 3 7 8 1 4 5 6
i=4 j=4
k=8

30. 30
Merge(A, left, middle, right)
1. n1 = middle – left + 1
2. n2 = right – middle
3. create array L[n1], R[n2]
4. for i = 0 to n1-1 do L[i] = A[left +i]
5. for j = 0 to n2-1 do R[j] = A[middle+1+j]
6. k = i = j = 0
7. while (i < n1 & j < n2)
8. if ( L[i] < R[j])
9. A[k++] = L[i++]
10. else
11. A[k++] = R[j++]
12. while i < n1
13. A[k++] = L[i++]
14. while j < n2
15. A[k++] = R[j++]

31. 31
6 2 8 4 3 7 5 1
6 2 8 4 3 7 5 1
Merge-Sort(A, 0, 7)
Divide
A:

32. 32
6 2 8 4
3 7 5 1
6 2 8 4
Merge-Sort(A, 0, 3) , divide
A:
Merge-Sort(A, 0, 7)

33. 33
3 7 5 1
8 4
6 2
6 2
Merge-Sort(A, 0, 1) , divide
A:
Merge-Sort(A, 0, 7)

34. 34
3 7 5 1
8 4
6
2
Merge-Sort(A, 0, 0) , base case
A:
Merge-Sort(A, 0, 7)

35. 35
3 7 5 1
8 4
6 2
Merge-Sort(A, 0, 0), return
A:
Merge-Sort(A, 0, 7)

36. 36
3 7 5 1
8 4
6
2
Merge-Sort(A, 1, 1) , base case
A:
Merge-Sort(A, 0, 7)

37. 37
3 7 5 1
8 4
6 2
Merge-Sort(A, 1, 1), return
A:
Merge-Sort(A, 0, 7)

38. 38
3 7 5 1
8 4
2 6
Merge(A, 0, 0, 1)
A:
Merge-Sort(A, 0, 7)

39. 39
3 7 5 1
8 4
2 6
Merge-Sort(A, 0, 1), return
A:
Merge-Sort(A, 0, 7)

40. 40
3 7 5 1
8 4
2 6
Merge-Sort(A, 2, 3)
4
8
, divide
A:
Merge-Sort(A, 0, 7)

41. 41
3 7 5 1
4
2 6
8
Merge-Sort(A, 2, 2), base case
A:
Merge-Sort(A, 0, 7)

42. 42
3 7 5 1
4
2 6
8
Merge-Sort(A, 2, 2), return
A:
Merge-Sort(A, 0, 7)

43. 43
4
2 6
8
Merge-Sort(A, 3, 3), base case
A:
Merge-Sort(A, 0, 7)

44. 44
3 7 5 1
4
2 6
8
Merge-Sort(A, 3, 3), return
A:
Merge-Sort(A, 0, 7)

45. 45
3 7 5 1
2 6
4 8
Merge(A, 2, 2, 3)
A:
Merge-Sort(A, 0, 7)

46. 46
3 7 5 1
2 6 4 8
Merge-Sort(A, 2, 3), return
A:
Merge-Sort(A, 0, 7)

47. 47
3 7 5 1
2 4 6 8
Merge(A, 0, 1, 3)
A:
Merge-Sort(A, 0, 7)

48. 48
3 7 5 1
2 4 6 8
Merge-Sort(A, 0, 3), return
A:
Merge-Sort(A, 0, 7)

49. 49
3 7 5 1
2 4 6 8
Merge-Sort(A, 4, 7)
A:
Merge-Sort(A, 0, 7)

50. 50
1 3 5 7
2 4 6 8
A:
Merge (A, 4, 5, 7)
Merge-Sort(A, 0, 7)

51. 51
1 3 5 7
2 4 6 8
Merge-Sort(A, 4, 7), return
A:
Merge-Sort(A, 0, 7)

52. 52
Quick Sort
• Divide:
• Pick any element p as the pivot, e.g, the first element
• Partition the remaining elements into
FirstPart, which contains all elements < p
SecondPart, which contains all elements ≥ p
• Recursively sort the FirstPart and SecondPart
• Combine: no work is necessary since sorting
is done in place

53. 53
Quick Sort
x < p p p ≤ x
Partition
FirstPart SecondPart
p
pivot
A:
Recursive call
x < p p p ≤ x
Sorted
FirstPart
Sorted
SecondPart
Sorted

54. 54
Quick Sort
Quick-Sort(A, left, right)
if left ≥ right return
else
middle = Partition(A, left, right)
Quick-Sort(A, left, middle–1 )
Quick-Sort(A, middle+1, right)
end if

55. 55
Partition
p
p x < p p ≤ x
p p ≤ x
x < p
A:
A:
A:
p

56. 56
Partition Example
A: 4 8 6 3 5 1 7 2

57. 57
Partition Example
A: 4 8 6 3 5 1 7 2
i=0
j=1

58. 58
Partition Example
A:
j=1
4 8 6 3 5 1 7 2
i=0
8

59. 59
Partition Example
A: 4 8 6 3 5 1 7 2
6
i=0
j=2

60. 60
Partition Example
A: 4 8 6 3 5 1 7 2
i=0
3
8
3
j=3
i=1

61. 61
Partition Example
A: 4 3 6 8 5 1 7 2
i=1
5
j=4

62. 62
Partition Example
A: 4 3 6 8 5 1 7 2
i=1
1
j=5

63. 63
Partition Example
A: 4 3 6 8 5 1 7 2
i=2
1 6
j=5

64. 64
Partition Example
A: 4 3 8 5 7 2
i=2
1 6 7
j=6

65. 65
Partition Example
A: 4 3 8 5 7 2
i=2
1 6 2
2 8
i=3
j=7

66. 66
Partition Example
A: 4 3 2 6 7 8
i=3
1 5
j=8

67. 67
Partition Example
A: 4 1 6 7 8
i=3
2 5
4
2 3

68. 68
A: 3 6 7 8
1 5
4
2
x < 4 4 ≤ x
pivot in
correct position
Partition Example

69. 69
Partition(A, left, right)
1. x = A[left]
2. i = left
3. for j = left+1 to right
4. {
5. if (A[j] < x){
6. i = i + 1
7. swap(A[i], A[j])
8. }
9. }
10. swap(A[i], A[left])
11. return i

70. 70
4 8 6 3 5 1 7 2
2 3 1 5 6 7 8
4
Quick-Sort(A, 0, 7)
Partition
A:

71. 71
2 3 1
5 6 7 8
4
2
1 3
Quick-Sort(A, 0, 7)
Quick-Sort(A, 0, 2)
A:
, partition

72. 72
2
5 6 7 8
4
1
1 3
Quick-Sort(A, 0, 7)
Quick-Sort(A, 0, 0) , base case
, return

73. 73
2
5 6 7 8
4
1
3
3
Quick-Sort(A, 0, 7)
Quick-Sort(A, 2, 2) , base case

74. 74
5 6 7 8
4
2
1 3
2
1 3
Quick-Sort(A, 0, 7)
Quick-Sort(A, 2, 2), return
Quick-Sort(A, 0, 2), return

75. 75
4
2
1 3
5 6 7 8
6 7 8
5
Quick-Sort(A, 0, 7)
Quick-Sort(A, 2, 2), return
Quick-Sort(A, 4, 7) , partition

76. 76
4
5
6 7 8
7 8
6
6
2
1 3
Quick-Sort(A, 0, 7)
Quick-Sort(A, 5, 7) , partition

77. 77
4
5
6
7 8
8
7
2
1 3
Quick-Sort(A, 0, 7)
Quick-Sort(A, 6, 7) , partition

78. 78
4
5
6
7
2
1 3
Quick-Sort(A, 0, 7)
Quick-Sort(A, 7, 7)
8
, return
, base case
8

79. 79
4
5
6 8
7
2
1 3
Quick-Sort(A, 0, 7)
Quick-Sort(A, 6, 7) , return

80. 80
4
5
2
1 3
Quick-Sort(A, 0, 7)
Quick-Sort(A, 5, 7) , return
6 8
7

81. 81
4
2
1 3
Quick-Sort(A, 0, 7)
Quick-Sort(A, 4, 7) , return
5 6 8
7

82. 82
4
2
1 3
Quick-Sort(A, 0, 7)
Quick-Sort(A, 0, 7) , done!
5 6 8
7

83. Radix Sort
Extra information: every integer can be
represented by at most k digits
– d1d2…dk where di are digits in base r
– d1: most significant digit
– dk: least significant digit

84. Radix Sort
• Algorithm
– sort by the least significant digit first (counting
sort)
=> Numbers with the same digit go to same bin
– reorder all the numbers: the numbers in bin 0
precede the numbers in bin 1, which precede the
numbers in bin 2, and so on
– sort by the next least significant digit
– continue this process until the numbers have been
sorted on all k digits

85. Radix Sort
• Assume each key has a link field. Then the keys in the
same bin are linked together into a chain:
– f[i], 0 ≤ i ≤ r (the pointer to the first record in bin i)
– e[i], (the pointer to the end record in bin i)
– The chain will operate as a queue.

86. Radix Sort Example
179 208 306 93 859 984 55 9 271 33
list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10]
e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9]
f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9]
179
859
9
208
306
55
984
93
33
271
271 93 33 984 55 306 208 179 859 9
list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10]

87. Radix Sort Example (Cont.)
e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9]
f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9]
179
859
9
208
306 55 984 93
33 271
271 93 33 984 55 306 208 179 859 9
list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10]
306 208 9 33 55 859 271 179 984 93
list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10]

88. Radix Sort Example (Cont.)
e[0] e[1] e[2] e[3] e[4] e[5] e[6] e[7] e[8] e[9]
f[0] f[1] f[2] f[3] f[4] f[5] f[6] f[7] f[8] f[9]
179
55
33
9 859 984
306
271
9 33 55 93 179 208 271 306 859 948
list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10]
306 208 9 33 55 859 271 179 984 93
list[1] list[2] list[3] list[4] list[5] list[6] list[7] list[8] list[9] list[10]
93
208

89. Heap Sort
• Slowest O(N log N) algorithm.
• Although the slowest of the O(N log N)
algorithms, it has less memory demands than
Merge and Quick sort.

90. Heap Sort
Works by transferring items to a heap, which
is basically a binary tree in which all parent
nodes have greater values than their child
nodes. The root of the tree, which is the
largest item, is transferred to a new array and
then the heap is reformed. The process is
repeated until the sort is complete.

91. Converting An Array Into A Max Heap
26
15 48 19
5
1 61
77
11 59
77
15 1 5
61
48 19
59
11 26
[1]
[2] [3]
[4] [5]
[6] [7]
[8] [9] [10]
[2] [3]
[4] [5]
[6] [7]
[8] [9] [10]
(a) Input array (b) Initial heap
[1]

92. Heap Sort Example
61
1 5
48
15 19
59
11 26
[2] [3]
[4] [5]
[6] [7]
[8] [9]
[1]
Heap size = 9, Sorted =
[77]
59
5
48
15 19
26
11 1
[2] [3]
[5]
[6] [7]
[8]
[1]
Heap size = 8, Sorted =
[61, 77]