Comparison sorting algorithms work by making pairwise comparisons between elements to determine the order in a sorted list. They have a lower bound of Ω(n log n) time complexity due to needing to traverse a decision tree with a minimum of n log n comparisons. Counting sort is a non-comparison sorting algorithm that takes advantage of key assumptions about the data to count and place elements directly into the output array in linear time O(n+k), where n is the number of elements and k is the range of possible key values.

1. Sorting Algorithms

2. Comparison Sorting
 A comparison sort is a type of sorting algorithm that only
reads the list elements through a single comparison operation
(often a "less than or equal to" operator or a three-way
comparison) and determines which of two elements should
occur first in the final sorted list.
 What is common to all these algorithms?
– Make comparisons between input elements like:
ai < aj, ai ≤ aj, ai = aj,
ai ≥ aj, or
ai > aj
2

3. Examples of Comparison Sorting
1. Bubble sort
2. Insertion sort
3. Selection sort
4. Quick sort
5. Heap sort
6. Merge sort
7. Odd-even sort
8. Cocktail sort
9. Cycle sort
10. Merge insertion (Ford-Johnson) sort
11. Smoothsort
12. Timsort
3

4. Limitations of Comparison Sorting
 There are fundamental limits on the performance of
comparison sorts.
 To sort n elements, comparison sorts must make (n log n)
comparisons in the worst case.
 That is a comparison sort must have a lower bound of
Ω(n log n) comparison operations, which is known as linear or
linearithmic time.
 This is a consequence of the limited information available
through comparisons alone
4

5. Why does it takes so much of time?
 A decision tree is used to represent the comparisons of a
sorting algorithm. Assume that all inputs are distinct. A
decision tree compares all possible inputs to each other to
determine the sequence of outputs.
 Decision Tree for three numbers a1, a2, a3 : If at the root, a1 ≤
a2 gο left and compare a2 to a3, otherwise go to right and
compare a1 to a3. Each path represents a different ordering of
a1, a2, a3.

6. Why does it takes so much of time?
 This type of decision tree will have n! leaves.
 Any comparison based sorting algorithm will have to go
through the steps in the decision tree as a minimum

7. Why does it takes so much of time?
 So for n inputs, the tree must have n! leaves. A binary tree of
height h has no more than 2h leaves.
 Now n ! <= 2h
 Taking log on both side: lg(n !) <= h
 According to stirling’s approximation we have:
 So it can be written h= Ω(n log n). This means we need to do at
least n log n comparisons to reach the bottom of the tree

8. How Fast Can We Sort?
• Selection Sort, Bubble Sort, Insertion Sort, Cocktail
sort, Cycle sort: O(n2)
• Smooth sort, Odd-even sort, Timsort: O(n)
• Heap Sort, Merge sort: O(n log n)
• Quicksort:
– Average: O(n log n)
– Best: O(n log n) (simple partition) or O(n) (three-way
partition and equal keys)
8

9. Non Comparison Sorting
 There are some sorting algorithms that perform sorting without
comparing the elements rather by making certain assumptions
about the data. They are called the non comparison sorting.
 Non comparison sorting include:
1. Counting sort (indexes using key values)
2. Radix sort (examines individual bits of keys)
3. Bucket sort (examines bits of keys)
 These are Linear sorting algorithms. Linear sorts are NOT
“comparison sorts”.
 They make certain assumptions about the data. These type of
sorting algorithm does not need to go through the comparison
decision tree.
9

10. Counting sort
 Counting sort assumes that each of the n input elements is an
integer in the range 0 to k. that is n is the number of elements and
k is the highest value element.
 Consider the input set : 4, 1, 3, 4, 3. Then n=5 and k=4
 Counting sort determines for each input element x, the number of
elements less than x. And it uses this information to place
element x directly into its position in the output array. For
example if there exits 17 elements less that x then x is placed into
the 18th position into the output array.
 The algorithm uses three array:
Input Array: A[1..n] store input data where A[j] {1, 2, 3, …, k}
Output Array: B[1..n] finally store the sorted data
Temporary Array: C[1..k] store data temporarily

11. Counting Sort
1. Counting-Sort(A, B, k)
2. Let C[0…..k] be a new array
3. for i=0 to k
4.
C[i]= 0;
5. for j=1 to A.length or n
6.
C[ A[j] ] = C[ A[j] ] + 1;
7. for i=1 to k
8.
C[i] = C[i] + C[i-1];
9. for j=n or A.length down to 1
10.
B[ C[ A[j] ] ] = A[j];
11.
C[ A[j] ] = C[ A[j] ] - 1;

12. Counting Sort
1. Counting-Sort(A, B, k)
2. Let C[0…..k] be a new array
3. for i=0 to k
4.
C[i]= 0;
5. for j=1 to A.length or n
6.
C[ A[j] ] = C[ A[j] ] + 1;
7. for i=1 to k
8.
C[i] = C[i] + C[i-1];
9. for j=n or A.length down to 1
10.
B[ C[ A[j] ] ] = A[j];
11.
C[ A[j] ] = C[ A[j] ] - 1;
[Loop 1]
[Loop 2]
[Loop 3]
[Loop 4]

13. Counting-sort example
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
C:
1
B:
2
3
4
5
6
7
8

14. Executing Loop 1
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
0
0
0
0
0
C: 0
1
B:
2
3
4
5
6
7
8

15. Executing Loop 2
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
0
1
0
0
0
C: 0
1
B:
2
3
4
5
6
7
8

16. Executing Loop 2
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
0
1
0
0
1
C: 0
1
B:
2
3
4
5
6
7
8

17. Executing Loop 2
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
0
1
1
0
1
C: 0
1
B:
2
3
4
5
6
7
8

18. Executing Loop 2
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
0
1
1
0
1
C: 1
1
B:
2
3
4
5
6
7
8

19. Executing Loop 2
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
0
2
1
0
1
C: 1
1
B:
2
3
4
5
6
7
8

20. Executing Loop 2
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
0
2
2
0
1
C: 1
1
B:
2
3
4
5
6
7
8

21. Executing Loop 2
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
0
2
2
0
1
C: 2
1
B:
2
3
4
5
6
7
8

22. Executing Loop 2
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
0
2
3
0
1
C: 2
1
B:
2
3
4
5
6
7
8

23. End of Loop 2
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
0
2
3
0
1
C: 2
1
B:
2
3
4
5
6
7
8

24. Executing Loop 3
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
0
2
3
0
1
C: 2
0
C: 2
1
B:
1
2
3
4
5
2
2
3
0
1
2
3
4
5
6
7
8

25. Executing Loop 3
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
3
0
1
2
C: 2
2
0
1
2
3
4
5
C: 2
2
4
3
0
1
1
B:
2
3
4
5
6
7
8

26. Executing Loop 3
1
2
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
1
2
3
4
5
C: 2
2
4
3
0
1
0
1
2
3
4
5
2
4
7
0
1
A:
C: 2
1
B:
2
3
4
5
6
7
8

27. Executing Loop 3
1
2
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
1
2
3
4
5
C: 2
2
4
7
0
1
0
1
2
3
4
5
2
4
7
A:
C: 2
1
B:
2
3
4
1
7
5
6
7
8

28. Executing Loop 3
1
2
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
1
2
3
4
5
C: 2
2
4
7
7
1
0
1
2
3
4
5
2
4
7
7
A:
C: 2
1
B:
2
3
4
5
8
6
7
8

29. End of Loop 3
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
2
4
7
7
8
C: 2
1
B:
2
3
4
5
6
7
8

30. Executing Loop 4
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
2
4
7
7
8
C: 2
1
B:
2
3
4
5
6
7
8

31. Executing Loop 4
1
A:
2
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
C: 2
1
B:
1
2
3
4
5
2
4
7
7
8
2
3
4
5
6
7
J=8, then A[ j ]=A[8]=3
And B[ C[ A[j] ] ]
=B[ C[ 3 ] ]
=B[ 7]
So B[ C[ A[j] ] ] ←A[ j ]
=B[7]←3
8

32. Executing Loop 4
1
A:
2
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
C:
1
2
3
4
5
2
2
4
6
7
8
1
B:
2
3
4
5
6
7
3
J=8, then A[ j ]=A[8]=3
Then C[ A[j] ]
= C[ 3 ]
=7
So C[ A[j] ] = C[ A[j] ] -1
=7-1=6
8

33. Executing Loop 4
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
2
4
6
7
8
C: 1
1
B:
2
0
3
4
5
6
7
3
8

34. Executing Loop 4
1
C:
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
1
2
4
5
7
8
1
B:
2
0
3
4
5
6
7
3
3
8

35. Executing Loop 4
1
C:
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
1
2
3
5
7
8
1
B:
2
0
3
4
2
5
6
7
3
3
8

36. Executing Loop 4
1
C:
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
0
2
3
5
7
8
1
B: 0
2
0
3
4
2
5
6
7
3
3
8

37. Executing Loop 4
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
2
3
4
7
8
C: 0
1
B: 0
2
0
3
4
5
6
7
2
3
3
3
8

38. Executing Loop 4
1
3
4
5
6
7
8
2
5
3
0
2
3
0
3
0
A:
2
1
2
3
4
5
2
3
4
7
7
C: 0
1
B: 0
2
0
3
4
5
6
7
8
2
3
3
3
5

39. End of Loop 4
1
2
3
4
5
6
7
8
A: 2
5
3
0
2
3
0
3
1
2
3
4
5
2
2
4
7
7
0
C: 0
1
B: 0
2
3
4
5
6
7
8
0
2
2
3
3
3
5
Sorted data in Array B

40. Time Complexity Analysis
1. Counting-Sort(A, B, k)
2. Let C[0…..k] be a new array
3. for i=0 to k
4.
C[i]= 0;
5. for j=1 to A.length or n
6.
C[ A[j] ] = C[ A[j] ] + 1;
7. for i=1 to k
8.
C[i] = C[i] + C[i-1];
9. for j=n or A.length down to 1
10.
B[ C[ A[j] ] ] = A[j];
11.
C[ A[j] ] = C[ A[j] ] - 1;
[Loop 1]
Loop 1 and 3
takes O(k) time
[Loop 2]
[Loop 3]
[Loop 4]
Loop 2 and 4
takes O(n) time

41. Time Complexity Analysis
• So the counting sort takes a total time of: O(n + k)
• Counting sort is called stable sort.
– A sorting algorithm is stable when numbers with
the same values appear in the output array in the
same order as they do in the input array.

42. Counting Sort Review
• Why don’t we always use counting sort?
– Depends on range k of elements.
• Could we use counting sort to sort 32 bit integers? Why or
why not?

43. Counting Sort Review
• Assumption: input taken from small set of numbers of size k
• Basic idea:
– Count number of elements less than you for each element.
– This gives the position of that number – similar to selection
sort.
• Pro’s:
– Fast
– Asymptotically fast - O(n+k)
– Simple to code
• Con’s:
– Doesn’t sort in place.
– Requires O(n+k) extra storage.

44. MD. Shakhawat Hossain
Student of Computer Science & Engineering Dept.
University of Rajshahi