Although, we have been analyzing sorting based in comparing numbers there are ways to improve the lower bound of Ω(n log n).

1. Analysis of Algorithms
Sorting in linear time
Andres Mendez-Vazquez
September 23, 2015
1 / 34

2. Outline
1 Introduction
2 Counting Sort
Counting Sort: Definitions
Complexity
3 Least Significant Digit Radix Sort
Radix Sort: Definitions
Representation
Implementation Using Queues
Complexity
4 Bucket Sort
Introduction
Algorithm
Example
Complexity Analysis
5 Exercises
2 / 34

3. About Sorting
Until now we had the following constraint
Given two numbers x and y, we compare to know if
x < y
x > y
x = y



The Trichotomy Law (1)
What if we do something different?
How to avoid to compare?
Maybe, we can do the following
1 You can try to know the position of the numbers in someway !!!
2 You can try to use the relative position of the building blocks of the
numbers using certain numeric system !!!
3 You can use the idea of histograms !!!
3 / 34

4. About Sorting
Until now we had the following constraint
Given two numbers x and y, we compare to know if
x < y
x > y
x = y



The Trichotomy Law (1)
What if we do something different?
How to avoid to compare?
Maybe, we can do the following
1 You can try to know the position of the numbers in someway !!!
2 You can try to use the relative position of the building blocks of the
numbers using certain numeric system !!!
3 You can use the idea of histograms !!!
3 / 34

5. About Sorting
Until now we had the following constraint
Given two numbers x and y, we compare to know if
x < y
x > y
x = y



The Trichotomy Law (1)
What if we do something different?
How to avoid to compare?
Maybe, we can do the following
1 You can try to know the position of the numbers in someway !!!
2 You can try to use the relative position of the building blocks of the
numbers using certain numeric system !!!
3 You can use the idea of histograms !!!
3 / 34

6. About Sorting
Until now we had the following constraint
Given two numbers x and y, we compare to know if
x < y
x > y
x = y



The Trichotomy Law (1)
What if we do something different?
How to avoid to compare?
Maybe, we can do the following
1 You can try to know the position of the numbers in someway !!!
2 You can try to use the relative position of the building blocks of the
numbers using certain numeric system !!!
3 You can use the idea of histograms !!!
3 / 34

7. About Sorting
Until now we had the following constraint
Given two numbers x and y, we compare to know if
x < y
x > y
x = y



The Trichotomy Law (1)
What if we do something different?
How to avoid to compare?
Maybe, we can do the following
1 You can try to know the position of the numbers in someway !!!
2 You can try to use the relative position of the building blocks of the
numbers using certain numeric system !!!
3 You can use the idea of histograms !!!
3 / 34

8. About Sorting
Until now we had the following constraint
Given two numbers x and y, we compare to know if
x < y
x > y
x = y



The Trichotomy Law (1)
What if we do something different?
How to avoid to compare?
Maybe, we can do the following
1 You can try to know the position of the numbers in someway !!!
2 You can try to use the relative position of the building blocks of the
numbers using certain numeric system !!!
3 You can use the idea of histograms !!!
3 / 34

9. Outline
1 Introduction
2 Counting Sort
Counting Sort: Definitions
Complexity
3 Least Significant Digit Radix Sort
Radix Sort: Definitions
Representation
Implementation Using Queues
Complexity
4 Bucket Sort
Introduction
Algorithm
Example
Complexity Analysis
5 Exercises
4 / 34

10. Counting Sort: Definitions
Counting sort features
Counting sort is good for sorting integers in a narrow range.
It assumes the input numbers (keys) are in the range 0, ..., k.
It uses an auxiliary array C[0, ..., k] to hold the number of items less
than i for 0 ≤ i ≤ k.
Complexity O(n + k), if k = O(n), then the running time isΘ(n).
Counting sort is stable; it keeps records in their original order.
5 / 34

11. Counting Sort: Definitions
Counting sort features
Counting sort is good for sorting integers in a narrow range.
It assumes the input numbers (keys) are in the range 0, ..., k.
It uses an auxiliary array C[0, ..., k] to hold the number of items less
than i for 0 ≤ i ≤ k.
Complexity O(n + k), if k = O(n), then the running time isΘ(n).
Counting sort is stable; it keeps records in their original order.
5 / 34

12. Counting Sort: Definitions
Counting sort features
Counting sort is good for sorting integers in a narrow range.
It assumes the input numbers (keys) are in the range 0, ..., k.
It uses an auxiliary array C[0, ..., k] to hold the number of items less
than i for 0 ≤ i ≤ k.
Complexity O(n + k), if k = O(n), then the running time isΘ(n).
Counting sort is stable; it keeps records in their original order.
5 / 34

13. Counting Sort: Definitions
Counting sort features
Counting sort is good for sorting integers in a narrow range.
It assumes the input numbers (keys) are in the range 0, ..., k.
It uses an auxiliary array C[0, ..., k] to hold the number of items less
than i for 0 ≤ i ≤ k.
Complexity O(n + k), if k = O(n), then the running time isΘ(n).
Counting sort is stable; it keeps records in their original order.
5 / 34

14. Counting Sort: Definitions
Counting sort features
Counting sort is good for sorting integers in a narrow range.
It assumes the input numbers (keys) are in the range 0, ..., k.
It uses an auxiliary array C[0, ..., k] to hold the number of items less
than i for 0 ≤ i ≤ k.
Complexity O(n + k), if k = O(n), then the running time isΘ(n).
Counting sort is stable; it keeps records in their original order.
5 / 34

15. Counting Sort: Definitions
Counting sort algorithm - Assume indexing starting at 1
Counting-Sort(A, B, k)
1 let C [0..k] be a new array
2 for i = 0 to k
3 C [i] = 0
4 for j = 1 to A.length
5 C [A [j]] = C [A [j]] + 1
6 //C [i] contains the number of elements equal to i
7 for i = 1 to k
8 C [i] = C [i] + C [i − 1]
9 // C [i] contains the number of elements≤ i
10 for j = A.length to 1
11 B [C [A [j]]] = A [j]
12 C [A [j]] = C [A [j]] − 1
6 / 34

16. Counting Sort: Definitions
Counting sort algorithm - Assume indexing starting at 1
Counting-Sort(A, B, k)
1 let C [0..k] be a new array
2 for i = 0 to k
3 C [i] = 0
4 for j = 1 to A.length
5 C [A [j]] = C [A [j]] + 1
6 //C [i] contains the number of elements equal to i
7 for i = 1 to k
8 C [i] = C [i] + C [i − 1]
9 // C [i] contains the number of elements≤ i
10 for j = A.length to 1
11 B [C [A [j]]] = A [j]
12 C [A [j]] = C [A [j]] − 1
6 / 34

17. Counting Sort: Definitions
Counting sort algorithm - Assume indexing starting at 1
Counting-Sort(A, B, k)
1 let C [0..k] be a new array
2 for i = 0 to k
3 C [i] = 0
4 for j = 1 to A.length
5 C [A [j]] = C [A [j]] + 1
6 //C [i] contains the number of elements equal to i
7 for i = 1 to k
8 C [i] = C [i] + C [i − 1]
9 // C [i] contains the number of elements≤ i
10 for j = A.length to 1
11 B [C [A [j]]] = A [j]
12 C [A [j]] = C [A [j]] − 1
6 / 34

18. Counting Sort: Definitions
Counting sort algorithm - Assume indexing starting at 1
Counting-Sort(A, B, k)
1 let C [0..k] be a new array
2 for i = 0 to k
3 C [i] = 0
4 for j = 1 to A.length
5 C [A [j]] = C [A [j]] + 1
6 //C [i] contains the number of elements equal to i
7 for i = 1 to k
8 C [i] = C [i] + C [i − 1]
9 // C [i] contains the number of elements≤ i
10 for j = A.length to 1
11 B [C [A [j]]] = A [j]
12 C [A [j]] = C [A [j]] − 1
6 / 34

19. Counting Sort: Definitions
Counting sort algorithm - Assume indexing starting at 1
Counting-Sort(A, B, k)
1 let C [0..k] be a new array
2 for i = 0 to k
3 C [i] = 0
4 for j = 1 to A.length
5 C [A [j]] = C [A [j]] + 1
6 //C [i] contains the number of elements equal to i
7 for i = 1 to k
8 C [i] = C [i] + C [i − 1]
9 // C [i] contains the number of elements≤ i
10 for j = A.length to 1
11 B [C [A [j]]] = A [j]
12 C [A [j]] = C [A [j]] − 1
6 / 34

20. Counting Sort: Definitions
Counting sort algorithm - Assume indexing starting at 1
Counting-Sort(A, B, k)
1 let C [0..k] be a new array
2 for i = 0 to k
3 C [i] = 0
4 for j = 1 to A.length
5 C [A [j]] = C [A [j]] + 1
6 //C [i] contains the number of elements equal to i
7 for i = 1 to k
8 C [i] = C [i] + C [i − 1]
9 // C [i] contains the number of elements≤ i
10 for j = A.length to 1
11 B [C [A [j]]] = A [j]
12 C [A [j]] = C [A [j]] − 1
6 / 34

21. Counting Sort: Definitions
Counting sort algorithm - Assume indexing starting at 1
Counting-Sort(A, B, k)
1 let C [0..k] be a new array
2 for i = 0 to k
3 C [i] = 0
4 for j = 1 to A.length
5 C [A [j]] = C [A [j]] + 1
6 //C [i] contains the number of elements equal to i
7 for i = 1 to k
8 C [i] = C [i] + C [i − 1]
9 // C [i] contains the number of elements≤ i
10 for j = A.length to 1
11 B [C [A [j]]] = A [j]
12 C [A [j]] = C [A [j]] − 1
6 / 34

22. Counting Sort: Definitions
Counting sort algorithm - Assume indexing starting at 1
Counting-Sort(A, B, k)
1 let C [0..k] be a new array
2 for i = 0 to k
3 C [i] = 0
4 for j = 1 to A.length
5 C [A [j]] = C [A [j]] + 1
6 //C [i] contains the number of elements equal to i
7 for i = 1 to k
8 C [i] = C [i] + C [i − 1]
9 // C [i] contains the number of elements≤ i
10 for j = A.length to 1
11 B [C [A [j]]] = A [j]
12 C [A [j]] = C [A [j]] − 1
6 / 34

23. Counting Sort: Definitions
Counting sort algorithm - Assume indexing starting at 1
Counting-Sort(A, B, k)
1 let C [0..k] be a new array
2 for i = 0 to k
3 C [i] = 0
4 for j = 1 to A.length
5 C [A [j]] = C [A [j]] + 1
6 //C [i] contains the number of elements equal to i
7 for i = 1 to k
8 C [i] = C [i] + C [i − 1]
9 // C [i] contains the number of elements≤ i
10 for j = A.length to 1
11 B [C [A [j]]] = A [j]
12 C [A [j]] = C [A [j]] − 1
6 / 34

24. Counting Sort: Definitions
Counting sort algorithm - Assume indexing starting at 1
Counting-Sort(A, B, k)
1 let C [0..k] be a new array
2 for i = 0 to k
3 C [i] = 0
4 for j = 1 to A.length
5 C [A [j]] = C [A [j]] + 1
6 //C [i] contains the number of elements equal to i
7 for i = 1 to k
8 C [i] = C [i] + C [i − 1]
9 // C [i] contains the number of elements≤ i
10 for j = A.length to 1
11 B [C [A [j]]] = A [j]
12 C [A [j]] = C [A [j]] − 1
6 / 34

25. Counting Sort: Definitions
Counting sort algorithm - Assume indexing starting at 1
Counting-Sort(A, B, k)
1 let C [0..k] be a new array
2 for i = 0 to k
3 C [i] = 0
4 for j = 1 to A.length
5 C [A [j]] = C [A [j]] + 1
6 //C [i] contains the number of elements equal to i
7 for i = 1 to k
8 C [i] = C [i] + C [i − 1]
9 // C [i] contains the number of elements≤ i
10 for j = A.length to 1
11 B [C [A [j]]] = A [j]
12 C [A [j]] = C [A [j]] − 1
6 / 34

26. Counting Sort: Definitions
Counting sort algorithm - Assume indexing starting at 1
Counting-Sort(A, B, k)
1 let C [0..k] be a new array
2 for i = 0 to k
3 C [i] = 0
4 for j = 1 to A.length
5 C [A [j]] = C [A [j]] + 1
6 //C [i] contains the number of elements equal to i
7 for i = 1 to k
8 C [i] = C [i] + C [i − 1]
9 // C [i] contains the number of elements≤ i
10 for j = A.length to 1
11 B [C [A [j]]] = A [j]
12 C [A [j]] = C [A [j]] − 1
6 / 34

27. Outline
1 Introduction
2 Counting Sort
Counting Sort: Definitions
Complexity
3 Least Significant Digit Radix Sort
Radix Sort: Definitions
Representation
Implementation Using Queues
Complexity
4 Bucket Sort
Introduction
Algorithm
Example
Complexity Analysis
5 Exercises
7 / 34

28. Final Complexity
We have that
Complexity O(n + k), if k = O(n), then the running time isΘ(n).
8 / 34

29. Remarks
You can use it for
It is efficient if the range of input data is not significantly greater than the
number of objects to be sorted.
It is STABLE
A sorting algorithm is stable if whenever there are two records R and S
with the same key and with R appearing before S in the original list, R
will appear before S in the sorted list.
This why
It is often used as a sub-routine to another sorting algorithm like radix sort.
9 / 34

30. Remarks
You can use it for
It is efficient if the range of input data is not significantly greater than the
number of objects to be sorted.
It is STABLE
A sorting algorithm is stable if whenever there are two records R and S
with the same key and with R appearing before S in the original list, R
will appear before S in the sorted list.
This why
It is often used as a sub-routine to another sorting algorithm like radix sort.
9 / 34

31. Remarks
You can use it for
It is efficient if the range of input data is not significantly greater than the
number of objects to be sorted.
It is STABLE
A sorting algorithm is stable if whenever there are two records R and S
with the same key and with R appearing before S in the original list, R
will appear before S in the sorted list.
This why
It is often used as a sub-routine to another sorting algorithm like radix sort.
9 / 34

32. Outline
1 Introduction
2 Counting Sort
Counting Sort: Definitions
Complexity
3 Least Significant Digit Radix Sort
Radix Sort: Definitions
Representation
Implementation Using Queues
Complexity
4 Bucket Sort
Introduction
Algorithm
Example
Complexity Analysis
5 Exercises
10 / 34

33. Radix Sort: Definitions
Radix sort interesting facts!
Radix sort is how IBM made its money, using punch card readers for
census tabulation in early 1900’s.
A radix sorting algorithm was originally used to sort punched cards in
several passes.
How?
It sorts each digit (or field/column) separately.Example:
3 4 5 1
1 2 2 4
7 8 9 1
1 2 2 5
=⇒
1 2 2 4
1 2 2 5
3 4 5 1
7 8 9 1
It starts with the least-significant digit
Radix sort must use a stable sort.
11 / 34

34. Radix Sort: Definitions
Radix sort interesting facts!
Radix sort is how IBM made its money, using punch card readers for
census tabulation in early 1900’s.
A radix sorting algorithm was originally used to sort punched cards in
several passes.
How?
It sorts each digit (or field/column) separately.Example:
3 4 5 1
1 2 2 4
7 8 9 1
1 2 2 5
=⇒
1 2 2 4
1 2 2 5
3 4 5 1
7 8 9 1
It starts with the least-significant digit
Radix sort must use a stable sort.
11 / 34

35. Radix Sort: Definitions
Radix sort interesting facts!
Radix sort is how IBM made its money, using punch card readers for
census tabulation in early 1900’s.
A radix sorting algorithm was originally used to sort punched cards in
several passes.
How?
It sorts each digit (or field/column) separately.Example:
3 4 5 1
1 2 2 4
7 8 9 1
1 2 2 5
=⇒
1 2 2 4
1 2 2 5
3 4 5 1
7 8 9 1
It starts with the least-significant digit
Radix sort must use a stable sort.
11 / 34

36. Radix Sort: Definitions
Radix sort interesting facts!
Radix sort is how IBM made its money, using punch card readers for
census tabulation in early 1900’s.
A radix sorting algorithm was originally used to sort punched cards in
several passes.
How?
It sorts each digit (or field/column) separately.Example:
3 4 5 1
1 2 2 4
7 8 9 1
1 2 2 5
=⇒
1 2 2 4
1 2 2 5
3 4 5 1
7 8 9 1
It starts with the least-significant digit
Radix sort must use a stable sort.
11 / 34

37. Radix Sort: Definitions
Radix sort interesting facts!
Radix sort is how IBM made its money, using punch card readers for
census tabulation in early 1900’s.
A radix sorting algorithm was originally used to sort punched cards in
several passes.
How?
It sorts each digit (or field/column) separately.Example:
3 4 5 1
1 2 2 4
7 8 9 1
1 2 2 5
=⇒
1 2 2 4
1 2 2 5
3 4 5 1
7 8 9 1
It starts with the least-significant digit
Radix sort must use a stable sort.
11 / 34

38. Outline
1 Introduction
2 Counting Sort
Counting Sort: Definitions
Complexity
3 Least Significant Digit Radix Sort
Radix Sort: Definitions
Representation
Implementation Using Queues
Complexity
4 Bucket Sort
Introduction
Algorithm
Example
Complexity Analysis
5 Exercises
12 / 34

39. It is based in the following idea
First
Every number can be represented in each base. For example:
1024 = 1 × 103
+ 0 × 102
+ 2 × 101
+ 4 × 100
(2)
Thus in general, given a radix b
x = xdbd
+ xd−1bd−1
+ ... + x0b0
(3)
It can be proved inductively
We can sort by using Least-Significative to Most-significative Order and
we keep an stable sort using this order
13 / 34

40. It is based in the following idea
First
Every number can be represented in each base. For example:
1024 = 1 × 103
+ 0 × 102
+ 2 × 101
+ 4 × 100
(2)
Thus in general, given a radix b
x = xdbd
+ xd−1bd−1
+ ... + x0b0
(3)
It can be proved inductively
We can sort by using Least-Significative to Most-significative Order and
we keep an stable sort using this order
13 / 34

41. It is based in the following idea
First
Every number can be represented in each base. For example:
1024 = 1 × 103
+ 0 × 102
+ 2 × 101
+ 4 × 100
(2)
Thus in general, given a radix b
x = xdbd
+ xd−1bd−1
+ ... + x0b0
(3)
It can be proved inductively
We can sort by using Least-Significative to Most-significative Order and
we keep an stable sort using this order
13 / 34

42. However
Remark 1
A most significant digit (MSD) radix sort can be used to sort keys in
lexicographic order.
Remark 2
Unlike a least significant digit (LSD) radix sort, a most significant digit
radix sort does not necessarily preserve the original order of duplicate keys.
14 / 34

43. However
Remark 1
A most significant digit (MSD) radix sort can be used to sort keys in
lexicographic order.
Remark 2
Unlike a least significant digit (LSD) radix sort, a most significant digit
radix sort does not necessarily preserve the original order of duplicate keys.
14 / 34

44. Example
Example
3 2 9
4 5 7
6 5 7
8 3 9
4 3 6
7 2 0
3 5 5
=⇒
7 2 0
3 5 5
4 3 6
4 5 7
6 5 7
3 2 9
8 3 9
=⇒
7 2 0
3 2 9
4 3 6
8 3 9
3 5 5
4 5 7
6 5 7
=⇒
3 2 9
3 5 5
4 3 6
4 5 7
6 5 7
7 2 0
8 3 9
15 / 34

45. Example
Example
3 2 9
4 5 7
6 5 7
8 3 9
4 3 6
7 2 0
3 5 5
=⇒
7 2 0
3 5 5
4 3 6
4 5 7
6 5 7
3 2 9
8 3 9
=⇒
7 2 0
3 2 9
4 3 6
8 3 9
3 5 5
4 5 7
6 5 7
=⇒
3 2 9
3 5 5
4 3 6
4 5 7
6 5 7
7 2 0
8 3 9
15 / 34

46. Example
Example
3 2 9
4 5 7
6 5 7
8 3 9
4 3 6
7 2 0
3 5 5
=⇒
7 2 0
3 5 5
4 3 6
4 5 7
6 5 7
3 2 9
8 3 9
=⇒
7 2 0
3 2 9
4 3 6
8 3 9
3 5 5
4 5 7
6 5 7
=⇒
3 2 9
3 5 5
4 3 6
4 5 7
6 5 7
7 2 0
8 3 9
15 / 34

47. Example
Example
3 2 9
4 5 7
6 5 7
8 3 9
4 3 6
7 2 0
3 5 5
=⇒
7 2 0
3 5 5
4 3 6
4 5 7
6 5 7
3 2 9
8 3 9
=⇒
7 2 0
3 2 9
4 3 6
8 3 9
3 5 5
4 5 7
6 5 7
=⇒
3 2 9
3 5 5
4 3 6
4 5 7
6 5 7
7 2 0
8 3 9
15 / 34

48. Radix Sort: Algorithm Using the Least Significative Digit
Algorithm
Radix-Sort(A, d)
1 for i = 1 to d
2 use a stable sort to sort array A on digit i
16 / 34

49. Outline
1 Introduction
2 Counting Sort
Counting Sort: Definitions
Complexity
3 Least Significant Digit Radix Sort
Radix Sort: Definitions
Representation
Implementation Using Queues
Complexity
4 Bucket Sort
Introduction
Algorithm
Example
Complexity Analysis
5 Exercises
17 / 34

50. Iterative version using queues
How? First, assume Radix = 10
The integers are enqueued into an array of ten separate queues based on
their digits from right to left.
Example: 170, 045, 075, 090, 002, 024, 802, 066
0 : 170, 190
1 : none
2 : 002, 802
3 : none
4 : 024
5 : 045, 075
6 : 066
7 : none
8 : none
9 : none
18 / 34

51. Iterative version using queues
How? First, assume Radix = 10
The integers are enqueued into an array of ten separate queues based on
their digits from right to left.
Example: 170, 045, 075, 090, 002, 024, 802, 066
0 : 170, 190
1 : none
2 : 002, 802
3 : none
4 : 024
5 : 045, 075
6 : 066
7 : none
8 : none
9 : none
18 / 34

52. Iterative version using queues
Then, the queues are dequeued back into an array of integers, in
increasing order
170, 090, 002, 802, 024, 045, 075, 066
Then
You repeat again!!!
19 / 34

53. Iterative version using queues
Then, the queues are dequeued back into an array of integers, in
increasing order
170, 090, 002, 802, 024, 045, 075, 066
Then
You repeat again!!!
19 / 34

54. Outline
1 Introduction
2 Counting Sort
Counting Sort: Definitions
Complexity
3 Least Significant Digit Radix Sort
Radix Sort: Definitions
Representation
Implementation Using Queues
Complexity
4 Bucket Sort
Introduction
Algorithm
Example
Complexity Analysis
5 Exercises
20 / 34

55. Radix Sort: Proving the Complexity
Lemma 1
Given n d-digit numbers in which each digit can take on up to k possible
values, RADIX-SORT correctly sorts these numbers in Θ(d(n + k)) time.
Proof
Quite simple!!!
21 / 34

56. Radix Sort: Proving the Complexity
Lemma 1
Given n d-digit numbers in which each digit can take on up to k possible
values, RADIX-SORT correctly sorts these numbers in Θ(d(n + k)) time.
Proof
Quite simple!!!
21 / 34

57. Thanks Carlos Alcala Oracle Class 2014 for the Example
Imagine the following
That you have a sequence of n IP Addresses. For example:
n IP addresses



192.168.45.120
192.128.15.120
100.192.168.45
...
92.16.4.120
Thus, Why not use the chunks in the IP to sort the IP addresses
(Think Columns)
n IP addresses



192.168.45.120
192.128.15.120
100.192.168.45
...
92.16.4.120
22 / 34

58. Thanks Carlos Alcala Oracle Class 2014 for the Example
Imagine the following
That you have a sequence of n IP Addresses. For example:
n IP addresses



192.168.45.120
192.128.15.120
100.192.168.45
...
92.16.4.120
Thus, Why not use the chunks in the IP to sort the IP addresses
(Think Columns)
n IP addresses



192.168.45.120
192.128.15.120
100.192.168.45
...
92.16.4.120
22 / 34

59. Yes!!!
Yes!!!
After all each chunk is a number between 0 and 255 i.e between 0 to
28 − 1 ⇒ size chunks is r = 8
We have then for each IP address a size of b = 32 bits
Lemma 2
Given n b-bit numbers and any positive ingeter r ≤ b, RADIX-SORT
correctly sort these numbers in Θ(b
r (n + 2r ))time.
Proof
At the Board...
23 / 34

60. Yes!!!
Yes!!!
After all each chunk is a number between 0 and 255 i.e between 0 to
28 − 1 ⇒ size chunks is r = 8
We have then for each IP address a size of b = 32 bits
Lemma 2
Given n b-bit numbers and any positive ingeter r ≤ b, RADIX-SORT
correctly sort these numbers in Θ(b
r (n + 2r ))time.
Proof
At the Board...
23 / 34

61. Yes!!!
Yes!!!
After all each chunk is a number between 0 and 255 i.e between 0 to
28 − 1 ⇒ size chunks is r = 8
We have then for each IP address a size of b = 32 bits
Lemma 2
Given n b-bit numbers and any positive ingeter r ≤ b, RADIX-SORT
correctly sort these numbers in Θ(b
r (n + 2r ))time.
Proof
At the Board...
23 / 34

62. Yes!!!
Yes!!!
After all each chunk is a number between 0 and 255 i.e between 0 to
28 − 1 ⇒ size chunks is r = 8
We have then for each IP address a size of b = 32 bits
Lemma 2
Given n b-bit numbers and any positive ingeter r ≤ b, RADIX-SORT
correctly sort these numbers in Θ(b
r (n + 2r ))time.
Proof
At the Board...
23 / 34

63. Final Remarks
Final Remarks
LSD radix sorts have resurfaced as an alternative to high performance
comparison-based sorting algorithms (like heapsort and mergesort)
that require O(n log n) comparisons. YES BIG DATA!!!
For more, look at V. J. Duvanenko, "In-Place Hybrid Binary-Radix
Sort", Dr. Dobb’s Journal, 1 October 2009
24 / 34

64. Final Remarks
Final Remarks
LSD radix sorts have resurfaced as an alternative to high performance
comparison-based sorting algorithms (like heapsort and mergesort)
that require O(n log n) comparisons. YES BIG DATA!!!
For more, look at V. J. Duvanenko, "In-Place Hybrid Binary-Radix
Sort", Dr. Dobb’s Journal, 1 October 2009
24 / 34

65. Outline
1 Introduction
2 Counting Sort
Counting Sort: Definitions
Complexity
3 Least Significant Digit Radix Sort
Radix Sort: Definitions
Representation
Implementation Using Queues
Complexity
4 Bucket Sort
Introduction
Algorithm
Example
Complexity Analysis
5 Exercises
25 / 34

66. Bucket Sort: Definitions
Assumption
The keys are in the range [0, 1).
Actually you can have a range [0, N) and divide the Keys by N
You have n of them
Basic idea when you have
1 Create n linked lists (buckets) to divide interval [0, 1) into
subintervals of size 1/n.
2 Add each input element to the appropriate bucket.
3 Concatenate the buckets.
Complexity
Expected total time is O(n), where n is the size of the original sequence.
26 / 34

67. Bucket Sort: Definitions
Assumption
The keys are in the range [0, 1).
Actually you can have a range [0, N) and divide the Keys by N
You have n of them
Basic idea when you have
1 Create n linked lists (buckets) to divide interval [0, 1) into
subintervals of size 1/n.
2 Add each input element to the appropriate bucket.
3 Concatenate the buckets.
Complexity
Expected total time is O(n), where n is the size of the original sequence.
26 / 34

68. Bucket Sort: Definitions
Assumption
The keys are in the range [0, 1).
Actually you can have a range [0, N) and divide the Keys by N
You have n of them
Basic idea when you have
1 Create n linked lists (buckets) to divide interval [0, 1) into
subintervals of size 1/n.
2 Add each input element to the appropriate bucket.
3 Concatenate the buckets.
Complexity
Expected total time is O(n), where n is the size of the original sequence.
26 / 34

69. Bucket Sort: Definitions
Assumption
The keys are in the range [0, 1).
Actually you can have a range [0, N) and divide the Keys by N
You have n of them
Basic idea when you have
1 Create n linked lists (buckets) to divide interval [0, 1) into
subintervals of size 1/n.
2 Add each input element to the appropriate bucket.
3 Concatenate the buckets.
Complexity
Expected total time is O(n), where n is the size of the original sequence.
26 / 34

70. Bucket Sort: Definitions
Assumption
The keys are in the range [0, 1).
Actually you can have a range [0, N) and divide the Keys by N
You have n of them
Basic idea when you have
1 Create n linked lists (buckets) to divide interval [0, 1) into
subintervals of size 1/n.
2 Add each input element to the appropriate bucket.
3 Concatenate the buckets.
Complexity
Expected total time is O(n), where n is the size of the original sequence.
26 / 34

71. Bucket Sort: Definitions
Assumption
The keys are in the range [0, 1).
Actually you can have a range [0, N) and divide the Keys by N
You have n of them
Basic idea when you have
1 Create n linked lists (buckets) to divide interval [0, 1) into
subintervals of size 1/n.
2 Add each input element to the appropriate bucket.
3 Concatenate the buckets.
Complexity
Expected total time is O(n), where n is the size of the original sequence.
26 / 34

72. Bucket Sort: Definitions
Assumption
The keys are in the range [0, 1).
Actually you can have a range [0, N) and divide the Keys by N
You have n of them
Basic idea when you have
1 Create n linked lists (buckets) to divide interval [0, 1) into
subintervals of size 1/n.
2 Add each input element to the appropriate bucket.
3 Concatenate the buckets.
Complexity
Expected total time is O(n), where n is the size of the original sequence.
26 / 34

73. Outline
1 Introduction
2 Counting Sort
Counting Sort: Definitions
Complexity
3 Least Significant Digit Radix Sort
Radix Sort: Definitions
Representation
Implementation Using Queues
Complexity
4 Bucket Sort
Introduction
Algorithm
Example
Complexity Analysis
5 Exercises
27 / 34

74. Bucket Sort Algorithm
Algorithm assuming
Buket-Sort(A)
1 let B [0..n − 1] be a new array
2 n = A.length
3 for i = 0 to n − 1
4 make B [i] an empty list
5 for i = 0 to n
6 insert A [i] into list B [ nA [i] ]
7 for i = 0 to n − 1
8 sort list B [i] with insertion sort
9 concatenate the list B [0], B [1], ... , B [n − 1] together in order
28 / 34

75. Bucket Sort Algorithm
Algorithm assuming
Buket-Sort(A)
1 let B [0..n − 1] be a new array
2 n = A.length
3 for i = 0 to n − 1
4 make B [i] an empty list
5 for i = 0 to n
6 insert A [i] into list B [ nA [i] ]
7 for i = 0 to n − 1
8 sort list B [i] with insertion sort
9 concatenate the list B [0], B [1], ... , B [n − 1] together in order
28 / 34

76. Bucket Sort Algorithm
Algorithm assuming
Buket-Sort(A)
1 let B [0..n − 1] be a new array
2 n = A.length
3 for i = 0 to n − 1
4 make B [i] an empty list
5 for i = 0 to n
6 insert A [i] into list B [ nA [i] ]
7 for i = 0 to n − 1
8 sort list B [i] with insertion sort
9 concatenate the list B [0], B [1], ... , B [n − 1] together in order
28 / 34

77. Bucket Sort Algorithm
Algorithm assuming
Buket-Sort(A)
1 let B [0..n − 1] be a new array
2 n = A.length
3 for i = 0 to n − 1
4 make B [i] an empty list
5 for i = 0 to n
6 insert A [i] into list B [ nA [i] ]
7 for i = 0 to n − 1
8 sort list B [i] with insertion sort
9 concatenate the list B [0], B [1], ... , B [n − 1] together in order
28 / 34

78. Bucket Sort Algorithm
Algorithm assuming
Buket-Sort(A)
1 let B [0..n − 1] be a new array
2 n = A.length
3 for i = 0 to n − 1
4 make B [i] an empty list
5 for i = 0 to n
6 insert A [i] into list B [ nA [i] ]
7 for i = 0 to n − 1
8 sort list B [i] with insertion sort
9 concatenate the list B [0], B [1], ... , B [n − 1] together in order
28 / 34

79. Bucket Sort Algorithm
Algorithm assuming
Buket-Sort(A)
1 let B [0..n − 1] be a new array
2 n = A.length
3 for i = 0 to n − 1
4 make B [i] an empty list
5 for i = 0 to n
6 insert A [i] into list B [ nA [i] ]
7 for i = 0 to n − 1
8 sort list B [i] with insertion sort
9 concatenate the list B [0], B [1], ... , B [n − 1] together in order
28 / 34

80. Bucket Sort Algorithm
Algorithm assuming
Buket-Sort(A)
1 let B [0..n − 1] be a new array
2 n = A.length
3 for i = 0 to n − 1
4 make B [i] an empty list
5 for i = 0 to n
6 insert A [i] into list B [ nA [i] ]
7 for i = 0 to n − 1
8 sort list B [i] with insertion sort
9 concatenate the list B [0], B [1], ... , B [n − 1] together in order
28 / 34

81. Bucket Sort Algorithm
Algorithm assuming
Buket-Sort(A)
1 let B [0..n − 1] be a new array
2 n = A.length
3 for i = 0 to n − 1
4 make B [i] an empty list
5 for i = 0 to n
6 insert A [i] into list B [ nA [i] ]
7 for i = 0 to n − 1
8 sort list B [i] with insertion sort
9 concatenate the list B [0], B [1], ... , B [n − 1] together in order
28 / 34

82. Bucket Sort Algorithm
Algorithm assuming
Buket-Sort(A)
1 let B [0..n − 1] be a new array
2 n = A.length
3 for i = 0 to n − 1
4 make B [i] an empty list
5 for i = 0 to n
6 insert A [i] into list B [ nA [i] ]
7 for i = 0 to n − 1
8 sort list B [i] with insertion sort
9 concatenate the list B [0], B [1], ... , B [n − 1] together in order
28 / 34

83. Outline
1 Introduction
2 Counting Sort
Counting Sort: Definitions
Complexity
3 Least Significant Digit Radix Sort
Radix Sort: Definitions
Representation
Implementation Using Queues
Complexity
4 Bucket Sort
Introduction
Algorithm
Example
Complexity Analysis
5 Exercises
29 / 34

84. Bucket Sort
Example
30 / 34

85. Outline
1 Introduction
2 Counting Sort
Counting Sort: Definitions
Complexity
3 Least Significant Digit Radix Sort
Radix Sort: Definitions
Representation
Implementation Using Queues
Complexity
4 Bucket Sort
Introduction
Algorithm
Example
Complexity Analysis
5 Exercises
31 / 34

86. We have the following
We need to analyze the algoritm
But we have an insertion sort at each bucket!!!
Therefore
Let ni the random variable on the size of the bucket.
We get the following complexity function
T (n) = Θ (n) +
n−1
i=0
O n2
i (4)
Look at the Board!!!
32 / 34

87. We have the following
We need to analyze the algoritm
But we have an insertion sort at each bucket!!!
Therefore
Let ni the random variable on the size of the bucket.
We get the following complexity function
T (n) = Θ (n) +
n−1
i=0
O n2
i (4)
Look at the Board!!!
32 / 34

88. We have the following
We need to analyze the algoritm
But we have an insertion sort at each bucket!!!
Therefore
Let ni the random variable on the size of the bucket.
We get the following complexity function
T (n) = Θ (n) +
n−1
i=0
O n2
i (4)
Look at the Board!!!
32 / 34

89. Final Complexity is
After using the expected value
E (T (n)) = Θ (n)
33 / 34

90. Exercises
From Cormen’s book solve the following
8.1-1
8.1-3
8.2-2
8.2-3
8.3-2
8.3-4
8.4-2
34 / 34