Data analysis and algorithms

1. UNIT 2
UNIT 2
DIVIDE AND CONQUER AND
GREEDY ALGORITHMS

2. CONTENTS
• Divide and Conquer: General Method
• Binary Search
• Finding Maximum and Minimum
• Merge Sort
• Quick Sort
• Greedy Algorithms: General Method
• Single Source Shortest Path Problem
• Container Loading
• Knapsack Problem
• Huffman Codes.

3. Divide and Conquer: General Method
A typical Divide and Conquer algorithm solves a problem using following three steps:
1.Divide: This involves dividing the problem into smaller sub-problems.
2.Conquer: Solve sub-problems by calling recursively until solved.
3.Combine: Combine the sub-problems to get the final solution of the whole problem.

4. Divide and Conquer
Using the Divide and Conquer technique, we divide a problem into subproblems. When the solution
to each subproblem is ready, we 'combine' the results from the subproblems to solve the main
problem.
Suppose we had to sort an array A. A subproblem would be to sort a sub-section of this array starting
at index p and ending at index r, denoted as A[p..r].
Divide
If q is the half-way point between p and r, then we can split the subarray A[p..r] into two
arrays A[p..q] and A[q+1, r].
Conquer
In the conquer step, we try to sort both the subarrays A[p..q] and A[q+1, r]. If we haven't yet reached
the base case, we again divide both these subarrays and try to sort them.
Combine
When the conquer step reaches the base step and we get two sorted subarrays A[p..q] and A[q+1,
r] for array A[p..r], we combine the results by creating a sorted array A[p..r] from two sorted
subarrays A[p..q] and A[q+1, r].

5. Binary Search
• Binary Search is defined as a searching algorithm used in a sorted array by repeatedly
dividing the search interval in half.
• The idea of binary search is to use the information that the array is sorted and reduce the time
complexity to O(log N)

6. Binary Search : Algorithm
• Divide the search space into two halves by finding the middle index “mid”.
• Compare the middle element of the search space with the key.
• If the key is found at middle element, the process is terminated.
• If the key is not found at middle element, choose which half will be used as the next search space.
• If the key is smaller than the middle element, then the left side is used for next search.
• If the key is larger than the middle element, then the right side is used for next search.
• This process is continued until the key is found or the total search space is exhausted.

7. Binary Search : Example
Find the number 23 using binary search
Step 1:
Step 2:
Step 3:

8. Finding Maximum and Minimum : Algorithm
In Divide and Conquer approach:
Step 1: Find the mid of the array.
Step 2: Find the maximum and minimum of the left subarray
recursively.
Step 3: Find the maximum and minimum of the right
subarray recursively.
Step 4: Compare the result of step 3 and step 4
Step 5: Return the minimum and maximum.

9. Merge Sort
ALGORITHM
MERGE_SORT(arr, beg, end)
if beg < end
set mid = (beg + end)/2
MERGE_SORT(arr, beg, mid)
MERGE_SORT(arr, mid + 1, end)
MERGE (arr, beg, mid, end)
end of if
END MERGE_SORT
• It uses the divide and conquer approach to sort the
elements. It is one of the most popular and efficient
sorting algorithm.
• It divides the given list into two equal halves, calls
itself for the two halves and then merges the two
sorted halves.
• We have to define the merge() function to perform
the merging.
• The sub-lists are divided again and again into halves
until the list cannot be divided further.
• Then we combine the pair of one element lists into
two-element lists, sorting them in the process.
• The sorted two-element pairs is merged into the four-
element lists, and so on until we get the sorted list.

10. void merge(int a[], int beg, int mid, int end)
{
int i, j, k;
int n1 = mid - beg + 1;
int n2 = end - mid;
int LeftArray[n1], RightArray[n2]; //temporary arrays
/* copy data to temp arrays */
for (int i = 0; i < n1; i++)
LeftArray[i] = a[beg + i];
for (int j = 0; j < n2; j++)
RightArray[j] = a[mid + 1 + j];
i = 0, /* initial index of first sub-array */
j = 0; /* initial index of second sub-array */
k = beg; /* initial index of merged sub-array */
while (i < n1 && j < n2)
{
if(LeftArray[i] <= RightArray[j])
{
a[k] = LeftArray[i];
i++;
}
Merge Sort : Algorithm
else
{
a[k] = RightArray[j];
j++;
}
k++;
}
while (i<n1)
{
a[k] = LeftArray[i];
i++;
k++;
}
while (j<n2)
{
a[k] = RightArray[j];
j++;
k++;
}
}

11. Merge Sort : Example

12. Quick Sort
• It is an algorithm of Divide & Conquer type.
• Divide:Rearrange the elements and split arrays into two sub-arrays and an
element in between search that each element in left sub array is less than or equal
to the average element and each element in the right sub- array is larger than the
middle element.
• Conquer: Recursively, sort two sub arrays.
• Combine: Combine the already sorted array.

13. Quick Sort : Algorithm
function quick_sort(array):
if array is empty:
return array
else:
pivot = array[-1]
less = [x for x in array if x < pivot]
greater = [x for x in array if x >= pivot]
return quick_sort(less) + [pivot] + quick_sort(greater)

14. Quick Sort : Example

15. Greedy Algorithm : General Method
A greedy algorithm is a strategy that makes the best choice at each stage to eventually lead
to a globally optimal solution. It picks the best solution at the moment without regard for
consequences.
To make a greedy algorithm, you can:
• Identify an optimal substructure or subproblem in the problem
• Determine what the solution will include
• Create an iterative way to go through all of the subproblems and build a solution

16. Huffman Code
• Huffman coding is a lossless data compression algorithm.
• The idea is to assign variable-length codes to input characters, lengths of the assigned
codes are based on the frequencies of corresponding characters.
• The variable-length codes assigned to input characters are Prefix Codes, means the codes
(bit sequences) are assigned in such a way that the code assigned to one character is not the
prefix of code assigned to any other character.
• This is how Huffman Coding makes sure that there is no ambiguity when decoding the
generated bitstream.

17. Huffman Code : Algorithm
There are mainly two major parts in Huffman
Coding
• Build a Huffman Tree from input characters.
• Traverse the Huffman Tree and assign codes
to characters.
Algorithm Huffman (c)
{
n= |c|
Q = c
for i<-1 to n-1
do
{
temp <- get node ()
left (temp] Get_min (Q) right [temp] Get Min (Q)
a = left [templ b = right [temp]
F [temp] = f[a] + [b]
insert (Q, temp)
}
return Get_min (0)
}

18. Huffman Code : Example
Question

19. Huffman Code : Example
Steps to build Huffman Tree
Input is an array of unique characters along with their frequency of occurrences and output is Huffman Tree
Step 2 : c+d
Step 1 : a+b

20. Huffman Code : Example
Step 3 :

21. Huffman Code : Example
Step 4 :

22. Huffman Code : Example
Step 5 :

23. Huffman Code : Example
Steps to print codes from Huffman Tree:
Traverse the tree formed starting from the root. Maintain an auxiliary array. While moving to the left child, write 0 to the
array. While moving to the right child, write 1 to the array. Print the array when a leaf node is encountered