1. Chapter 9
Divide and Conquer
Divide and conquer: the
algorithmic version of recursion
Divide and conquer is closely related to recursion—you could even say that it is
nothing more. It has the same structure as recursion. We often implement divide
and conquer algorithms in terms of recursive functions, but we do not need to. The
defining characteristic of these algorithms is that we reduce a problem to smaller
versions of the same problem
Example: Merge sort
For example, if we want to sort a sequence, then we can split it into two sub-
sequences that we sort and then we can merge the two. The step where we sort
the two sub-sequences is the recursive step. It might feel circular to sort in terms
of sorting, but we are saved from circularity by base cases in the recursion.
2. The merge sort algorithm works like this:
1) Split the input in two
2) Sort the two halves
3) Combine then by merging them
Since we cut the input in half every time we call for a recursive solution, we never
get more than O(log n) function calls deep. Since the number of calls at each level
grows by powers of two as the size of the problems shrink by powers of two, we
end up with a running time of O(n log n).
Notice that this is faster than the other sorting algorithms we have seen. It is
optimal in the sense that no comparison-based sorting algorithms can run faster
than O(n log n).
Examples
Some are just a rephrasing of
algorithms you have already seen…
Divide and conquer is a powerful approach to designing new algorithms, but we
can also rephrase some we have already seen as divide and conquer.
3. DnC formulation of linear search:
1) Check the first element in our sequence and report if it matches
2) Do a linear sort on the rest of the sequence
Selection sort:
1) move the smallest element to the first location in the sequence,
2) sort the rest of the sequence
The "first location in the sequence" is the first location in the sequence we consider
in the recursion. Not the full sequence.
Binary search:
1) check the middle element. Report it found if it is a match.
2) Do a binary search in the left or right half
4. Sum (less likely to lose significant bits):
1) Compute the sum of both halves of the input
2) Add the sums together
Sum (less likely to lose significant bits):
1) Add all pairs of the input
2) Compute the sum of these
Common running times
Honestly, I find it easier just to memorise
these than redo the math each time…
They keep popping up…
There are a few recurrence equations that show up a lot for divide and conquer
running times. There are some general techniques when the recurrence doesn’t
match this but sometimes you just have to work out the math. If you can recognise
one of these recurrences for your algorithm, then just use these equations.
5. We have already seen why T(n) = O(1) + 2T(n/2) is in O(n log n)
It isn’t hard to see that T(n) = T(n - 1) + O(1) must be O(n) either. The argument is
not different from the one we had for the running time of linear search.
For selection sort we can argue that we recurse to depth n and at each level we
perform a search that costs O(n). The running time must then be O(n²) — as we
also argued first time we saw selection sort.
6. Binary search was already a divide and conquer algorithm (although we didn’t
implement it using recursion). We now it can reach call depth O(log n) but it only
does constant time at each level.
It might be less obvious what the running time of T(n) = 2T(n/2) + O(1) is, but we
can argue as follows
1) We cut the problem size in half each time we solve a recursive problem,
therefore we go to call depth log n
2) We double the number of operations we must do at each level
3) You have to know this sum, but it is easy (see next slide)
This sketch doesn’t give us the exact sum, but we can see that if we double the
number we add at each step up to n, then the sum is less than 2n
7. We can also consider summing in the other direct, starting with n. If we half the
number each time we add a new one, then regardless of where we stop, we will
have a sum that is less than 2n.
In this algorithm we spend linear time per level and cut the problem size in two
when going from one level to the next. The sum is the same as before (more or
less) and we have a linear time algorithm.
Exercises!
Time to put divide-and-
conquer into practice
