Divide and Conquer

Divide-and-Conquer
Republic of Yemen
THAMAR UNIVERSITY
Faculty of Computer Science&
Information System
1 Eng: Mohammed Hussein

Divide-and-Conquer

Divide-and-Conquer
 The whole problem we want to solve may too big to understand or solve at
once.
 We break it up into smaller pieces, solve the pieces separately, and
combine the separate pieces together.
 Divide-and conquer is a general algorithm design paradigm:
1. Divide: divide the input data S in two or more disjoint subsets S1,
S2, …
2. Recur: solve the subproblems recursively
3. Conquer: combine the solutions for S1, S2, …, into a solution for S
 The base case for the recursion are subproblems of constant size
 Analysis can be done using recurrence equations

Recurrence Equation Analysis
 The conquer step of merge-sort consists of merging two sorted sequences, each with
n/2 elements and implemented by means of a doubly linked list, takes at most bn steps,
for some constant b.
 Likewise, the basis case (n < 2) will take at b most steps.
 Therefore, if we let T(n) denote the running time of merge-sort:
 We can therefore analyze the running time of merge-sort by finding a closed form
solution to the above equation.
 That is, a solution that has T(n) only on the left-hand side.






2if)2/(2
2if
)(
nbnnT
nb
nT

Solving recurrences
 Recurrences are like solving integrals, differential equations, etc.
 Should learn a few tricks.
 Three ways of solving recurrences
1. Guess-and-Test Method(If we know the answer)
2. RecursionTree Method (Very useful !)
3. MasterTheorem (Save our effort)

Guess-and-Test Method
 In the guess-and-test method, we guess a closed form solution and then try to prove it is
true by induction:
 Guess that:T(n) < c n log n.
 Wrong: we cannot make this last line be less than cn log n
ncn
nbncnncn
nbnncn
nbnnnc
nbnnTnT
log
loglog
log)2log(log
log))2/log()2/((2
log)2/(2)(











2iflog)2/(2
2if
)(
nnbnnT
nb
nT
Log(x/y)= log x – log y
Log 2 =1
‫بالتعويض‬

Guess-and-Test Method, Part 2
 Recall the recurrence equation:
 Guess #2 that:T(n) < cn log2 n.
 if c > b.
 So,T(n) is O(n log2 n).
 In general, to use this method, you need to have a good guess and you need to be good at
induction proofs.
ncn
nbncnncnncn
nbnnncn
nbnncn
nbnnnc
nbnnTnT
2
2
22
2
2
log
loglog2log
log))2(log2loglog2(log
log)2log(log
log))2/(log)2/((2
log)2/(2)(












2iflog)2/(2
2if
)(
nnbnnT
nb
nT
(x-y) 2= x2 -2xy +y2
Log 2 =1
nj <= nd if j <= d
Big-Oh of highest degree

Recurrences
 In Mathematic a recurrence is a function that is defined in
terms of one or more base cases, and itself, with smaller
arguments
 Examples:

Master Method (tree)
 The master theorem concerns the recurrence
 a≥1 , b> 1
 n is the size of the problem.
 a is the number of subproblems in the recursion. (‫)الفزوع‬
 n/b is the size of each subproblem. (Here it is assumed that
all subproblems are essentially the same size.)
 f (n) is the cost of the work done outside the recursive calls
)()/()( nfbnaTnT 
merge-sort ‫خوارسمية‬
‫االستدعاء‬ ‫عمليات‬ ‫عند‬‫الذاتي‬‫الدمج‬ ‫عملية‬ ‫تنفيذ‬ ‫مستويات‬ ‫مختلف‬ ‫في‬

The Tree Definition
 The root of this tree is node A.
 Definitions:
 Parent (A)
 Children (E, F)
 Siblings (C, D)
 Root (A)
 Leaf / Leaves
 K, L, F, G, M, I, J…
 The degree of a node is the number of sub-trees of the node.
 The level of a node:
 Initially letting the root be at level one
 the level of the node’s parent plus one. Level H=level(Parent (H)+1));
 The height or depth of a tree is the maximum level of any node in the tree.
 height =Max(level(tree))

Divide and conquer (tree)
 We analyze this in some generality:
 suppose we have a pieces, each of size n/b and merging takes time f(n).
 (In this example a=b=2 and f(n)=O(log n) but it will not always be true that
a=b sometimes the pieces will overlap.)
 For simplicity, let's assume n is a power of b, and that the recursion stops
when n is 1.
 Notice that the size of a node depends only on its level: size(i) = n/(bi).
 What is time taken by a node at level i? : time(i) = f(n/ bi)
 How many levels can we have before we get down to n=1?
 For bottom level, n/bi =1, so n=bi and i=(log n)/(log b).
 How many items at level i?: ai.
)()/()( nfbnaTnT 
time(i) = f(n/ bi)

The Recursion Tree
 Draw the recursion tree for the recurrence relation and look for a pattern:
 At each successive level of recursion the sub problems get halved in size (n/2).
 At the (log n) th level the sub problems get down to size 1, and so the recursion ends.
Level/
depth
T’s size
0 1 n
1 2 n/2
k 2k n/2k
… … …






2if)2/(2
2if
)(
nbnnT
nb
nT
time
bn
bn
bn
…
Total time = bn + bn log n
(last level plus all previous levels)
2k =n  k= log n for each level.
log n
Notice that the size of a node depends only on its level
time=b x size = bn

Steps using recursion tree.
 Height = k=log n
 Levels = log n + 1
 Cost/level = cn
 Bottom node number n = 2log n for
height log n dividing problem by 2
each time.
 Recall that a balanced binary tree of
height k has k + 1 levels.
 Recall: a
log an = n
 so 2
log 2n = 2
log n =n
 Bottom cost = T(1) * n = cn
 Note that T(1) is problem size n=1,
there are n subproblems at bottom.
13 Eng: Mohammed Hussein 2k =n  k= log n for each level.

Merge Sort - Finding O
 Recall that a balanced binary tree of height k has k + 1 levels.
 A problem of size n=16 divided as described by the following recurrence equation
would then be represented by the tree.
 Use the implied coefficient c for arithmetic purposes:
16=2
4

Master Method
 Many divide-and-conquer recurrence equations have the form:
 The MasterTheorem:






dnnfbnaT
dnc
nT
if)()/(
if
)(
.1somefor)()/(provided
)),((is)(then),(is)(if3.
)log(is)(then),log(is)(if2.
)(is)(then),(is)(if1.
log
1loglog
loglog










nfbnaf
nfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
very small f(n)
moderate f(n)
very large f(n)

Analysis of Merge Sort
Analysis of Merge Sort
 Divide:Trivial.
 Conquer:Recursively sort 2subarrays.
 Combine:Linear-time merge.
 Using Master theorem

Analysis of Binary Search
 Recurrence for the worst case

Master Method, Example 1
 The form:
 Example:






dnnfbnaT
dnc
nT
if)()/(
if
)(
log
1loglog
loglog










nfbnaf
nfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
)()2/(4)( nOnTnT 
Solution: logba =log24=2, so case 1 says T(n) is Θ(n2).

 The form:
 Example:






dnnfbnaT
dnc
nT
if)()/(
if
)(
log
1loglog
loglog










nfbnaf
nfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
)log()2/(2)( nnnTnT 
Solution: logba=log22=1, so case 2 says T(n) is Θ(n log2 n).

 The form:
 Example:






dnnfbnaT
dnc
nT
if)()/(
if
)(
log
1loglog
loglog










nfbnaf
nfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
)log()3/()( nnnTnT 
Solution: logba=0, so case 3 says T(n) is Θ(n log n).

 The form:
 Example:






dnnfbnaT
dnc
nT
if)()/(
if
)(
log
1loglog
loglog










nfbnaf
nfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
)()2/(8)( 2
nnTnT 
Solution: logba=3, so case 1 says T(n) is Θ(n3).

 The form:
 Example:






dnnfbnaT
dnc
nT
if)()/(
if
)(
log
1loglog
loglog










nfbnaf
nfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
)()3/(9)( 3
nnTnT 
Solution: logba=2, so case 3 says T(n) is Θ(n3).

 The form:
 Example:






dnnfbnaT
dnc
nT
if)()/(
if
)(
log
1loglog
loglog










nfbnaf
nfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
)1()2/()(  nTnT
Solution: logba=0, so case 2 says T(n) is Θ(log n).
(binary search)

 The form:
 Example:






dnnfbnaT
dnc
nT
if)()/(
if
)(
log
1loglog
loglog










nfbnaf
nfnTnnf
nnnTnnnf
nnTnOnf
a
kaka
aa
b
bb
bb
)(log)2/(2)( nnTnT 
Solution: logba=1, so case 1 says T(n) is Θ(n).
(heap construction)

Divide and Conquer

More Related Content

What's hot

Similar to Divide and Conquer

More from Mohammed Hussein

Recently uploaded

In this document

Divide and Conquer