The document discusses divide and conquer algorithms. It provides examples of divide and conquer sorting algorithms like merge sort and quicksort. It also covers the master theorem for analyzing recursive algorithms, applying it to cases where the recurrence relation involves arithmetic or geometric growth. Key aspects like partitioning in quicksort and merging in merge sort are also summarized.

1. Divide and Conquer
Pasi Fränti
2.11.2016

2. 2
Divide and Conquer
1. Divide to sub-problems
2. Solve the sub-problems
3. Conquer the solutions
By recursion!

3. 3
StupidExample(N)
IF N=0 THEN RETURN O(1)
ELSE
FOR i←1 TO N DO N
WRITE(‘x’); O(1)
StupidExample(N-1); T(N-1)
END;
1)1()(  NTNNT
Stupid example

4.     
 
      
   
          
   
   2
1
1
0
1
2
1
1
0
...
4134321
)3(321
31221
)2(21
2111
)1(1)(
NN
NN
Ni
TkiN
NTNNNNN
NTNNN
NTNNN
NTNN
NTNN
NTNNT
N
i
k
i















Analysis by substitution method
)1(1)(  NTNNT
k=N
RepeatuntilT(0)
k+…+3+2+1 → 1+2+3+…+k

5. Sort(A[i, j])
q  FindMin(A[i, j]);
Swap(A[i],A[q])
Sort(A[i+1, j]);
FindMin(A[i, j])
q  FindMin(A[i+1, j]);
IF A[i]<A[q] THEN RETURN i
ELSE RETURN q;
)(
)1(1)(
2
NO
NTNNT


Sorting algorithm
Recursive “through the bones”

6. 

 

otherwise
if)(
)(
a
cNedNcNTb
NT
Master Theorem
Arithmetic case
Time complexity function:
Solution:









1)(
0,1)(
0,1)(
)(
)/(
2
bbO
dbNO
dbNO
NT
cN

7.     
      
      cNcNcN
bbecbcNbNdcTb
bbecNbcNbNdcNTb
becNbNdcNTb
edNcNbTNT
///
223
2
...1...
12)3(
1)2(
)()(




Master Theorem
Proof for arithmetic case
Substitution:

8.       cNcNcN
bbecbcNbNdcTb ///
...1... 
Master Theorem
Proof for arithmetic case
Case 1: 0,1  db
a1 0   1cN
   NacNNT 
N/c terms

9.       cNcNcN
bbecbcNbNdcTb ///
...1... 
Master Theorem
Proof for arithmetic case
Case 2: 0,1  db
a1 cid
cN
i
1
       2
2
2
22
21 N
c
N
c
N
cNcNNT 
  1cN
Arithmetic
sum

10.       cNcNcN
bbecbcNbNdcTb ///
...1... 
Master Theorem
Proof for arithmetic case
Case 3: 1b
ab cN
 cN
bd 
cN
be
Dominating
term
     cNcN
bbedaNT 

11. Quicksort(A[i, j])
IF i < j THEN O(1)
k ← Partition(A[i, j]); O(N)
Quicksort(A[i, k]); T(N/2)
Quicksort(A[k+1, j]); T(N/2)
ELSE RETURN;
     221 NTNTNNT 
Quicksort

12. Partition algorithm
 Choose (any) element as pivot-value p.
 Arrange all x≤p to the left side of list.
 Arrange all x>p to the right side.
 Time complexity O(N).
7 3 112 20 136 1 10 145 8
7 3 11 2 20 13 6 1 10 14 5 8
p=7
x≤7 x>7

13. Partition(A[i, j])
pivot = A[j];
r = i-1;
FOR k = i TO j-1 DO
IF (A[k] ≤ pivot) THEN
r = r + 1;
SWAP(A[r], A[k]);
SWAP(A[r+1], A[j]);
RETURN r+1;
r∙∙∙ k
x≤p x>p
∙∙∙
unprocessed
FOR increases k
R increases
when needed
Partition algorithm

14. Selection(A[i, j], k)
IF i=j THEN
RETURN A[i];
ELSE
q ← Partition(A, i, j);
size ← (q-i)+1;
IF k ≤ size THEN Selection(A[i, q], k);
ELSE Selection(A[q+1, j], k-size);
2)
2
()(  dN
N
TNT
Selection in linear time
Find the kth smallest

15. edNcNTbNT  )()(
Master Theorem
Geometric case









cbNO
cbNNO
cbNO
NT
bc
)(
)log(
),(
)(
log
Time complexity function:
Solution:

16. Master Theorem
Proof of geometric case
   
    
   
      
        
)1(
...1
...
)1(
)1(
21
2
122
122
2









bbbe
bcbcbcbdTb
bebcbcbdcTb
becbcdcTb
eNdecNdcNTbb
edNcNTbNT
pp
ppp
pppp
ppp
Np
cN
c
p
log

Extract
psteps

17. Case 1: cb 
Master Theorem
Proof of geometric case
  )1(...1 12
2




























 p
p
pp
bbbe
b
c
b
c
b
c
bdTb 
Np
cN
c
p
log

Nb Nc
log
 N
N
b
dc
cc
b
d
c
b
d
b
c
bd
p
p
p
p








 

1
1
Nb Nc
log
1
11
0 




 a
a
a
nn
i
i
Geometric sum
Geometric sum
a>1

18. Case 2: cb 
Master Theorem
Proof of geometric case
Nb Nc
log
Nb Nc
log
Log N terms
Np
cN
c
p
log

NN log
=1
N
Geometric sum
   NNNT log
  )1(...1 12
2




























 p
p
pp
bbbe
b
c
b
c
b
c
bdTb 

19. Case 3: cb 
Master Theorem
Proof of geometric case
p
b
…summation…
p
bp
b
Np
cN
c
p
log

  bNp cc
NbbNT loglog

 
 
b
bN
bN
Nb
NbN
c
cc
cc
cc
ccc
N
c
c
c
cb
log
loglog
loglog
loglog
logloglog







  )1(...1 12
2




























 p
p
pp
bbbe
b
c
b
c
b
c
bdTb 
<1

20. MergeSort(A[i, j])
IF i<j THEN
k := (i+j)/2; O(1)
MergeSort (A[i, k]); T(N/2)
MergeSort(A[k+1, j]); T(N/2)
Merge(A[i, j], k); c∙N
Merge sort
Main algorithm

21. Merge(A[i, j], k)
{
l←i; m←k+1; t←i;
WHILE (l ≤ k) OR (m ≤ j)
IF l>k THEN B[t]←A[m]; m←m+1;
ELSEIF m>j THEN B[t]←A[l]; l←l+1;
ELSEIF A[l]<A[m] THEN B[t]←A[m]; m←m+1;
ELSE B[t]←A[l]; l←l+1;
t←t+1;
FOR t ← i TO j DO A[t]←B[t];
}
Merge sort
Merge step

22. 47)2/(21)2/(2)(
)()37()(


NNTTNTNT
NONONT
merge
merge
Since we have b=c  O(N log N)
Merge sort
Time complexity

23. 47)2/(2)(  NNTNT
)log(
49log7
247log2
)221(473)2(2
24244777)2(2
24477)4)2(7)2(2(2
24477)2(22
47)427)2(2(2
log
0
2
log
333
233
232
2
2
2
NNO
NNN
NN
NNT
NNNNT
NNNNT
NNNT
NNNT
N
i
iN









Merge sort
Substitution method

24. 3 6 2 4
 2 3 4 5
1 8 1 2 0
1 4 4 9 6
1 0 8 7 2
7 2 4 8
8 4 9 8 2 8 0
Karatsuba-Ofman multiplication
Multiplication of two n-digit numbers
School book algorithm:
)()( 2
nOnT 

25. Karatsuba-Ofman multiplication
Straightforward divide-and-conquer
tvktusvksu
tvktuksvksu
vuktskyx
nn
nnn
nn



2
22
22
)(
)()(
vkuy
tksx
n
n


2
2
One n-digit number divided into two n/2-digit
numbers (most and least significant)
3624=36∙102+24
2345=23∙102+45
Multiplication reformulated:

26. Karatsuba-Ofman multiplication
Time complexity analysis
tvktusvksu nn
 2
)(






2
n
T 





2
n
T 





2
n
T 





2
n
T
2
n
n
     24log2
5.4
2
4 nnn
n
TnT 






n nn

27. Karatsuba-Ofman multiplication
Divide-and-Conquer revised
     tvktvsuvutsksu
tvktusvksu
nn
nn


2
2
)(






2
n
T 





2
n
T 





2
n
T
     58.13log2
5.7
2
3 nnn
n
TnT 






+ 6 summations and 1.5 multiplications by kn