Divide-and-Conquer & Dynamic ProgrammingDivide-and-Conqu.docx

Divide-and-Conquer & Dynamic Programming
Divide-and-Conquer: Divide a problem to independent
subproblems, find the solutions of the subproblems, and then
merge the solutions of the subproblems to the solution of the
original problem.
Dynamic Programming: Solve the subproblems (they may
overlap with each other) and save their solutions in a table, and
then use the table to solve the original problem.
Example 1: Compute Fibonacci number f(0)=0, f(1）=1，
f(n)=f(n-1)+f(n-2) Using Divide-and-Conquer:
F(n) = F(n-1) + F(n-
2)
F(n-2) + F(n-3) + F(n-3)
+ F(n-4)
F(n-3)+F(n-4) + F(n-4)+F(n-5) + F(n-4)+F(n-5)
+ F(n-5) + F(n-6)
…………………….
Computing time: T(n) = T(n-1) + T(n-2), T(1) = 0
T(n)＝O(2 ） Using Dynamic Programmin: Computing
time=O(n)
n
Chapter 8 Dynamic Programming (Planning)
F(0)
F(1)
F(2)
F(3)
F(4)

Example 2
The matrix-train mutiplication problem
*
Structure of an optimal parenthesization
Matrix Size
A1 30×35
A2 35×15
A3 15×5
A4 5×10
A5 10×20
A6 20×25
Input
6
5
3
2
4

1
3
3
3
3
3
3
3
3
5
1
2
3
4
5
i
j
1 2 3 4 5
s[i,j]
6
5
3
2
4
i

j
m[i,j]
15125
10500
5375
3500
5000
0
11875
7125
2500
1000
0
9375
4375
750
0
7875
2625
0
15750
0
0

1 2 3 4 5 6
１
i-1
k
j
Matrix-Chain-Order(p)
１ n := length[p] -1;
2 for i = 1 to n
3 do m[i,i] := 0;
4 for l =2 to n
5 do {for i=1 to n-l +1
6 do { j := i+ l-1;
7 m[i,j] := ;
8 for k = i to j-1
9 do {q := m[i,k]+m[k+1,j] +p p p ;
10 if q < m[i,j]
11 then {m[i,j] :=q;
s[i,j] := k}; }; };
13 return m, s;
Input of algorithm: p , p , … , p （The size of A
= ｐ *ｐ）
Computing time O(n )
8
0
1

n
i
i+1
i
3
Example 3
Longest common subsequence (LCS)
A problem from Bioinformatics: the DNA of one organism may
be
S1 = ACCGGTCGAGTGCGCGGAAGCCGGCCGAAA, while
the DNA of another organism may be S2 =
GTCGTTCTTAATGCCGTTGCTCTGTAAA. One goal of
comparing two strands of DNA is to determine how “similar”
the two strands are, as some measure of how closely related the
two organisms are.
Problem Formulization
Given a sequence X = ( ), another sequence Z = (
) is a subsequence of X if there exists a strictly increasing
sequence ( ) of indices of X such that for all j = 1, 2,
…k, we have .
Theorem
Let X = ( ) and Y = ( ) be sequence,
and Z = ( ) be any LCS of X and Y.
Find a match
Didn’t find a match

Procedure Print-LCS(c,X,Y,i,j)
1 if i = 0 or j = 0
then return
2 if c[i,j] = c[i-1,j-1] + 1&& X[i]=Y[j]
then
{ Print-LCS(c,X,i-1,j-1)
print X[i]
}
else if c[i-1, j] > c[i,j-1]
then Print-LCS(c,X,i-1,j)
else Pring-LCS(c,X,i,j-1)
Procedure LCS-Length(X,Y)
m := length[X];
n := length[Y];
for i := 0 to m
do c[i, 0] := 0
for j := 0 to n
do c[0,j] := 0
for i := 1 to m
do for j := 1 to n
do if X[i] = Y[j]
then c[i,j] := c[i-1, j-1] +1
else if c[i-1,j] > c[i,j-1]
then c[i,j] := c[i-1,j]
else c[i,j] := c[i,j-1]
return c
Let c[i,j] be the length of an LCS of the sequences
Find a match
Didn’t find a match

Chapter 9 Greedy Technique
Greedy Technique: It makes a locally optimal choice in the hope
that this choice will lead to a globally optimal solution.
Example 1 Activity-selection problem
1 resource: such as classroom
n activities ： S={1,2,…n}
Activity i has a start time and a finish time
i.e., activity i takes place during time interval
The activity-selection problem is to select a maximum-size of
mutually compatibal activities.
Example
1 1 4
2 3 5
3 0 6
4 5 7
5 3 8
6 5 9
7 6 10
8 8 11
9 8 12
10 2 13
11 12 14
Computing time = O(n)
Greedy-Activity-Selector(S,f)
1 n := length[S];

2 A = {1};
3 j = 1;
4 for i = 2 to n
5 do if
6 then { A = AU{i}
7 j = i;};
8 return A;
Example 2 Huffman Code
Considering the problem of designing a binary character code
wherein each character is represented by a unique binary code
Fixed length code: each character is coded with same length.
For example: to code 6 characters a, b, c, d, e, f, 3 bits are
needed, where a=000, b=001, c=010, d=011, e=100, f=101.
Variable-length code: giving frequent characters short
codewords and infrequent characters long codewords.
a b c d
e f
Frequency (in thousands) 45 13 12 16
9 5
Fixed-length codeword 000 001 010 011
100 10
Variable-length codeword 0 101 100 111
1101 1100
A data file of 100,000 characters contains only the characters a
– f with the frequencies indicated. 300,000 bits are needed for
fixed length code. Using variable-length code only (45x1+ 13x3
＋12x3+ 16x3 + 9x4 + 5x4 )x1000 = 224000 bits are needed.

Prefix codes No codeword is also a prefix of some other
codeword
100
ａ:45 55
25 30
ｃ:12 b:13 14
d:16
f:5
e:9
0
1
0
1
0

1
0
1
0
1
Constructing a Huffman code
Greedy technique: the character with higher frequency has
smaller depth.
0
1

0
1
0
1
0
1
0
1
0
1
（a) f:5 e:9 c:12 b:13 d:16 a:45
(b) c:12 b:13 14 d:16 a:45
f:5 e:9
(c) 14 d:16 25 a:45
(d) 25 30 a:45
f:5 e:9 c:12 b:13
c:12 b:13 14 d:16
f:5 e:9
(e) a:45 55
(f) 100
25 30
a:45 55
c:12 b:3 14 d:16
25 30
f:5 e:9
c:12 e:13 14 d:16
f:5 e:9

C: the set of characters. f(c): c’s frequency
Huffman(C)
１ n:= |C|; Q := C;
2 for i = 1 to n-1
3 do {z := New-Node();
4 x := left[z] :=Extract-Min(Q ); y := right[z]
:=Extract-Min(Q)
5 f[z] := f[x] + f[y]
6 Insert(Q, z)};
7 Return Q
Computing time Q is a priority queue. Initialization of Q: O(n)
for statement: n－1 times of each using O(logn) time.
Totoally, O(nlogn) time.
Example 3 Single-source shortest-paths problem
For a given vertex called the source in a weighted connected
graph, find shortest paths to all other vertices.
Input:
Graph G=(V,E)
Weight w(u,v) for each edge (u,v)
Adjacent list Adj[v] for each vertex v
Output:
p(v): parent of v on shortest path from s to v
d[v]: weight of the shortest path from s to v
V={s,u,v,x,y},
E={(s,u),(s,x),(x,y),(u,x),(x,u),(u,v),(x,v),(v,y),(y,v),(y,s)}
w(s,u)=10,w(s,x)=5,w(x,y)=2,…
Adj[s]={x,u}, Adj[x]={y,u,v},…
p(x)=s, p(u)=x, p(y)=x, p(v)=u
d[s]=0,d[x]=5,d[y]=7,d[u]=8,d[v]=9
s
y
v

x
u
7
２
５
3
2
9
10
4
1
6
s
y
v
x
u

7
２
５
3
2
9
10
4
1
6
Find shortest paths from source s
Initilization Relaxiation
Repeatly selecting an edge to improve the shortest paths found
so far.
Supposing that edge (u,v) is selected and d[v] is the weight of
the shortest path from s to v found so far, if d[v] < d[u]+w(u,v)
then the shortest path from s to v is improved as follows:
p(v)=u and d[v]=d[u]+w(u,v)
u
v
s
w(u,v)
d[u]
d[v]
s
y
v
x
u

7
２
５
3
2
9
10
4
1
6
∞
u
v
y
x
s
∞
∞
∞
Red vertices: processed
Green vertices: unprocessed
0

7
２
５
3
2
9
10
4
1
6
∞
u
v
y
x
s
∞
∞
∞
0

7
２
５
3
2
9
10
4
1
6
8
u
v
y
x
s
14
7
5
0

7
２
５
3
2
9
10
4
1
6
10
u
v
y
x
s
∞
∞
5
0
7

２
５
3
2
9
10
4
1
6
8
u
v
y
x
s
13
7
5
0
7
２
５

3
2
9
10
4
1
6
8
u
v
y
x
s
9
7
5
0
7
２
５
3
2

9
10
4
1
6
8
u
v
y
x
s
9
7
5
Dijkstra’s Algorithm
S: Set of the processed vertices
Q: Set of the unprocessed vertices. Each vertex v of Q has a
value d[v]. Q is constructed to be a priority queue.
Adjacent list Adj[v] for each vertex v in Graph G.
Greedy-technique: repeatly select a vertex from Q who is
closest to s so far.
*
0

7
２
５
3
2
9
10
4
1
6
10
u
v
y
x
s
∞
∞
5
0

7
２
５
3
2
9
10
4
1
6
8
u
v
y
x
s
13
7
5
0

7
２
５
3
2
9
10
4
1
6
8
u
v
y
x
s
9
7
5
0

7
２
５
3
2
9
10
4
1
6
8
u
v
y
x
s
9
7
5
Initialization
S:
Q: s,u,v,x,y
p:p[s]=p[u]=p[v]=p[x]=p[y]=NIL
d:d[s]=0,d[u]=d[v]=d[x]=d[y]=∞
S: s
Q: u,x,v,y
p: p[s]=p[v]=p[y]=NIL, p[u]=p[x]=s
d: d[s]=0,d[u]=10, d[v]=∞，d[x]=5,d[y]=∞

S: s,x
Q: u,v,y
p: p[s]=NIL,p[x]=s, p[y]= p[u]=p[v]=x
d: d[s]=0,d[u]=8,d[v]=14,d[x]=5,d[y]=7
S: s,x,y
Q: u,v
p: p[s]=NIL, p[x]=s, p[y]= p[u]=x, p[v]=y
d: d[s]=0,d[u]=8,d[v]=13,d[x]=5,d[y]=7
S: s,x,y,u
Q: v
p: p[s]=NIL, p[x]=s, p[y]= p[u]=x, p[v]=u
d: d[s]=0,d[u]=8,d[v]=9,d[x]=5,d[y]=7
S: s,x,y,u,v
Q:
p: p[s]=NIL, p[x]=s, p[y]= p[u]=x, p[v]=u
d: d[s]=0,d[u]=8,d[v]=9,d[x]=5,d[y]=7
Q is a priority queue.
One Extract-Min(Q) operation uses O(log |V|) time. One
Relax(u,v,w) uses constant time and revise d[v] in Q uses
O(log|V|) time.
Totally, the algorithm runs in O((|V|+|E|)log|V|) =
O((n+m)logn) time.
Computing Time of Dijkstra’s Algorithm
G=G(V,E), where |V|=n, |E|=m
S: Set of the processed vertices
Q: Set of the unprocessed vertices. Each vertex v of Q has a
value d[v]. Q is constructed to be a priority queue.
Adjacent list Adj[v] for each vertex v in Graph G.

7500
5)
100
(10
5)
5
100
(
))
A
(A
(A
computing
for
tions
multiplica
of
Number
5250
5)
5
(10
5)
100
10
(
)
)A
A
((A
computing

for
tions
multiplica
of
Number
ely.
resepectiv
5,
5
5,
100
100,
10
columns
and
rows
of
number
with the
matrices
the
be

A
,
A
,
A
let
example,
For
tions.
multiplica
scalar
of
number
the
changes
zing
parenthesi
of
way
The
3
2
1
3
2
1
3
2
1
=

´
´
+
´
´
=
=
´
´
+
´
´
=
´
´
´
tions.
multiplica
scalar
of
number
the
minimizes
way that
a
in
product
the
ze
parenthesi
fully

,
demension
has
matrix
2
1
for
where
matrices,
of
,
,
,
,
chain
a
given
:
problem
chain
-
Matrix
tion
Multiplica
3

2
1
1
3
2
1
n
i
i-
i
n
A
A
A
A
p
p
A
,...,n
,
i
n
A
A
A
A
L
L
´
=
r
q
p
B
A
r

q
q
p
´
´
=
´
´
for
tions
multiplica
of
number
:
Note
j
if i
p
p
p
,j]
m[k
m[i,k]
j,
i
if
m[i,j]
.

p
p
p
,j]
m[k
m[i,k]
m[i,j]
A
A
A
k
A
A
m[i,j]
j
k
i
j
k
i
j
k
k
i
i..j
i..j
j
i
î
í
ì
<
+
+
+
=

=
+
+
+
=
´
=
=
-
£
£
+
}
1
{
Min
0
Therefore,
1
then
),
of
zation
parenthesi
optimal
(the
)
of
zation
parenthesi

optimal
(the
)
of
zation
parenthesi
optimal
(the
i.e.,
,
at
zed
parenthezi
is
of
zation
parenthesi
optimal
an
If
.
compute

to
needed
tions
multiplica
scalar
of
number
minimum
the
Let
1
1
-
j
k
i
..
1
..
..
.
be
of
size
the

and
Let
1
1
1
..
i
i-
i
j
i
i
i
j
i
p
p
A
A
A
A
A
A
´
=
+
+
L
3
s[2,5]
7125
11375

20
10
35
0
4375
]
5
,
5
[
]
4
,
2
[
7125
20
5
35
1000
2625
]
5
,
4
[
]
3
,
2
[
113000
20
15
35
2500

0
]
5
,
3
[
]
2
,
2
[
min
]
5
,
2
[
5
4
1
5
3
1
5
2
1
=
=
ï
î
ï
í
ì
=
×
×

+
+
=
+
+
=
×
×
+
+
=
+
+
=
×
×
+
+
=
+
+
=
p
p
p
m
m
p
p
p
m
m
p
p
p
m

m
m
.
computing
in
cost
optimal
the
achived
of
index
:
compute
to
needed
ions
mutiplicat
scalar
of
number
minimum
the
:

Output
m[i,j]
k
s[i,j]
A
m[i,j]
i..j
)
,...,
,
(
1
0
n
p
p
p
p
=
n
n
m
m
y
y
y
y
Y
x
x
x
x
X
...

...
1
2
1
1
2
1
-
-
=
=
n
n
m
m
y
y
y
y
Y
x
x
x
x
X

...
...
1
2
1
1
2
1
-
-
=
=
n
n
m
m
y
y
y
y
Y
x
x
x
x

X
...
...
1
2
1
1
2
1
-
-
=
=
m
x
x
x
,...,
,
2
1
n
y
y

y
,...,
,
2
1
k
z
z
z
,...,
,
2
1
.
and
X
of
LCS
an
is
Z
that
implies
then
,
If

3.
.
and
X
of
LCS
an
is
Z
that
implies
then
,
If
2.
.
and
X
of
LCS
an

is
Z
and
then
,
If
1.
1
1
-
m
1
1
-
m
1
-
k
-
-
¹
¹
¹
¹
=
=
=
n
m
k

n
m
m
k
n
m
n
n
m
k
n
m
Y
x
z
y
x
Y
x
z
y
x
Y
y
x
z
y
x
j
i
z
x
j
=
k
z

z
z
,...,
,
2
1
k
i
i
i
,...,
,
2
1
m
x
x
x
,...,
,
2
1
ï
î
ï
í
ì
¹
>
-
-
=
>
+
-
-

=
=
=
.
and
0
j
i,
if
])
,
1
[
],
1
,
[
max(
,
and
0
,
if
1
]
1
,

1
[
,
0
or
0
if
0
]
,
[
j
i
j
i
y
x
j
i
c
j
i
c
y
x
j
i
j
i
c

j
i
j
i
c
j
j
j
i
i
i
y
y
y
y
Y
x
x
x
x
X
...
...
1

2
1
1
2
1
-
-
=
=
.
and
j
i
Y
X
.
in
activities
selected
previously
all
with
compatible
is
if

into
activity
Put
far.
so
selected
activities
the
of
set
the
be
Let
:
repeatly
following
the
of
what
do
3
2

each
For
(2)
.
1
activity
select
(1)
others.
of
an those
earlier th
is
e
finish tim
its
if
earlier
activity
select the
:
hnique
Greedy tec
.
...

that
Suppose
2
1
A
i
A
i
A
,...,n
,
i
f
f
f
n
=
£
£
£
i
f
s
i
)
,
[
i
f
s
i

i
f
s
i
i
j
i
f
s
³
thus
is
file
a
encode
to
required
bits
of
number
The
tree
in the
leaf

s
c'
of
depth
the
:
)
(
file
in the
character
of
frequency
the
:
)
(
c
d
c
c
f
T
å
Î
=
C
c
T
c

d
c
f
T
B
)
(
)
(
)
(
0
d[s]
4
NIL
[v]
3
d[v]
do
2
V[G]
x
each verte
1
)
,
(

¬
¬
¥
¬
Î
-
-
p
for
s
G
Source
Single
Initialize
}
u;
[v]
3
v];
w[u,
d[u]
d[v]
{
then
2
v)
w(u,
d[u]
d[v]
if

1
)
,
,
(
Relax
¬
+
¬
+
>
p
w
v
u
}
w);
v,
Relax(u,
do
8
S
v
&
Adj[u]
for v
7
;
{u}

S
S
6
Min(Q);
-
Extract
u
{
do
5
0
Q
while
4
V[G];
Q
3
0;
S
2
s);
Source(G,

-
Single
-
Initialize
1
)
,
,
(
Ï
Î
¬
¬
¹
¬
¬
U
s
w
G
Dijkstra
}
w);
v,
Relax(u,
do
8
S
v
&
Adj[u]

for v
7
;
{u}
S
S
6
Min(Q);
-
Extract
u
{
do
5
0
Q
while
4
V[G];
Q

3
0;
S
2
s);
Source(G,
-
Single
-
Initialize
1
)
,
,
(
Ï
Î
¬
¬
¹
¬
¬
U
s
w
G
Dijkstra
}
w);
v,
Relax(u,
do

8
S
v
&
Adj[u]
for v
7
;
{u}
S
S
6
Min(Q);
-
Extract
{u
do
5
0
Q
while
4

V[G];
Q
3
;
0
S
2
s);
Source(G,
-
Single
-
Initialize
1
)
,
,
(
Ï
Î
¬
¬
¹
¬
¬
U
s
w
G
Dijkstra

Divide-and-Conquer & Dynamic ProgrammingDivide-and-Conqu.docx

Divide-and-Conquer & Dynamic ProgrammingDivide-and-Conqu.docx

Recommended

Recommended

More Related Content

Similar to Divide-and-Conquer & Dynamic ProgrammingDivide-and-Conqu.docx

Similar to Divide-and-Conquer & Dynamic ProgrammingDivide-and-Conqu.docx (20)

More from jacksnathalie

More from jacksnathalie (20)

Recently uploaded

Recently uploaded (20)

Divide-and-Conquer & Dynamic ProgrammingDivide-and-Conqu.docx