Theory of computation

1. Design and Analysis of
Approximation Algorithms
Ding-Zhu Du

2. Text Books
• Ding-Zhu Du and Ker-I Ko, Design and
Analysis of Approximation Algorithms
(Lecture Notes).
Chapters 1-8.

3. Schedule
• Introduction
• Greedy Strategy
• Restriction
• Partition
• Guillotine Cut
• Relaxation
• Linear Programming
• Local Ratio
• Semi-definite Programming

4. Rules
• You may discuss each other on 5
homework assignments. But, do not copy
each other.
• Each homework is 10 points. 4 top scores
will be chosen.
• Midterm Exam (take-home) is 30 points.
• Final Exam (in class) is 30 points.
• Final grade is given based on the total
points (A ≥ 80; 80 > B ≥ 60 ; 60 > C ≥ 40).

5. Chapter 1 Introduction
• Computational Complexity (background)
• Approximation Performance Ratio
• Early results

6. Computability
• Deterministic Turing Machine
• Nondeterministic Turing Machine
• Church-Turing Thesis

7. Deterministic Turing Machine
(DTM)
Finite Control
tape
head

8. The tape has the left end but infinite to the
right. It is divided into cells. Each cell
contains a symbol in an alphabet Γ. There
exists a special symbol B which represents
the empty cell.
a l p h a B e

9. • The head scans at a cell on the tape and
can read, erase, and write a symbol on the
cell. In each move, the head can move to
the right cell or to the left cell (or stay in
the same cell).
a

10. • The head scans at a cell on the tape and
can read, erase, and write a symbol on the
cell. In each move, the head can move to
the right cell or to the left cell (or stay in
the same cell).
a

11. • The finite control has finitely many states
which form a set Q. For each move, the
state is changed according to the
evaluation of a transition function
δ : Q x Γ → Q x Γ x {R, L}.

12. • δ(q, a) = (p, b, L) means that if the head
reads symbol a and the finite control is in
the state q, then the next state should be p,
the symbol a should be changed to b, and
the head moves one cell to the left.
p
a b
q

13. • δ(q, a) = (p, b, R) means that if the head
reads symbol a and the finite control is in
the state q, then the next state should be p,
the symbol a should be changed to b, and
the head moves one cell to the right.
q
p
a b

14. • There are some special states: an initial
state s and an final states h.
• Initially, the DTM is in the initial state and
the head scans the leftmost cell. The tape
holds an input string.
s

15. • When the DTM is in the final state, the DTM
stops. An input string x is accepted by the
DTM if the DTM reaches the final state h.
• Otherwise, the input string is rejected.
h
x

16. • The DTM can be represented by
M = (Q, Σ, Γ, δ, s)
where Σ is the alphabet of input symbols.
• The set of all strings accepted by a DTM
$M$ is denoted by L(M). We also say that
the language L(M) is accepted by M.

17. • The transition diagram} of a DTM is an
alternative way to represent the DTM.
• For M = (Q, Σ, Γ, δ, s), the transition diagram of
M is a symbol-labeled digraph G=(V, E)
satisfying the following:
V = Q (s = , h = )
E = { p q | δ(p, a) = (q, b, D)}.
a/b,D

18. M=(Q, Σ, Γ, δ, s) where Q = {s, p, q, h},
Σ = {0, 1}, Г = {0, 1, B}.
δ 0 1 B
s (p, 0, R) (s, 1, R) -
p (q, 0, R) (s, 1, R) -
q (q, 0, R) (q, 1, R) (h, B, R)
L(M) = (0+1)*00(0+1)*.
s p q h
1/1,R
0/0,R
1/1,R
0/0,R
0/0,R; 1/1,R
B/B,R

19. Nondeterministic Turing
Machine (NTM)
Finite Control
tape
head

20. The tape has the left end but infinite to the
right. It is divided into cells. Each cell
contains a symbol in an alphabet Γ. There
exists a special symbol B which represents
the empty cell.
a l p h a B e

21. • The finite control has finitely many states
which form a set Q. For each move, the
state is changed according to the
evaluation of a transition function
δ : Q x Γ → 2^{Q x Γ x {R, L}}.

22. Church-Turing Thesis
• Computability is Turing-Computability.

23. Multi-tape DTM
Input tape
(read only)
Storage
tapes
Output tape
(possibly, write only)

24. Time of TM
• TimeM (x) = # of moves that TM M takes on
input x.
• TimeM(x) < infinity iff x ε L(M).

25. Space
• SpaceM(x) = # of cell that M visits on the
work (storage) tapes during the
computation on input x.
• If M is a multitape DTM, then the work
tapes do not include the input tape and the
write-only output tape.

26. Time Bound
M is said to have a time bound t(n) if for
every x with |x| < n,
TimeM(x) < max {n+1, t(n)}

27. Complexity Class
• A language L has a (deterministic) time-
complexity t(n) if there is a multitape TM M
accepting L, with time bound t(n).
• DTIME(t(n)) = {L(M) | DTM M has a time
bound t(n)}
• NTIME(t(n)) = {L(M) | NTM M has a time
bound t(n)}

28. • P = U DTIME(n )
• NP = U NTIME(n )
c
c
C>0
C>0

29. NP Class

30. Earlier Results on Approximations
• Vertex-Cover
• Traveling Salesman Problem
• Knapsack Problem

31. Performance Ratio
MAX.
for
)
(
)
(
sup
MIN
for
)
(
)
(
sup
input
Approx
inpu
Opt
r
input
Opt
input
Approx
r
input
input



32. Constant-Approximation
• c-approximation is a polynomial-time
approximation satisfying:
1 < approx(input)/opt(input) < c for MIN
or
1 < opt(input)/approx(input) < c for MAX

33. Vertex Cover
• Given a graph G=(V,E), find a minimum
subset C of vertices such that every edge
is incident to a vertex in C.

34. Vertex-Cover
• The vertex set of a maximal matching
gives 2-approximation, i.e.,
approx / opt < 2

35. Traveling Salesman
• Given n cities with a distance table, find a
minimum total-distance tour to visit each
city exactly once.

36. Traveling Salesman with triangular
inequality
• Traveling around a minimum spanning
tree is a 2-approximation.

37. Traveling Salesman with Triangular
Inequality
• Minimum spanning tree + minimum-length
perfect matching on odd vertices is 1.5-
approximation

38. Minimum perfect matching on odd vertices
has weight at most 0.5 opt.

39. Lower Bound
1+ε 1+ε
1+ε
1 1
1+ε 1+ε








n
n
n
n
as
2
3
)
1
)(
1
2
(
2
)
1
(
2



40. Traveling Salesman without
Triangular Inequality
Theorem For any constant c> 0, TSP has no c-
approximation unless NP=P.
Given a graph G=(V,E), define a distance table
on V as follows:






E
v
u
c
V
E
v
u
v
u
d
)
,
(
if
,
|
|
)
,
(
if
,
1
)
,
(

41. Contradition Argument
• Suppose c-approximation exists. Then we
have a polynomial-time algorithm to solve
Hamiltonian Cycle as follow:
C-approximation solution < cn
if and only if
G has a Hamiltonian cycle

42. Knapsack
}.
,
max{
Output
.
such that
Choose
.
Sort
.
any
for
Hence
.
any
for
Assume
}.
1
,
0
{
t.
s.
max
1
1
1
1
1
2
2
1
1
2
2
1
1
2
2
1
1
























k
k
i
i
G
k
i
i
k
i
i
n
n
i
i
i
n
n
n
n
c
c
c
s
S
s
k
s
c
s
c
s
c
i
opt
c
i
S
s
x
S
x
s
x
s
x
s
x
c
x
c
x
c




43. 2-approximation
.
1
1
1
1
2 G
k
G
G
k
i
i
k
i
i
c
c
c
opt
c
c
opt
c









 


44. PTAS
• A problem has a PTAS (polynomial-time
approximation scheme) if for any ε > 0, it
has a (1+ε)-approximation.

45. Knapsack has PTAS
• Classify: for i < m, ci < a= cG,
for i > m+1, ci > a.
• Sort
• For
.
2
2
1
1
m
m
s
c
s
c
s
c


 
;
0
)
(
set
then
,
if
},
,
,
1
{






I
c
S
s
n
m
I
I
i
i




1

46. ).
(
max
)
(
Output
.
)
(
set
and
such that
maximum
choose
then
,
If
1
1
I
c
I
c
c
c
I
c
s
S
s
m
k
S
s
I
output
k
i
i
I
i
i
I
i
i
k
i
i
I
i
i

















Proof. }.
in
1
|
}
,...,
1
{
{
*
Let opt
x
n
m
i
I i 




47. 




























1
)
(
.
1
)
(
)
(
)
(
Hence,
.
*)
(
*)
(
then
,
If
.
*)
(
then
,
If
1
*
1
*
1
output
output
output
output
k
I
i
i
m
i
i
I
i
i
m
i
i
I
c
opt
opt
I
c
a
I
c
opt
I
c
a
I
c
c
I
c
opt
s
S
s
opt
I
c
s
S
s

48. Time
.
)
(
time
gives
This
.
/
)
1
(
2
|
|
with
hose
consider t
only
need
we
Therefore,
.
2
)
/
)
1
(
2
(
then
,
)/
2(1
|
I
|
If
.
2
Note
)
/
1
(
/
)
1
(
2 








O
G
I
i
i
G
n
n
O
I
I
opt
c
a
c
c
opt














49. Fully PTAS
• A problem has a fully PTAS if for any ε>0,
it has (1+ε)-approximation running in time
poly(n,1/ε).

50. Fully FTAS for Knapsack
exists.
subset
such
no
if
)
,
(
}}.
,...,
1
{
,
|
{
min
and
,
such that
i}
...,
{1,
in
indeces
of
subset
a
is
)
,
(
)
,
(
)
,
(
)
,
(
nil
j
i
c
i
I
j
c
s
s
S
s
j
c
j
i
c
I
i
i
I
i
i
j
i
c
i
i
j
i
c
i
i
j
i
c
i
i

















51. Pseudo Polynomial-time Algorithm
for Knapsak
• Initially,
otherwise
,
if
{1}
,
0
if
)
,
1
(
,
1
for
1
1













nil
c
j
j
j
c
c
c
, ... c
j n
sum 

53. ).
,
1
(
)
,
(
Set
.
but
,
)
,
1
(
2.
Case
);
,
1
(
)
,
(
Set
,
)
,
1
(
1.
Case
do
to
0
for
do
to
1
for
)
,
1
(
j
i
c
j
i
c
S
s
s
nil
c
j
i
c
j
i
c
j
i
c
nil
c
j
i
c
c
j
n
i
i
c
j
i
c
k
k
i
i
i
sum














 



54. }.
)
,
(
|
max{
Output
}.
{
)
,
1
(
)
,
(
set
else
)
,
1
(
)
,
(
set
then
If
.
)
,
1
(
and
,
,
)
,
1
(
4.
Case
}.
{
)
,
1
(
)
,
(
Set
.
)
,
1
(
and
,
,
)
,
1
(
3.
Case
)
,
1
(
)
,
1
(
)
,
1
(
)
,
1
(
nil
j
i
c
j
opt
i
c
j
i
c
j
i
c
j
i
c
j
i
c
s
s
s
nil
j
i
c
S
s
s
nil
c
j
i
c
i
c
j
i
c
j
i
c
nil
j
i
c
S
s
s
nil
c
j
i
c
i
j
i
c
k
k
c
j
i
c
k
k
i
c
j
i
c
k
k
i
i
i
c
j
i
c
k
k
i
i
i
i
i












































55. Time
• outside loop: O(n)
• Inside loop: O(nM) where M=max ci
• Core: O(n log (MS))
• Total O(n M log (MS))
• Since input size is O(n log (MS)), this is a
pseudo-polynomial-time due to M=2
3
log M

56. h
n
h
n
h
h
h
h
i
n
n
n
n
i
i
c
h
opt
x
c
x
c
x
c
c
S
h
n
h
n
O
x
x
S
x
s
x
s
x
s
x
c
x
c
x
c
M
h
n
c
c
)
1
1
(
show
We
.
Let
)).
)
1
(
log(
)
1
(
(
in time
computed
be
can
solution
optimal
Its
}.
1
,
0
{
s.t.
'
'
'
max
Consider
)
1
(
'
Set
2
2
1
1
4
2
2
1
1
2
2
1
1





















 





57. )
1
1
1
(
1
)
1
(
)
*
)
1
)
1
(
(
*
)
1
)
1
(
((
)
1
(
)
*
'
*
'
(
)
1
(
)
'
'
(
*
*
*
Suppose
1
1
1
1
1
1
1
1
.
2
2
1
1





























h
opt
h
M
opt
h
n
M
x
M
h
n
c
x
M
h
n
c
h
n
M
x
c
x
c
h
n
M
x
c
x
c
x
c
x
c
c
x
c
x
c
x
c
opt
n
n
n
n
n
h
n
h
n
h
n
h
h
n
n






58. Thanks, End

59. Lecture 3 Complexity of
Approximation
• L-reduction
• Two subclass of PTAS
• Set-cover ((ln n)-approximation)
• Independent set (n -approximation)
c