1 Linear Programming and its Usage inApproximation Algorithms for NP Hard        Optimization Problems             M. Reza...
2               Outline          • Introduction          • Linear Programming Overview          • NP Complexity Class     ...
3                  Introduction   • One of the most challenging problem in complexity     theory is problem P vs. NP.     ...
4   • So it seems that research on approximation     algorithms may lead us to good results about P vs.     NP.   • Genera...
5       • Linear Programming was this brake through         method.       • In this talk I focus on LP and its usage in NP...
6                Linear Programming Overview            • General formulation of linear programming is              presen...
7      • There are several Polynomial Algorithms for this        problem such as Ellipsoid and Interior point        metho...
8                  NP Complexity Class    • NP Complexity Class is only related to Decision      Problem.    • For example...
9       • We know the following definition about NP:                       L ∈ NP ⇔ ∃V(.,.) ∈ P, ∃P(.), ∀x ∈ Σ∗ ,         ...
10      • Example:                SAT = {< Φ >: Φ is a satisfying assignment}.                SAT ∈ NP Because :          ...
11                Definition :                L ≤ P L* ⇔ ∃f (Polynomial Time Function) ∀x ∈ L ⇔ f ( x) ∈ L* .             ...
12          Example :          SAT = {< Φ >| Φ is satisfiable assignment}.          3 - CNF = {< Φ >| Φ is 3 - cnf and sat...
13                             Φ ≡ (¬x1 → x2 ) ↔ ( x3 ∧ x2 )                                                     <->      ...
14   Example :   INTEGER − PROGRAMMING = {< Am×n , bm×1 >| Am×n and bm×1 are integer matrices,   ∃X n×1 ∈ Ζ n , Am×n X n×1...
15   • We conclude this section by the following theorem:                         Every language in NP is polynomial time ...
16                  Approximation Algorithms      • The following describes the connection of NP        optimization Compl...
17• Bound of Approximation:   • Suppose that problem P has an optimum solution with cost C-     op. Algorithm X solves pro...
18 • Theorem:      It does not exist any polynomial time approximation      algorithm with Polynomial Bound for OPT-TSP un...
19  • I think that it’s now time to study Randomized    Rounding technique for solving optimization problems.  • I explore...
20                Case Study1: Minimum Weight Vertex                Cover• Now we study randomized rounding method in  app...
21                                     Fig2: Vertex CoverLinear Programming and its Usage in Approximation Algorithms for ...
22           • It is proved that decision version of this             optimization problem is NP Hard.           • At the ...
23       • As it was known it is believed that there is no         polynomial time algorithm for integer programming.     ...
24       • The Last method is called Relaxation to linear         programming.         programming       • It’s now time t...
25             Proof :             C :: The soulution of algorithm.             C* :: The real optimal set of integer prog...
26  • Hastad proved that there is no Polynomial time algorithm for    vertex cover that achieves an approximation better t...
27                  Case Study2: MAXSAT Problem                  MAXSAT :                  Given K - CNF Formula with each...
28      • We state the following randomized algorithm for this        problem. (Johnson 1974).            1) for variables...
29             Proof :             Define the following random varibles :             I j = 1 if clause C j satisfiable if...
30         • We can extend Johnson algorithm as follow:                     1) for variables x1 ,...x n randomly and indep...
31          • Now we model it with IP and relax it.   Integer Programming Model for MAXSAT :   Max ∑ w jz j               ...
32        Algorithm       1) Solve the LP and find vector (y* , z * ).        2) use extended Johnson algorithm with p i =...
33        Proof :        ∑ w {1 − ∏ (1 − p )∏ ( p )} =            Cj                 j                           xi       ...
34Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
35                    •     Algorithm                    2. Compute the value from Johnson                       algorithm...
36                  Conclusion     • In this talk the randomized rounding method is explored       with two example.     •...
Upcoming SlideShare
Loading in …5
×

Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems

3,058 views

Published on

Presented at Computer Science Department, Sharif University of Technology (Optimization Seminar ).

Published in: Education
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,058
On SlideShare
0
From Embeds
0
Number of Embeds
97
Actions
Shares
0
Downloads
44
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems

  1. 1. 1 Linear Programming and its Usage inApproximation Algorithms for NP Hard Optimization Problems M. Reza Rahimi, November 2005.
  2. 2. 2 Outline • Introduction • Linear Programming Overview • NP Complexity Class • Approximation Algorithms • Case Study1: Minimum Weight Vertex Cover • Case Study2: MAXSAT Problem • ConclusionLinear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  3. 3. 3 Introduction • One of the most challenging problem in complexity theory is problem P vs. NP. NP • Research on this open problem leads researchers to think of new methods and different approaches. • For example PCP, IP, approximation algorithms,... • For some problems it is proved that there does not exist any suitable approximation unless P=NP.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  4. 4. 4 • So it seems that research on approximation algorithms may lead us to good results about P vs. NP. • General Method for Approximation Algorithms of NP Hard Optimization is Greedy Method. • But we must search for Suitable Framework for studying NP problems.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  5. 5. 5 • Linear Programming was this brake through method. • In this talk I focus on LP and its usage in NP Hard Optimization Problems.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  6. 6. 6 Linear Programming Overview • General formulation of linear programming is presented as follow: Maximize C1X1 + C2X2 + … + CdXd Subject to A1,1X1 + … + A1,dXd ≤ b1 A2,1X1 + … + A2,dXd ≤ b2 … An,1X1 + … + An,dXd ≤ bn All calculation and Numbers are on Real Numbers.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  7. 7. 7 • There are several Polynomial Algorithms for this problem such as Ellipsoid and Interior point method. method • I neglect talking about them. • For these algorithms please refer to optimization references.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  8. 8. 8 NP Complexity Class • NP Complexity Class is only related to Decision Problem. • For example: SAT = {< Φ >: Φ is a satisfying assignment}. • So for any optimization problem we consider its related decision problem. • This will give us intuition about its hardness.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  9. 9. 9 • We know the following definition about NP: L ∈ NP ⇔ ∃V(.,.) ∈ P, ∃P(.), ∀x ∈ Σ∗ , 1. x ∈ L ⇒ ∃y, y ≤ P( x ) and V(x, y) accepts. 2. x ∉ L ⇒ ∀y, y ≤ P( x ) and V(x, y) rejects. • We can look at this process like this: Y Find Certificate Y y V(X,Y) X x X xLinear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  10. 10. 10 • Example: SAT = {< Φ >: Φ is a satisfying assignment}. SAT ∈ NP Because : 1) if Φ 0 ∈ SAT ⇒ We are given True assignment , and could check it in polynomial time. 2)if Φ 0 ∉ SAT ⇒ Then there is no true assignment. • There is one another important concept in complexity which is Reduction.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  11. 11. 11 Definition : L ≤ P L* ⇔ ∃f (Polynomial Time Function) ∀x ∈ L ⇔ f ( x) ∈ L* . f L L* f L L L* Fig1: Graphical Diagram of Reduction ConceptLinear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  12. 12. 12 Example : SAT = {< Φ >| Φ is satisfiable assignment}. 3 - CNF = {< Φ >| Φ is 3 - cnf and satisfiable}. we show that : SAT ≤ P 3 − CNF . (3 − CNF is in this form for example : (x1 ∨ x 2 ∨ x 3 ) ∧ ( x1 ∨ x 2 ∨ x 3 ) ∧ (x1 ∨ x 2 ∨ x 3 )Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  13. 13. 13 Φ ≡ (¬x1 → x2 ) ↔ ( x3 ∧ x2 ) <-> y1 y2 -> ^ -x1 x2 x3 x4 Φ ≡ ( y1 ↔ (¬x1 → x2 )) ∧ (( x3 ∧ x4 ) ↔ y2 ) ∧ ( y1 ↔ y2 ) Now each paranthesis can be converted into 3 - OR form using truth table.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  14. 14. 14 Example : INTEGER − PROGRAMMING = {< Am×n , bm×1 >| Am×n and bm×1 are integer matrices, ∃X n×1 ∈ Ζ n , Am×n X n×1 ≤ bm×1}. Obviously INTEGER − PROGRAMMING ∈ NP. We will show that : 3 − CNF ≤ P INTEGER − PROGRAMMING. Proof : ¬X 1 ∨ X 2 ∨ X 3 ↔ (1 − X 1 ) + X 2 + X 3 ¬X 1 ∨ X 2 ∨ X 3 ≡ 1 ↔ (1 − X 1 ) + X 2 + X 3 ≥ 1 X i ∈ Ζ, 0 ≤ X i ≤ 1Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  15. 15. 15 • We conclude this section by the following theorem: Every language in NP is polynomial time reducible to SAT language. • Language like SAT, INTEGER-PROGRAMMING, Hamiltonian Cycle, and 3-CNF are also the same and they belong to NP-Complete set.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  16. 16. 16 Approximation Algorithms • The following describes the connection of NP optimization Complete problem and its related optimization problem. Decision Version : TSP = {< Gn×n , b >| G n×n is complete non negative weighted graph and there exists Hamiltonian Cycle such that its cost is less than or equal b}. Optimization Version : OP − TSP : we have complete non negative weighted graph G n×n , find the Hamiltonian Cycle which its cost is minimun.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  17. 17. 17• Bound of Approximation: • Suppose that problem P has an optimum solution with cost C- op. Algorithm X solves problem P with bound ρ(n) if it finds feasible solution with cost C such as: max {c/c-op, c-op/c} ≤ ρ(n). • Note that with this definition we always have ρ(n) ≥1. • In the following I try to show that sometimes, finding good approximation for optimization problems is very hard.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  18. 18. 18 • Theorem: It does not exist any polynomial time approximation algorithm with Polynomial Bound for OPT-TSP unless P=NP. • Proof: we prove that if such an algorithm exists then we can Solve Hamiltonian Cycle in Polynomial Time. For each graph G with n vertex convert It into weighted complete graph G1 as G has Hamiltonian Cycle the following procedure: Iff 4) Assign 1 to each of its edge. OPT-TSP(G1)=n 5) Assign nρ(n) to the other edges.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  19. 19. 19 • I think that it’s now time to study Randomized Rounding technique for solving optimization problems. • I explore it with two examples.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  20. 20. 20 Case Study1: Minimum Weight Vertex Cover• Now we study randomized rounding method in approximation algorithms with an example.• Definition: • Vertex Cover: In undirected graph G=(V,E), Set A of vertices is said vertex cover if for every (u,v) ε E then u ε A or v ε A . • Minimum Weight Vertex Cover: In undirected graph G=(V,E) which each vertex has w(v) as positive weight Set A of vertices is said minimum weight vertex cover if for every (u,v) ε E then u ε A or v ε A and w(A) is the least.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  21. 21. 21 Fig2: Vertex CoverLinear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  22. 22. 22 • It is proved that decision version of this optimization problem is NP Hard. • At the first step we model it as integer programming. x : V → {0,1} if v ∈ Minimum Weight Vertex Cover Set ⇒ x(v) = 1. else x(v) = 0. Integer Programming Method : min ∑ w(v)x(v) v∈V ∀u , v ∈ V x(v) + x (u ) ≥ 1 0 ≤ x (v ) ≤ 1Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  23. 23. 23 • As it was known it is believed that there is no polynomial time algorithm for integer programming. • We must think about another method. • We investigate its related linear programming. x : V → [0,1] Linear Programming Method : min ∑ w(v)x(v) v∈V ∀u , v ∈ V x(v) + x(u ) ≥ 1 0 ≤ x (v ) ≤ 1Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  24. 24. 24 • The Last method is called Relaxation to linear programming. programming • It’s now time to state the approximation algorithm. Approximation Min Weight Vertex Cover(G, w) 1) C ← Φ 2) Compute x an optimal solution to linear programming. 3) for each v ∈ V 4) do if x (v) ≥ 0.5 / * Rounding Method * / 5) then C ← C ∪ {v} 6) Return C.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  25. 25. 25 Proof : C :: The soulution of algorithm. C* :: The real optimal set of integer programming. Z* :: The optimal value of the linear programming. Obviously we have Z* ≤ w(C* ). Obviously C is vertex cover. Z* = ∑ w(v) x(v) ≥ ∑ w(v) x(v) v∈V v∈V , x ( v ) ≥ 0.5 ≥ ∑ w(v)0.5 =∑ w(v)0.5 = 0.5w(C ). v∈C v∈V , x ( v ) ≥ 0.5 0.5w(C) ≤ Z* ≤ w(C* ) ⇒ { ρ = 2}Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  26. 26. 26 • Hastad proved that there is no Polynomial time algorithm for vertex cover that achieves an approximation better that 1.16 unless P=NP (1997). • Now we consider another example.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  27. 27. 27 Case Study2: MAXSAT Problem MAXSAT : Given K - CNF Formula with each Clause C j has weight Wj ≥ 0. Find assignment to variables that maximizes : ∑w C j is true j Example : Φ = (x1 + x 2 + x 3 + x 4 )( x1 + x 2 + x 3 + x 4 )(x1 + x 2 + x 3 + x 4 ). C1 = (x1 + x 2 + x 3 + x 4 ) → w 1 C 2 = ( x1 + x 2 + x 3 + x 4 ) → w 2 C3 = (x1 + x 2 + x 3 + x 4 ) → w 3 Max ∑w C j is true jLinear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  28. 28. 28 • We state the following randomized algorithm for this problem. (Johnson 1974). 1) for variables x1 ,...x n randomly assign to 0 or 1 with probability 0.5. 2) return ∑w C J istrue j Claim : 1 −1 The preceding algorithm has [(1- k )] approximation bound. 2Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  29. 29. 29 Proof : Define the following random varibles : I j = 1 if clause C j satisfiable if not 0. ∑ w =∑ w I C j is true j Cj j j 1 p(I j = 1 ) = (1 - ). 2k 1 p(I j = 0 ) = ( ). 2k 1 E{ ∑ w j} =E{∑ w jI j} =∑ w jE{I j} =(1- C j is true Cj Cj )∑ w j . 2k C j 1 −1 ρ = [(1 - k )] 2Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  30. 30. 30 • We can extend Johnson algorithm as follow: 1) for variables x1 ,...x n randomly and independently assign to 1 or 0 with probability p i and 1 - p i 2) return ∑w C J istrue j Claim : E{ ∑ w } = ∑ w (1 − ∏ (1 − p )∏ p ) C j is true j Cj j if xi i i if xi The proof is just the same.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  31. 31. 31 • Now we model it with IP and relax it. Integer Programming Model for MAXSAT : Max ∑ w jz j Relaxation Cj ∀C j ∑ y + ∑ (1- y ) ≥ z if x i i i j if x i 0 ≤ yi ≤ 1 0 ≤ zj ≤1 yi , z j ∈ Ζ Relaxation for MAXSAT : Max ∑ w jz j Cj ∀C j ∑ y + ∑ (1- y ) ≥ z if x i i if x i i j 0 ≤ yi ≤ 1 0 ≤ zj ≤1 yi , z j ∈ ℜLinear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  32. 32. 32 Algorithm 1) Solve the LP and find vector (y* , z * ). 2) use extended Johnson algorithm with p i = y*i . 3) return Lemma : for any feasible solution (y* , z* ) to LP and for any clause C j with k Literals, we have 1 1 - ∏ (1- y i )∏ (y i ) ≥ (1 − (1 − ) k ) z j if x i if x i k Claim : The above algorithm has (1- 1/e) -1 approximation ratio.Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  33. 33. 33 Proof : ∑ w {1 − ∏ (1 − p )∏ ( p )} = Cj j xi i i xi 1 ∑ w j {1 − ∏ (1 − yi )∏ ( yi )} ≥ (1 − (1 − ) k )∑ w j z j = * * * Cj xi xi k Cj 1 k * 1 k * (1 − (1 − ) ) Z LP ≥ (1 − (1 − ) ) Z IP k k [ρ = (1 − 1 / e) −1 ]Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  34. 34. 34Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  35. 35. 35 • Algorithm 2. Compute the value from Johnson algorithm (w1). 3. Compute the value from LP Based algorithm (w2). 4. Return MAX(w1,w2).Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
  36. 36. 36 Conclusion • In this talk the randomized rounding method is explored with two example. • This new method opens new insight into approximation algorithms and complexity theory. The EndLinear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems

×