1. Logic-Based Methods for Global Optimization J. N. Hooker Carnegie Mellon University, USA November 2003
2. <ul><li>Basic Idea </li></ul><ul><li>Assume the problem becomes convex when certain variables are fixed. </li></ul><ul><ul><li>If these variables are discrete, we can reformulate the problem as disjunctions of convex constraints. </li></ul></ul><ul><ul><li>If some of them are continuous, discretize them to obtain an approximate global solution. </li></ul></ul><ul><li>Motivation is to take advantage of advanced solution methods: </li></ul><ul><ul><li>Branch-and-bound method chooses the appropriate disjunct in each constraint. </li></ul></ul><ul><ul><li>Nonlinear programming method solves convex subproblems that result when disjuncts are chosen. </li></ul></ul>
3. <ul><li>Outline </li></ul><ul><li>General form of problem </li></ul><ul><li>Structural design example </li></ul><ul><li>Disjunctive formulation </li></ul><ul><li>Branch and bound with convex relaxations </li></ul><ul><ul><li>Big-M formulation </li></ul></ul><ul><ul><li>Convex hull formulation </li></ul></ul><ul><li>Logic-based outer approximation </li></ul><ul><li>Logic-based Benders decomposition </li></ul><ul><li>Branch and bound with convex quasi-relaxations </li></ul><ul><li>Realistic structural design problem </li></ul><ul><ul><li>Solution by MILP </li></ul></ul><ul><ul><li>Solution by quasi-relaxation </li></ul></ul><ul><li>Other applications </li></ul>
4. General Form of Problem Logical conditions on y Assume that when is fixed to , we get a convex problem convex functions of x Vector of functions If is continuous, discretize it, to get approximate global solution.
5. Objective is defined in the constraints We assume one per constraint. Many problems have this form. If not, constraints can in principle be put into this form by change of variable.
6. For example, consider Use the change of variable and the constraints have the desired form: One y j per constraint Logical condition
7. Structural Design Example = compression of bar j = thickness (cross-sectional area) of bar j cost = Choose bar thickness that minimizes cost. This example is intended only to illustrate the algorithms. A more realistic model for structural design is presented at the end of the talk. displacement = displacement = cost of steel penalty for displacement load = 10 load = 20
8. Global optimization problem: Hooke’s law or many closely spaced values for continuous problem displace-ment thickness Can be written in desired form:
9. How to Solve It? Can in principle use branch and bound by branching on . But continuous relaxations at nodes of the search tree are in general nonconvex . So we write the problem in disjunctive form.
10. Disjunctive Formulation <ul><li>Now each disjunct is convex. We will solve by: </li></ul><ul><li>Branch and bound with convex relaxations (use disjunctive programming or MINLP) . </li></ul><ul><li>Logic-based outer approximation with linear relaxations. </li></ul><ul><li>Relaxations can be large when there are many disjunctions. In this case consider: </li></ul><ul><li>Logic-based Benders decomposition with discrete relaxation. </li></ul><ul><li>Branch and bound with convex quasi-relaxations (requires that constraint functions satisfy certain properties). </li></ul>
11. Recall the example… Disjunctive formulation is: Disjuncts are convex (linear)
12. Branch and Bound with Convex Relaxations <ul><li>Two convex relaxations of a disjunction </li></ul><ul><ul><li>Big-M : Write a big-M formulation with 0-1 variables and take its continuous relaxation (i.e., drop the integrality requirement on the 0-1 variables). </li></ul></ul><ul><ul><li>Convex hull : Write a convex hull formulation with 0-1 variables and take its continuous relaxation. </li></ul></ul><ul><li>Two solution options </li></ul><ul><ul><li>Disjunctive programming : branch on disjunctions. </li></ul></ul><ul><ul><li>Mixed integer nonlinear programming (MINLP): branch on 0-1 variables. </li></ul></ul><ul><li>Optimal value of relaxation provides a lower bound that is used in a branch-and-bound scheme. </li></ul>
13. Big-M formulation of disjunction The disjunction: Big-M formulation: Where M v is a vector of valid upper bounds on the component functions of g ( x,v ). It is assumed that x is bounded above and below. To obtain relaxation, replace with
14. Example of big-M relaxation Disjunction… Relaxation… Projection is
15. The disjunction: Convex hull formulation of disjunction Stubbs & Mehrotra; Grossmann & Lee Assume each g ( x , v ) is bounded as well as convex. Also assume x L x x U Write every point in the relaxation as a convex combination of points satisfying the disjuncts Use change of variable Nonconvex
16. Restore convexity by multiplying by This is a convex hull relaxation (i.e., projects onto convex hull in x -space). But disaggregation of x adds many new variables. To get 0-1 formulation, replace convex with
17. Example of convex hull relaxation Disjunction… Convex hull relaxation…
18. Solve structural design example with big-M formulation Big-M formulation: Disjunctive formulation: Solve by disjunctive programming or MINLP. Get optimal solution at the root node. Thus
19. Solve structural design example with convex hull formulation Convex hull formulation: Disjunctive formulation: Solve by disjunctive programming or MINLP.
20. Logic-Based Outer Approximation T ü rkay and Grossmann <ul><li>Allows one to use linear relaxations. But one must solve a mixed integer linear programming (MILP) master problem repeatedly. </li></ul><ul><li>Solve a master problem containing 1 st -order approximations of the disjuncts to obtain a value for y . </li></ul><ul><ul><li>Solve with MILP, which uses linear relaxations. </li></ul></ul><ul><li>Solve the subproblem that results when y is fixed to this value, to get value for x . </li></ul><ul><ul><li>Compute 1 st -order approximations about previously obtained values of x, y . </li></ul></ul><ul><li>Continue until value of master problem best value obtained in a subproblem so far. </li></ul><ul><li>Begin with warm start by precomputing 1 st -order approximations about several values of ( x,y ). </li></ul>
21. Disjunctive formulation again: The master problem in iteration K + 1 is where ( x k ,y k ) are solutions from previous iterations. The nonlinear subproblem in iteration K is
22. Solve structural design example with logic-based outer approximation Master problem: Disjunctive formulation again: Disjuncts already linear
23. Solve master problem as MILP (Big-M formulation):` For warm start, solve subproblem for 2 y ’s: y 1 = (1,1), which yields x 1 = (20,20) y 2 = (2,2), which yields x 2 = (5,10). This results in the master problem:
24. Solve master problem and get which implies Subproblem solution is Next master problem is Solution is and the algorithm terminates with y = (1,2). same new
25. Logic-Based Benders Decomposition Hooker and Ottosson <ul><li>Can be useful when variables have a large number of discrete values, resulting in a large number of disjuncts. </li></ul><ul><li>Convergence can be slow. </li></ul><ul><li>Solve a master problem for y . </li></ul><ul><ul><li>The master problem incompletely describes the projection of the original problem onto the y -space . </li></ul></ul><ul><li>Solve the subproblem that results when y is fixed to this value. </li></ul><ul><ul><li>Obtain Benders cut from inference dual of the subproblem . </li></ul></ul><ul><ul><li>Add the cut to the master problem to rule out some solutions that are no better than the previous one. </li></ul></ul><ul><li>Continue until the master and subproblem converge in value. </li></ul><ul><li>Best to have a warm start with “don’t-be-stupid” constraints involving y . </li></ul>
26. Disjunctive formulation again: The nonlinear subproblem in iteration K is Lagrange multiplier The master problem in iteration K + 1 is Optimal value = Logical Benders cuts
27. Solve structural design example with logic-based Benders decomposition Initial master problem: One solution is Solve subproblem: Corresponds to y 1 = 1 Corresponds to y 2 = 1 Lagrange multipliers Subproblem solution is Don’t-be-stupid constraint
28. Since the master problem is Solution is Continue in this fashion. Master problem in iteration 4 is: Solution is The algorithm terminates with y = (1,2). same
29. Branch and Bound with Convex Quasi-Relaxations <ul><li>Does not require disjunctive formulation and is therefore useful when there are many discrete values. </li></ul><ul><li>But the constraint functions must have a certain form. </li></ul><ul><li>Solve the problem by branch and bound. </li></ul><ul><li>Obtain bounds from quasi-relaxations at each node. </li></ul><ul><li>Given problem P : </li></ul>a problem Q : is a quasi-relaxation of P if for any feasible solution x of P , there is a feasible solution x of Q with f ( x ) f ( x ) . Thus one can obtain a valid lower bound by solving a quasi-relaxation.
30. Consider the problem, Theorem. Suppose that each is either (a) convex [for ( i,j) J 1 ] or (b) concave in y j and homogeneous in x : Then the following is a convex quasi-relaxation : [for ( i,j) J 2 ]. Suppose also that
31. Why? Take any feasible solution of To obtain a feasible solution of do the following: Then concavity homogeneity
32. So we have a feasible solution of the quasi-relaxation with value that is less than or equal to (in fact equal to) that of the original problem. convex, because g j ( x , y ) is convex convex, because g j ( x , y ) is convex in x satisfied, by above argument satisfied, by construction satisfied, by construction
33. Solve continuous version of structural design example with quasi-relaxations Original formulation: Put in proper form: convex Concave in y j & homogeneous in 1 st argument ( s j , x j ) Discretize
34. The quasi-relaxation is: Can now re-aggregate s j :
35. Beginning of branch-and-bound tree Root node x 0 = 1177.8 = (0.667,0.667) y = (1,1) x 0 = 1322 = (0,0.667) y = (1,1) x 0 =1900 = (0,0) y = (1,1) feasible solution y 1 [0,1] y 1 [1.1,3] x 0 = 1283 = (0.816,0.667) y = (1.45,1) y 2 [0,1] y 2 [1.1,3] x 0 =1352 = (0,0.816) y = (1,1.45) Total 63 nodes out of 31 31 possible solutions. Get y = (1.1, 2.0) with z 0 = 1394.5 Global optimum is y = (1.126, 1.972) with z 0 = 1394.1
36. Degree of freedom i Load Bar j Length Elongation Cross-sectional area Displacement Realistic Structural Design Problem Hooke’s law Equilibrium Compatibility Elongation bounds Displacement bounds
37. Solution as MI(N)LP Ghattas and Grossmann The disjunctive formulation is Since everything is linear, the big-M and convex hull formulations are linear. Can solve as a mixed integer linear programming (MILP) problem. Discrete sizes for bar j
38. The convex hull MILP model is
39. Solution with convex quasi-relaxations Bollapragada, Ghattas and Hooker Check that the problem has the right form: convex Concave (linear) in y j and homogeneous in v j , f j
40. The quasi-relaxation is
41. 10-bar cantilever truss 25-bar electrical transmission tower Some problem instances
42. 72-bar building Use symmetries to help solve problem
44. Applications of Logic-Based Methods for Nonlinear Global Optimization <ul><li>Logic-based outer approximation applied to chemical processing network design </li></ul><ul><ul><li>Quesada and Grossmann 1992; T ü rkay and Grossmann 1996 </li></ul></ul><ul><li>Disjunctive programming applied to chemical processing network design </li></ul><ul><ul><li>Vecchietti and Grossmann 1999 </li></ul></ul><ul><li>Convex quasi-relaxations applied to truss structure design </li></ul><ul><ul><li>Bollapragada, Ghattas and Hooker 2001 </li></ul></ul><ul><li>Disjunctive programming applied to tray placement in distillation columns </li></ul><ul><ul><li>Barttfeld, Aguire and Grossmann 2003 </li></ul></ul><ul><li>Logic-based Benders decomposition applied to planning and scheduling (linear case) </li></ul><ul><ul><li>Hooker 2000, 2003.; Jain and Grossmann 2001 </li></ul></ul>
45. Surveys <ul><li>I. E. Grossmann, Review of nonlinear mixed-integer and disjunctive programming techniques for process systems engineering, Carnegie Mellon University, June 2001 </li></ul><ul><li>J. N. Hooker, Logic-Based Methods for Optimization , John Wiley & Sons, 2000. </li></ul>
Be the first to comment