Logic-Based Methods for Global Optimization J. N. Hooker Carnegie Mellon University, USA November 2003
Assume the problem becomes convex when certain variables are fixed.
If these variables are discrete, we can reformulate the problem as disjunctions of convex constraints.
If some of them are continuous, discretize them to obtain an approximate global solution.
Motivation is to take advantage of advanced solution methods:
Branch-and-bound method chooses the appropriate disjunct in each constraint.
Nonlinear programming method solves convex subproblems that result when disjuncts are chosen.
General form of problem
Structural design example
Branch and bound with convex relaxations
Convex hull formulation
Logic-based outer approximation
Logic-based Benders decomposition
Branch and bound with convex quasi-relaxations
Realistic structural design problem
Solution by MILP
Solution by quasi-relaxation
General Form of Problem Logical conditions on y Assume that when is fixed to , we get a convex problem convex functions of x Vector of functions If is continuous, discretize it, to get approximate global solution.
Objective is defined in the constraints We assume one per constraint. Many problems have this form. If not, constraints can in principle be put into this form by change of variable.
For example, consider Use the change of variable and the constraints have the desired form: One y j per constraint Logical condition
Structural Design Example = compression of bar j = thickness (cross-sectional area) of bar j cost = Choose bar thickness that minimizes cost. This example is intended only to illustrate the algorithms. A more realistic model for structural design is presented at the end of the talk. displacement = displacement = cost of steel penalty for displacement load = 10 load = 20
Global optimization problem: Hooke’s law or many closely spaced values for continuous problem displace-ment thickness Can be written in desired form:
How to Solve It? Can in principle use branch and bound by branching on . But continuous relaxations at nodes of the search tree are in general nonconvex . So we write the problem in disjunctive form.
Now each disjunct is convex. We will solve by:
Branch and bound with convex relaxations (use disjunctive programming or MINLP) .
Logic-based outer approximation with linear relaxations.
Relaxations can be large when there are many disjunctions. In this case consider:
Logic-based Benders decomposition with discrete relaxation.
Branch and bound with convex quasi-relaxations (requires that constraint functions satisfy certain properties).
Recall the example… Disjunctive formulation is: Disjuncts are convex (linear)
Branch and Bound with Convex Relaxations
Two convex relaxations of a disjunction
Big-M : Write a big-M formulation with 0-1 variables and take its continuous relaxation (i.e., drop the integrality requirement on the 0-1 variables).
Convex hull : Write a convex hull formulation with 0-1 variables and take its continuous relaxation.
Two solution options
Disjunctive programming : branch on disjunctions.
Mixed integer nonlinear programming (MINLP): branch on 0-1 variables.
Optimal value of relaxation provides a lower bound that is used in a branch-and-bound scheme.
Big-M formulation of disjunction The disjunction: Big-M formulation: Where M v is a vector of valid upper bounds on the component functions of g ( x,v ). It is assumed that x is bounded above and below. To obtain relaxation, replace with
Example of big-M relaxation Disjunction… Relaxation… Projection is
The disjunction: Convex hull formulation of disjunction Stubbs & Mehrotra; Grossmann & Lee Assume each g ( x , v ) is bounded as well as convex. Also assume x L x x U Write every point in the relaxation as a convex combination of points satisfying the disjuncts Use change of variable Nonconvex
Restore convexity by multiplying by This is a convex hull relaxation (i.e., projects onto convex hull in x -space). But disaggregation of x adds many new variables. To get 0-1 formulation, replace convex with
Example of convex hull relaxation Disjunction… Convex hull relaxation…
Solve structural design example with big-M formulation Big-M formulation: Disjunctive formulation: Solve by disjunctive programming or MINLP. Get optimal solution at the root node. Thus
Solve structural design example with convex hull formulation Convex hull formulation: Disjunctive formulation: Solve by disjunctive programming or MINLP.
Logic-Based Outer Approximation T ü rkay and Grossmann
Allows one to use linear relaxations. But one must solve a mixed integer linear programming (MILP) master problem repeatedly.
Solve a master problem containing 1 st -order approximations of the disjuncts to obtain a value for y .
Solve with MILP, which uses linear relaxations.
Solve the subproblem that results when y is fixed to this value, to get value for x .
Compute 1 st -order approximations about previously obtained values of x, y .
Continue until value of master problem best value obtained in a subproblem so far.
Begin with warm start by precomputing 1 st -order approximations about several values of ( x,y ).
Disjunctive formulation again: The master problem in iteration K + 1 is where ( x k ,y k ) are solutions from previous iterations. The nonlinear subproblem in iteration K is
Solve structural design example with logic-based outer approximation Master problem: Disjunctive formulation again: Disjuncts already linear
Solve master problem as MILP (Big-M formulation):` For warm start, solve subproblem for 2 y ’s: y 1 = (1,1), which yields x 1 = (20,20) y 2 = (2,2), which yields x 2 = (5,10). This results in the master problem:
Solve master problem and get which implies Subproblem solution is Next master problem is Solution is and the algorithm terminates with y = (1,2). same new
Logic-Based Benders Decomposition Hooker and Ottosson
Can be useful when variables have a large number of discrete values, resulting in a large number of disjuncts.
Convergence can be slow.
Solve a master problem for y .
The master problem incompletely describes the projection of the original problem onto the y -space .
Solve the subproblem that results when y is fixed to this value.
Obtain Benders cut from inference dual of the subproblem .
Add the cut to the master problem to rule out some solutions that are no better than the previous one.
Continue until the master and subproblem converge in value.
Best to have a warm start with “don’t-be-stupid” constraints involving y .
Disjunctive formulation again: The nonlinear subproblem in iteration K is Lagrange multiplier The master problem in iteration K + 1 is Optimal value = Logical Benders cuts
Solve structural design example with logic-based Benders decomposition Initial master problem: One solution is Solve subproblem: Corresponds to y 1 = 1 Corresponds to y 2 = 1 Lagrange multipliers Subproblem solution is Don’t-be-stupid constraint
Since the master problem is Solution is Continue in this fashion. Master problem in iteration 4 is: Solution is The algorithm terminates with y = (1,2). same
Branch and Bound with Convex Quasi-Relaxations
Does not require disjunctive formulation and is therefore useful when there are many discrete values.
But the constraint functions must have a certain form.
Solve the problem by branch and bound.
Obtain bounds from quasi-relaxations at each node.
Given problem P :
a problem Q : is a quasi-relaxation of P if for any feasible solution x of P , there is a feasible solution x of Q with f ( x ) f ( x ) . Thus one can obtain a valid lower bound by solving a quasi-relaxation.
Consider the problem, Theorem. Suppose that each is either (a) convex [for ( i,j) J 1 ] or (b) concave in y j and homogeneous in x : Then the following is a convex quasi-relaxation : [for ( i,j) J 2 ]. Suppose also that
Why? Take any feasible solution of To obtain a feasible solution of do the following: Then concavity homogeneity
So we have a feasible solution of the quasi-relaxation with value that is less than or equal to (in fact equal to) that of the original problem. convex, because g j ( x , y ) is convex convex, because g j ( x , y ) is convex in x satisfied, by above argument satisfied, by construction satisfied, by construction
Solve continuous version of structural design example with quasi-relaxations Original formulation: Put in proper form: convex Concave in y j & homogeneous in 1 st argument ( s j , x j ) Discretize
The quasi-relaxation is: Can now re-aggregate s j :
Beginning of branch-and-bound tree Root node x 0 = 1177.8 = (0.667,0.667) y = (1,1) x 0 = 1322 = (0,0.667) y = (1,1) x 0 =1900 = (0,0) y = (1,1) feasible solution y 1 [0,1] y 1 [1.1,3] x 0 = 1283 = (0.816,0.667) y = (1.45,1) y 2 [0,1] y 2 [1.1,3] x 0 =1352 = (0,0.816) y = (1,1.45) Total 63 nodes out of 31 31 possible solutions. Get y = (1.1, 2.0) with z 0 = 1394.5 Global optimum is y = (1.126, 1.972) with z 0 = 1394.1
Degree of freedom i Load Bar j Length Elongation Cross-sectional area Displacement Realistic Structural Design Problem Hooke’s law Equilibrium Compatibility Elongation bounds Displacement bounds
Solution as MI(N)LP Ghattas and Grossmann The disjunctive formulation is Since everything is linear, the big-M and convex hull formulations are linear. Can solve as a mixed integer linear programming (MILP) problem. Discrete sizes for bar j
The convex hull MILP model is
Solution with convex quasi-relaxations Bollapragada, Ghattas and Hooker Check that the problem has the right form: convex Concave (linear) in y j and homogeneous in v j , f j
The quasi-relaxation is
10-bar cantilever truss 25-bar electrical transmission tower Some problem instances
72-bar building Use symmetries to help solve problem