CPLEX is 5x faster on MISOCP. Learn how! Presentation at EURO 2015: - Beyond MIP with CPLEX - Solving an MISOCP - CPLEX is 5x faster - Other improvements

1. © 2015 IBM Corporation
Recent Developments in
IBM CPLEX Optimizer
Xavier Nodet, Program Manager, CPLEX Optimization Studio
EURO 2015, Glasgow, 2015-07-13

2. © 2015 IBM Corporation2
12.6.2, on convex MIQCP

3. © 2015 IBM Corporation3
Agenda
 CPLEX Optimization Studio 12.6.2
 Beyond Linear Programming
 Problem types in CPLEX
 (MI)SOCP
 Algorithms for MISOCP
 SOCP branch-and-bound
 Outer Approximation
 OA branch-and-cut
 Recent progress
 Propagation of quadratic objective/constraints (12.6.1)
 Cone disaggregation (12.6.2)
 Lift-and-Project cuts for MISOCP (12.6.2)
 Other improvements in 12.6.2

4. © 2015 IBM Corporation4
CPLEX Optimization Studio 12.6.2
 Available since June 2015
 The OPL IDE can use the DOcloud service
 Performance improvements for CPO
 Performance improvements for CPLEX

5. © 2015 IBM Corporation5
BEYOND LINEAR
PROGRAMMING

6. © 2015 IBM Corporation6
Everything CPLEX can handle
Convex Nonconvex
All variables
continuous
LP
Convex QP
Convex QCP
Nonconvex QP
Some or all
variables
Integer
MIP
Convex MIQP
Convex MIQCP
Nonconvex MIQP
Quadratic objective
Quadratic constraints
Main focus for today

7. © 2015 IBM Corporation7
Beyond Linear Programming
 CPLEX 9.0 (2003): convex Quadratically Constrained Problems
 All such problems transformed
using Second-Order Cones
 
 
 00,0
00
1010
2
2
0
2
0
1
2







cxxxxxc
cxxxc
psdQbxaQxx
n
j
jj
n
j
jj
TT
 






 

0,:R,: 0
2
0
1
21d
0 xxxxxL
d
i
i
d
0,0  QxxxpsdQ T

8. © 2015 IBM Corporation8
Convex programs
 Continuous relaxation is convex
 Local optimum is also a global optimum
 (Relatively) easy to solve, using Interior Point methods (aka Barrier algorithm)

9. © 2015 IBM Corporation9
Examples of SOCPs
From Lobo, Vandenberghe, Boyd, Lebret (1998)
 Portfolio optimization with loss-risk constraints
 Minimize out-of-beam sensitivity of an array of antennas
 Gentlest grasp of a solid by a robot hand
 Stiffest truss subject to weight constraints

10. © 2015 IBM Corporation10
Examples of MISOCPs
From Benson, Saglam (2005):
 Portfolio optimization, with transaction costs, conditional value-at-risk constraints,
diversification-by-sector constraints, buy-in-thresholds
 Minimizing delays in communication networks
 Coordinated Multi-point Transmission in Cellular Networks.
 Clustering around balls (Euclidian k-center)
 Power distribution systems
 Facility location and inventory management, with random demands, and possibly inventory
capacities and costs
 Minimizing fuel consumption of vessels approaching berths, with fuel consumption
approximated with hyperbolic inequalities

11. © 2015 IBM Corporation11
ALGORITHMS FOR MISOCP

12. © 2015 IBM Corporation12
Algorithms for MISOCP:
SOCP branch-and-bound
 Relax integrality constraints
 Branch on fractional variables
 Very similar to MILP
 But no good warm starting of SOCP solves,
 And reusing MILP techniques, such as cuts
and strong branching is difficult.
kxi  1 kxi
socp
socp socp

13. © 2015 IBM Corporation13
Algorithms for MISOCP:
Outer Approximation
 Let’s build an MILP relaxation of the MISOCP

14. © 2015 IBM Corporation14
Algorithms for MISOCP:
Outer Approximation
 Let’s build an MILP relaxation of the MISOCP:
 Start from optimal solution of (continuous) SOCP: x0

15. © 2015 IBM Corporation15
Algorithms for MISOCP:
Outer Approximation
 Let’s build an MILP relaxation of the MISOCP:
 Start from optimal solution of (continuous) SOCP: x0
 Add linear outer approximation of the nonlinear constraints

16. © 2015 IBM Corporation16
Algorithms for MISOCP:
Outer Approximation
 Let’s build an MILP relaxation of the MISOCP:
 Start from optimal solution of (continuous) SOCP: x0
 Add linear outer approximation of the nonlinear constraints
 Compute MIP solution: y0

17. © 2015 IBM Corporation17
Algorithms for MISOCP:
Outer Approximation
 Let’s build an MILP relaxation of the MISOCP:
 Start from optimal solution of (continuous) SOCP: x0
 Add linear outer approximation of the nonlinear constraints
 Compute MIP solution: y0
 In the SOCP, fix all integer variables to their values at y0

18. © 2015 IBM Corporation18
Algorithms for MISOCP:
Outer Approximation
 Let’s build an MILP relaxation of the MISOCP:
 Start from optimal solution of (continuous) SOCP: x0
 Add linear outer approximation of the nonlinear constraints
 Compute MIP solution: y0
 In the SOCP, fix all integer variables to their values at y0
 Solve the fixed SOCP: x1

19. © 2015 IBM Corporation19
Algorithms for MISOCP:
Outer Approximation
 Let’s build an MILP relaxation of the MISOCP:
 Start from optimal solution of (continuous) SOCP: x0
 Add linear outer approximation of the nonlinear constraints
 Compute MIP solution: y0
 In the SOCP, fix all integer variables to their values at y0
 Solve the fixed SOCP: x1
 Add to the MIP approximations computed from x1

20. © 2015 IBM Corporation20
Algorithms for MISOCP:
Outer Approximation
 Let’s build an MILP relaxation of the MISOCP:
 Start from optimal solution of (continuous) SOCP: x0
 Add linear outer approximation of the nonlinear constraints
 Compute MIP solution: y0
 In the SOCP, fix all integer variables to their values at y0
 Solve the fixed SOCP: x1
 Add to the MIP approximations computed from x1
 Iterate…

21. © 2015 IBM Corporation21
Algorithms for MISOCP:
Outer Approximation
 Let’s build an MILP relaxation of the MISOCP:
 Start from optimal solution of (continuous) SOCP: x0
 Add linear outer approximation of the nonlinear constraints
 Compute MIP solution: y0
 In the SOCP, fix all integer variables to their values at y0
 Solve the fixed SOCP: x1
 Add to the MIP approximations computed from x1
 Iterate…
 Eventually, this converges
 But we can do better…

22. © 2015 IBM Corporation22
Algorithms for MISOCP:
Outer Approximation Branch & Cut
 Let’s combine OA with a MIP tree

23. © 2015 IBM Corporation23
Algorithms for MISOCP:
Outer Approximation Branch & Cut
 Let’s combine OA with a MIP tree
 Build first round of OA, to get a MIP
mip
socp
oa

24. © 2015 IBM Corporation24
Algorithms for MISOCP:
Outer Approximation Branch & Cut
 Let’s combine OA with a MIP tree
 Build first round of OA, to get a MIP
 Start solving it by Branch & Cut
lp
lp

25. © 2015 IBM Corporation25
Algorithms for MISOCP:
Outer Approximation Branch & Cut
 Let’s combine OA with a MIP tree
 Build first round of OA, to get a MIP
 Start solving it by Branch & Cut
 At each integer feasible node
lp
Integer
feasible

26. © 2015 IBM Corporation26
Algorithms for MISOCP:
Outer Approximation Branch & Cut
 Let’s combine OA with a MIP tree
 Build first round of OA, to get a MIP
 Start solving it by Branch & Cut
 At each integer feasible node:
 Solve the SOCP with all integer variables
fixed to their values at the node
lp
socp Integer
feasible

27. © 2015 IBM Corporation27
Algorithms for MISOCP:
Outer Approximation Branch & Cut
 Let’s combine OA with a MIP tree
 Build first round of OA, to get a MIP
 Start solving it by Branch & Cut
 At each integer feasible node:
 Solve the SOCP with all integer variables
fixed to their values at the node
 Generate linear approximations
lp
socp
oa
Integer
feasible

28. © 2015 IBM Corporation28
Algorithms for MISOCP:
Outer Approximation Branch & Cut
 Let’s combine OA with a MIP tree
 Build first round of OA, to get a MIP
 Start solving it by Branch & Cut
 At each integer feasible node:
 Solve the SOCP with all integer variables
fixed to their values at the node
 Generate linear approximations
 Resolve the LP with these new approximations
lp
socp
oa
Integer
feasible

29. © 2015 IBM Corporation29
Algorithms for MISOCP:
Outer Approximation Branch & Cut
 Let’s combine OA with a MIP tree
 Build first round of OA, to get a MIP
 Start solving it by Branch & Cut
 At each integer feasible node:
 Solve the SOCP with all integer variables
fixed to their values at the node
 Generate linear approximations
 Resolve the LP with these new approximations
 Iterate until the node is not integer feasible
lp
socp
oa

30. © 2015 IBM Corporation30
Algorithms for MISOCP:
Outer Approximation Branch & Cut
 Let’s combine OA with a MIP tree
 Build first round of OA, to get a MIP
 Start solving it by Branch & Cut
 At each integer feasible node:
 Solve the SOCP with all integer variables
fixed to their values at the node
 Generate linear approximations
 Resolve the LP with these new approximations
 Iterate until the node is not integer feasible
 Remarks:
 OA constraints are valid for the whole tree
 A large number of MIP techniques can be applied
 Usually the fastest algorithm in CPLEX
lp
socp
oa

31. © 2015 IBM Corporation31
Choosing the MISOCP algorithm
 CPLEX has a heuristic to choose the algorithm
 User can control the choice with CPXPARAM_MIP_Strategy_MIQCPStrat
 0: let CPLEX choose (default)
 1: SOCP branch and bound
 2: OA branch and cut

32. © 2015 IBM Corporation32
RECENT PROGRESS

33. © 2015 IBM Corporation33
Propagation of Quadratic constraints and objective (12.6.1)
 Isolate a single variable x, with bounds L and U, in a quadratic constraint.
 This gives
02
 cbxax

34. © 2015 IBM Corporation34
Propagation of Quadratic constraints and objective (12.6.1)
 Isolate a single variable x, with bounds L and U, in a quadratic constraint.
 This gives
 Compute the two roots of this polynomial, R- and R+, with R- < R+
 If a > 0, then R- and R+ are valid bounds for the variable x, possibly improving L and U
02
 cbxax

35. © 2015 IBM Corporation35
Propagation of Quadratic constraints and objective (12.6.1)
 Isolate a single variable x, with bounds L and U, in a quadratic constraint.
 This gives
 Compute the two roots of this polynomial, R- and R+, with R- < R+
 If a > 0, then R- and R+ are valid bounds for the variable x, possibly improving L and U
 If a < 0, then:
 If R- < L, R+ is a lower bound
 If R+ > U, R- is an upper bound
 If both, then the problem is infeasible (as R- < R+)
02
 cbxax

36. © 2015 IBM Corporation36
Propagation of Quadratic constraints and objective (12.6.1)
 Isolate a single variable x, with bounds L and U, in a quadratic constraint or objective.
 This gives
 Compute the two roots of this polynomial, R- and R+, with R- < R+
 If a > 0, then R- and R+ are valid bounds for the variable x, possibly improving L and U
 If a < 0, then:
 If R- < L, R+ is a lower bound
 If R+ > U, R- is an upper bound
 If both, then the problem is infeasible (as R- < R+)
 Gave 12-20% speedup on convex MIQCPs (57% of models affected).
02
 cbxax

37. © 2015 IBM Corporation37
Cone disaggregation (12.6.2)
 We implemented one idea from Vielma, Dunning, Huchette, Lubin (2015)
 Consider a cone
 Divide both sides of the constraint by
 Introduce linearization variables
 And linking constraints
 More variables and constraints, but shorter cones
 Less OA constraints needed
 Automatically applied during presolve when using OA branch and cut
 






 

0,:R,: 0
2
0
1
21d
0 xxxxxL
d
i
i
d
djy
x
djxyx
xy
j
jj
d
j
j
,,1,0
,0
,,1,
,
0
0
2
0
1






0x
jy

38. © 2015 IBM Corporation38
Cuts for MIPs
 Cuts are an essential component of MIP
 Root cut loop:
 Solve the LP, giving optimal solution
 Try to find a cut
 Add it to the LP
xˆ

39. © 2015 IBM Corporation39
Cuts for MIPs
 Cuts are an essential component of MIP
 Root cut loop:
 Solve the LP, giving optimal solution
 Try to find a cut
 Add it to the LP
 Lift-and-Project cuts (Balas et al., 1993)
 Choose a split disjunction
on a fractional variable
 Generate and solve a large
Cut Generating LP
xˆ xˆ

40. © 2015 IBM Corporation40
Cuts for MIPs
 Cuts are an essential component of MIP
 Root cut loop:
 Solve the LP, giving optimal solution
 Try to find a cut
 Add it to the LP
 Lift-and-Project cuts (Balas et al., 1993)
 Choose a split disjunction
on a fractional variable
 Generate and solve a large
Cut Generating LP
 Either this LP has a solution -> cut
xˆ xˆ

41. © 2015 IBM Corporation41
Cuts for MIPs
 Cuts are an essential component of MIP
 Root cut loop:
 Solve the LP, giving optimal solution
 Try to find a cut
 Add it to the LP
 Lift-and-Project cuts (Balas et al., 1993)
 Choose a split disjunction
on a fractional variable
 Generate and solve a large
Cut Generating LP
 Either this LP has a solution -> cut
 Or you have a pair
such that
xˆ
 10
, xx
 10
,ˆ xxx
0
x 1
xxˆ

42. © 2015 IBM Corporation42
 MIP cuts can be applied to OA from an MISOCP, but…
 How can we use more directly the nonlinear constraints to generate the cuts?
Lift-and-Project cuts for MISOCP (12.6.2)

43. © 2015 IBM Corporation43
 MIP cuts can be applied to OA from an MISOCP, but…
 How can we use more directly the nonlinear constraints to generate the cuts?
 Penalize the nonlinear constraints (Bonami, 2011)
Lift-and-Project cuts for MISOCP (12.6.2)

44. © 2015 IBM Corporation44
 MIP cuts can be applied to OA from an MISOCP, but…
 How can we use more directly the nonlinear constraints to generate the cuts?
 Penalize the nonlinear constraints (Bonami, 2011)
 From , derive OA
 This gives a MIP
Lift-and-Project cuts for MISOCP (12.6.2)
xˆ

45. © 2015 IBM Corporation45
 MIP cuts can be applied to OA from an MISOCP, but…
 How can we use more directly the nonlinear constraints to generate the cuts?
 Penalize the nonlinear constraints (Bonami, 2011)
 From , derive OA
 This gives a MIP
 Use the smaller MLP (Bonami, 2012) to derive
Lift-and-Project cuts for MISOCP (12.6.2)
xˆ
 10
, xx

46. © 2015 IBM Corporation46
 MIP cuts can be applied to OA from an MISOCP, but…
 How can we use more directly the nonlinear constraints to generate the cuts?
 Penalize the nonlinear constraints (Bonami, 2011)
 From , derive OA
 This gives a MIP
 Use the smaller MLP (Bonami, 2012) to derive
 Build OA from those points (Kilinc et al, 2011)
Lift-and-Project cuts for MISOCP (12.6.2)
xˆ
 10
, xx

47. © 2015 IBM Corporation47
 MIP cuts can be applied to OA from an MISOCP, but…
 How can we use more directly the nonlinear constraints to generate the cuts?
 Penalize the nonlinear constraints (Bonami, 2011)
 From , derive OA
 This gives a MIP
 Use the smaller MLP (Bonami, 2012) to derive
 Build OA from those points (Kilinc et al, 2011)
 Only done in OA based algorithm
 Use CPXPARAM_MIP_Cuts_LiftProj if needed
Lift-and-Project cuts for MISOCP (12.6.2)
xˆ
 10
, xx

48. © 2015 IBM Corporation48
Performance gains on convex MIQCP

49. © 2015 IBM Corporation49
Other improvements in 12.6.2
 Improvements for large SOCP models
 Symmetries in LPs
 CPU binding: when not using all the cores
 Objective offsets, adding or removing a constant

51. © 2015 IBM Corporation51
Legal Disclaimer
• © IBM Corporation 2015. All Rights Reserved.
• The information contained in this publication is provided for informational purposes only. While efforts were made to verify
the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any
kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are
subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or
otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall
have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms
and conditions of the applicable license agreement governing the use of IBM software.
• References in this presentation to IBM products, programs, or services do not imply that they will be available in all
countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at
any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment
to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the
effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other
results.
• Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment.
The actual throughput or performance that any user will experience will vary depending upon many factors, including
considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage
configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve
results similar to those stated here.
• IBM, the IBM logo, CPLEX Optimizer, CPLEX Optimization Studio are trademarks of International Business Machines
Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States,
other countries, or both.
• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that
you use as follows; otherwise delete:
Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks
of Intel Corporation or its subsidiaries in the United States and other countries.