Practical Guidelines for Solving Difficult Mixed Integer Programs

© 2012 IBM Corporation
®
IBM ILOG Optimization Virtual Users Group
Session
August 28, 2013
Practical Guidelines for Solving Difficult
Mixed Integer Programs

2
IBM Software Group
Welcome to our August 2013 Virtual Users’ Group
Welcome back to Past Attendees and Welcome to New Clients!
– Please continue to check out our blog for new information regarding these sessions as
well as other interesting articles
https://www.ibm.com/developerworks/mydeveloperworks/blogs/sca/?lang=en_us
Please write your questions to the Q&A Group via SmartCloud Meeting
– As we are expecting a lot of participants in today’s session, please send your questions
via SmartCloud Meeting Chat:
• This is the conversation balloon tab in the right hand panel of the meeting center
• Address the question to the Q&A Group option in the drop down box
– If you have further questions after the meeting please contact Kitte Knight at
kknight@us.ibm.com
Our goal is to present topics of common interest in Optimization and create an open
forum for our clients to exchange information between us and other clients
If you should encounter any problems during the web conference please contact
LOTUS LIVE HELP NUMBER: 1-888-376-0105
2

3
IBM Software Group
Host: Aimee-Emery Ortiz
Product Marketing Manager –
Optimization Solutions
IBM Software Group
Speaker: Ed Klotz
Mathematical Programming
Specialist, IBM ILOG CPLEX, IBM
Software Group
@ibmoptimization

®
Practical Guidelines for Solving Difficult
Mixed Integer Programs
Ed Klotz, Ph.D.
IBM
klotz@us.ibm.com

5
IBM Software Group
Objective
CPLEX solves most MIPs to optimality (including many
considered unsolvable only a few years ago) effectively
with default settings
– With recent advances in MIP heuristics, CPLEX finds good
solutions to many of the MIPs it cannot solve to optimality
For the remaining difficult MIPs, parameter tuning may
suffice to solve them

6
IBM Software Group
Objective
If not, finding good or optimal solutions may be a time
consuming process
–– Finding good cuts that strengthen the formulationFinding good cuts that strengthen the formulation
• Simple, straightforward guidelines help derive these cuts or
determine which other approaches are suitable
– Column Generation or other decomposition methods
– Basis Reduction
– Other Reformulations
– Constraint Programming

7
IBM Software Group
Overview
Objective
Brief Review of Branch and Bound
Identifying Weak Formulations
Tuning CPLEX Parameters
Guidelines for Tightening Difficult MIPs
Example
Methods for Adding Cuts to a Model
Sanity checking/debugging
Summary

8
IBM Software Group
A mixed integer (quadratic) program (MIP/MIQP) is
an optimization problem of the form
integerallorsome jx
uxl
bAxtoSubject
QxxxcMinimize TT
≤≤
=
+
Review of Branch and Bound

9
IBM Software Group
Root;
v=3.5
x=2.3
Integer y=0.6
z=0.3
v ≤≤≤≤ 3 v > 4
x≤≤≤≤
2
x
>
3
y≤≤≤≤0
y>1
z≤≤≤≤0
z>1
Lower Bound
Integer
Upper Bound
Infeas
z=0.1
G
A
P
Fathomed
Branch and Bound for MIP
z≤≤≤≤0
z>
1

10
IBM Software Group
Key aspects of Branch and Bound
– Branching divides the feasible region in a manner that all integer
feasible solutions belong in one of the branches
• Standard B&B uses up and down branch on an integer variable
• But any number of branches on any object is possible
Special ordered sets, indicator constraints, hyperplane branching
– Child node objective (of node MIP or LP) can be no better than the
parent node objective
• Enables pruning
– At any point in the tree, a better solution obtained via branching (as
opposed to node heuristics) can only come from children of the
active node list
• Explains why the lower bound is the active node with minimal relaxation
objective value

11
IBM Software Group
Progress of the algorithm depends on:
– Ability to find integer feasible solutions
• # of integer infeasibilities at each node
– Ability to prune nodes
• Objective value of best integer feasible solution
– Ability to move lower bound
• # of other node relaxations with same objective value
• # of active nodes remaining
– Strength of the model formulation
– Node throughput
• Node relaxation solve times
• Cut, heuristic computation times

12
IBM Software Group
Nodes Cuts/
Node Left Objective IInf Best Integer Best Node ItCnt Gap
...
300 229 22.6667 40 31.0000 22.0000 4433 29.03%
400 309 cutoff 31.0000 22.3333 5196 27.96%
500 387 26.5000 31 31.0000 22.6667 6164 26.88%
...
7800 5260 28.5000 23 31.0000 25.6667 55739 17.20%
7900 5324 28.2500 26 31.0000 25.6667 56424 17.20%
8000 5385 27.3750 30 31.0000 25.7778 57267 16.85%
Optimizer Node Log shows algorithm progress
– Here we have progress in best node but not best integer
Node pruning Feasible solns
Strength /
lower bound
Node
throughput

13
IBM Software Group
Optimizer Node Log shows algorithm progress
– Progress in best integer but not best node:
Nodes Cuts/
Node Left Objective IInf Best Integer Best Node ItCnt Gap
…
300 296 2018.0000 27 3780.0000 560.0000 3703 85.19%
* 300+ 296 0 2626.0000 560.0000 3703 78.67%
* 393 368 0 2590.0000 560.0000 4405 78.38%
400 372 560.0000 291 2590.0000 560.0000 4553 78.38%
500 472 810.0000 175 2590.0000 560.0000 5747 78.38%
...
* 7740+ 5183 0 1710.0000 560.0000 66026 67.25%
7800 5240 1544.0000 110 1710.0000 560.0000 66279 67.25%
7900 5325 944.0000 176 1710.0000 560.0000 66801 67.25%
8000 5424 1468.0000 93 1710.0000 560.0000 67732 67.25%
Node pruning Feasible solns
Strength /
lower bound
Node
throughput

14
IBM Software Group
Identifying Weak Formulations
Convex hull of the integer feasible solutions provides the strongest
formulation
Add valid cuts to strengthen formulation
P1
P2
}:{P2P3
}0,:{P2
})0,:({P1
dCxRx
xbAxRx
xbAxZxconv
n
n
n
≤∈∩=
≥≤∈=
≥≤∈=
P3
22 dxcT
≤
33 dxcT
≤
n)(separatio:22)
(validity)11)
satisfymustCuts
i
T
i
i
T
i
dxcPx
Pxdxc
>∈∃
∈∀≤
11 dxcT
≤

15
IBM Software Group
Identifying Weak Formulations (example)
A difficult knapsack problem
Minimize
COST:
Subject To
c1: 12228 x1 + 36679 x2 + 36682 x3 + 48908 x4
+ 61139 x5 + 73365 x6 = 89716837
Bounds
x1 >= 0
…
x6 >= 0
Generals
x1 x2 x3 x4 x5 x6
End

16
IBM Software Group
LP solutions are trivial to find (even after many branches on the
variables), but MIP solutions may not be.
c1: 12228 x1 + 36679 x2 + 36682 x3 + 48908 x4 +
61139 x5 + 73365 x6 = 89716837
Domain of x1: [0, 89716837/12228]
Domain of x2: [0, (89716837 – 12228 x1)/36679]
…
Domain of x6: [0, (89716837 – (12228 x1 + 36679 x2
+ 36682 x3 + 48908 x4
+ 61139 x5)/73365]

17
IBM Software Group
c1:12228 x1 + 36679 x2 + 36682 x3 +
48908 x4 + 61139 x5 + 73365 x6 = 89716837
MIP:
89716837
73365
61139
48908
36682
36679
12228
36682
12228
48908
61139
73365
36679

18
IBM Software Group
c1:12228 x1 + 36679 x2 + 36682 x3 +
48908 x4 + 61139 x5 + 73365 x6 = 89716837
LP:
89716837
73365
61139
48908
36682
36679
12228
36682
12228
48908
61139
73365
36679

19
IBM Software Group
Tuning CPLEX parameters
Try the automatic Tuning Tool (starting with CPLEX 11)
Use node log to identify whether lack of progress involves best
integer, best node, lack of node throughput, or some
combination of all three.
– Too many parameters for arbitrary experimentation
– Set parameters based on lack of progress
• MIP Emphasis parameter sets multiple other parameters at once
Settings 1 and 4 well suited for lack of progress in the best integer
Settings 2 and 3 well suited for lack of progress in the best node
• In addition, particular parameters are well suited to address different
performance issues
More frequent heuristics, best estimate node selection, less backtracking well
suited for more progress in best integer
More aggressive probing and cuts, strong branching, aggressive symmetry
reductions well suited for more progress in best node
LP performance tuning, turning off heuristics and cuts can help with slow node
throughput

20
IBM Software Group
Copyright © ILOG 2008
Tuning CPLEX Parameters
How do I know if parameter settings won’t get
me the results I need? Should I move on, or is
success just around the corner?
– Use extremely aggressive settings to address lack
of progress. If that doesn’t show progress, time to
look elsewhere for performance improvement
• For lack of progress in the best node, use MIP emphasis 3,
strong branching (variableselect 3), presolve symmetry 5.
Disable heuristics to speed up nodes. Use a MIP start from
a previous run if available. If that doesn’t show progress in
the best node, unlikely that other parameter settings will.
• For lack of progress in the best integer, apply heuristics at
every node, turn on feasibility pump heuristic, increase sub
MIP node limit for heuristics, set backtrack tolerance to 1.

21
IBM Software Group
Identify the parts of the model that make it difficult
– Simplify the model if necessary
• Filtering: remove any constraints and integrality restrictions not
involved in the performance trouble
If you remove a set of constraints or restrictions and the MIP remains
hard, that set is not involved in the performance issue
• Try to reproduce the trouble in a smaller data instance
Determine how fractional solutions in the node relaxations
allow the objective to improve
Use fractional solutions to identify the constraints and
variables that will motivate additional cuts

22
IBM Software Group
Common tactics for deriving additional cuts
– Linear or logical combinations of constraints
• Associate a graph with the model structure
– Disjunctions
– Solve one or more related models
– Use infeasibility
• CPLEX Conflict Refiner
– Use solution objective value
• Feasibility on model with constraint on objective value added
• Especially true for models with soft constraints

23
IBM Software Group
Guidelines: Linear, Logical Combinations
Linear combinations of constraints
}1xi0,132
,131
,121:{ 3
≤≤≤+
≤+
≤+∈= +
xx
xx
xxZxP
.5).5,(.5,likesolutionsfractionaloffCut
1321:downroundy tointegralitUse
2/3321:2bydivides,constrainttheSum
≤++
≤++
xxx
xxx

24
IBM Software Group
But how do we find the constraints to combine in a practical
model?
><
≤+
><
≤+
><
≤+
><
sconstraint8000
132
sconstraint4000
131
sconstraint20000
121
sconstraint10000
xx
xx
xx

25
IBM Software Group
Conflict graph
– Associate a node with each binary variable
– Create an edge between the nodes of any pair of binary variables that
cannot both be 1
– At most one binary in a clique can be 1
– Conflict graph identifies a useful linear combination of constraints
– CPLEX creates the conflict graph and by default finds clique cuts
}1xi0,132
,131
,121:{ 3
≤≤≤+
≤+
≤+∈= +
xx
xx
xxZxP
x1
x3x2 1321:CutClique ≤++ xxx

26
IBM Software Group
Guidelines: Disjunctions
Gomory/MIR cut:
),,2(offcuts;232Combining,
22323
23322
3
}232321:{
}321:{
}10352314:{
5
1
3
1
2
1
4
1
4
1
2
1
4
1
4
1
2
1
2
1
4
1
4
13
2
5
4
5
4
33
3
≥+
≥⇒−≤+−⇒≥
≥⇒≥+−⇒≤
−=
+=+−++∈=
=++∈=
=++∈=
+
+
+
xx
xxxt
xxxt
t
xxxxxZxP
xxxZxP
xxxZxP

27
IBM Software Group
dxa
dxa
dxa
SS
SSxda
j
SSj
j
j
Sj
j
j
Sj
j
jj
≥
≥
≥
=∩
≥
∑
∑
∑
∪∈
∈
∈
21
2
1
Then
or
constraintedisjunctivheConsider t
Suppose
,0,,Given
21
2,1
φ

28
IBM Software Group
)min()max(
Then
or
constraintedisjunctivheConsider t
,,,0Given
2,1
2
1
,21
,
,
ddx
dx
dx
Sddx
j
Sj
jj
j
Sj
j
j
Sj
j
jjj
≥
≥
≥
≥
∑
∑
∑
∈
∈
∈
βα
β
α
βα

29
IBM Software Group
More general disjunctions involve solving LP subproblem to
generate cuts
– CPLEX does disjunctions on x <= k and x >= k+1 for fractional
integer variables at the root node LP
– Can also do disjunctions on arbitrary constraints
• Write a program to solve the LP, calculate the cut
• See “Integer Programming” by Wolsey for the details

30
IBM Software Group
Guidelines: When to derive cuts
CPLEX generates generic cuts, including
– Clique cuts
– Disjunctive cuts based on individual fractional variables
– Gomory cuts
– Zero half cuts
– See manual for complete list
– You don’t need to generate these yourself
CPLEX is unlikely to generate cuts more specific to particular models
– Concepts used for Clique, Disjunctive and Gomory cuts can often be
customized to particular models with more special structure

31
IBM Software Group
Statistics: 559 constraints, 1066 variables (516 binary, 516 general integer)
Node log:
Nodes Cuts/
Node Left Objective IInf Best Int Best Node ItCnt Gap
0 0 101984.7744 28 101984.7744 35
*0+ 0 0 4.10026e+08 101984.7744 35 99.98%
153036.9306 35 4.10026e+08 Cuts: 41 151 99.96%
*0+ 0 0 4.00022e+08 153036.9306 151 99.96%
…
*55950+ 0 1.02822e+07 202475.0432 98.03%
56000 infeasible 1.02822e+07 202518.1842 98.03%
Elapsed time = 186.20 sec. (tree size = 13.36 MB).
Example: Penalty variables

32
IBM Software Group
Node log (ctd):
Nodes Cuts/
Node Left Objective IInf Best Int Best Node Gap
7149e4 7726073 infeas 1.02822e+07 307724.1416 97.01%
7150e4 7727024 309418.1 33 1.02822e+07 307728.0479 97.01%
Elapsed time = 161631.76 sec. (tree size = 9072.93 MB).
Nodefile size = 8945.58 MB (3124.04 MB after compression)
7151e4 7727720 357983.4 22 1.02822e+07 307731.4823 97.01%
…

33
IBM Software Group
Example: Penalty Variables
Review the guidelines and tactics
– Simplify the model if necessary
–– Determine how fractional solutions affect the objectiveDetermine how fractional solutions affect the objective
– Use fractional solutions to motivate additional cuts
– Linear or logical combinations of constraints
– Disjunctions
– Solve one or more related models
– Use infeasibility
– Use solution objective value

34
IBM Software Group
Determine how fractional solutions affect the objective:Determine how fractional solutions affect the objective:
Min obj: 10000000 id134 + 10000000 id135 + ...
+ 10000000 id161 + 10000 id168 + 10000 id169
+ 1000 id170 + 1000 id171 + 34.299999237 id200
+ … + 10000 id2309
id78: id134 - id135 + 3 id200 + 3 id204 + 3 id220
+ 3 id228 + 3 id248 + ... + 3 id2096 + 3 id2144
+ 2 id2148 = 4
(Implied integer by integrality of other variables in the constraint)

35
IBM Software Group
Determine how fractional solutions affect theDetermine how fractional solutions affect the objective(ctdobjective(ctd))::
Nodes Cuts/
Node Left Objective IInf Best Int Best Node Itcnt Gap
…
*55950+ 11356 0 1.02822e+07 202475.0432 861218 98.03%
56000 11367 infeasible 1.02822e+07 202518.1842 862287 98.03%
Elapsed time = 186.20 sec.
Comparing the best integer and best node values, we see that
removing integrality enables solutions with the sum of the
expensive penalty variables << 1. But, we don't know yet whether
an integer solution exists with all such penalty variables set to 0.
Can we answer that question?

36
IBM Software Group
Yes we can, by solving a related problemsolving a related problem:
Add a constraint that sets all the expensive penalty
variables to 0:
conobj: id134 + id135 + id136 + id137 + … + id161 = 0
Results:
MIP - Integer infeasible or unbounded.
Current MIP best bound is infinite.
Solution time = 18.80 sec. Iterations = 409663
Nodes = 38384

37
IBM Software Group
Solve another related problemSolve another related problem, using solution objective valueusing solution objective value:
Nodes Cuts/
…
*55950+ 11356 0 1.02822e+7 202475.0432 861218 98.03%
56000 11367 infeasible 1.02822e+7 202518.1842 862287 98.03%
Elapsed time = 186.20 sec.
conobj: id134 + id135 + id136 + id137 + id138 + … + id161 >= 1
conobj: id134 + id135 + id136 + id137 + id138 + … + id161 = 1

38
IBM Software Group
0 0 1.01576e+07 16 1.01576e+07 54
1.01851e+07 20 Cuts: 42 82
…
*205 160 0 1.02922e+07 1.02098e+07 1233 0.80%
…
58200 319 infeasible 1.02822e+07 1.02806e+07 440316 0.02%
58300 226 cutoff 1.02822e+07 1.02811e+07 440441 0.01%
MIP - Integer optimal, tolerance (0.0001/1e-06):
Objective =1.0282191250e+07
Current MIP best bound = 1.0281164221e+07 (gap = 1027.03, 0.01%)
Solution time = 38.51 sec. Iterations = 440476 Nodes = 58326
(201)
Solve another related problemSolve another related problem, using solution objective valueusing solution objective value:
Nodes Cuts/

39
IBM Software Group
Summary
–– Determine how fractional solutions affect the objectiveDetermine how fractional solutions affect the objective
• Involved both integer feasible solutions and fractional solutions to
relaxations
–– Solve one or more related modelsSolve one or more related models
–– Use infeasibilityUse infeasibility
–– Use solution objective valueUse solution objective value
– More generally, don’t try to combine multiple objectives into a
single objective
• Solve separate problems with each individual objective

40
IBM Software Group
Methods for Adding Cuts to a Model
Explicit Cuts
– More MIP preprocessing to tighten the model
– Larger problem may increase memory, run time
User Cuts at start of optimization
– Added to cut pool rather than problem
– Less memory, node relaxation solve time
– MIP preprocessing not applied to user cuts
Cut Callback
– Use when need to add cuts at child nodes
• Child node information needed to derive cuts
• Too many cuts to add at start of optimization
– More work to implement

41
IBM Software Group
Debugging/Sanity checking
Sufficient (but not necessary) conditions that the cut(s) tightened the MIP
too much
– Adding the cuts makes a feasible MIP infeasible
– Or adding the cuts contradicts the upper or lower bound of the original
MIP
• Use CPLEX’s conflict refiner to obtain minimal set of infeasible
constraints
Lower Bound = 50
Upper Bound = 100
G
A
P
cuts Infeasible
Original MIP Tightened MIP

42
IBM Software Group
Sufficient (but not necessary) conditions that the cut(s) tightened the MIP too
much
– Reversing the direction of the (globally valid) cut and optimizing finds an
integer feasible solution
• Fix variables to solution for the tightened model with the bad cut, run
the conflict refiner on the resulting infeasible model
bad cut

43
IBM Software Group
Sufficient (but not necessary) conditions that the cut(s) didn’t
tighten the MIP enough
– Fractional solutions that we used to identify the cuts are not cut off
in the node LPs of the tightened MIP

44
IBM Software Group
Conclusions
Most MIPs easy to solve (defaults or parameter tuning)
The remaining MIPs can be very time consuming for the
practitioner
I hope the presentation didn’t just illustrate how to improve
performance on the particular example
Use the guidelines to help quickly identify the good cuts on your
own difficult MIPs
Sometimes a sharp cut found in an hour’s work is better than
the sharpest cut found in a week’s (or more) work

45
IBM Software Group
Additional Reading
Wolsey, L. Integer Programming.
Polya, How to Solve It.
Bixby, Fenelon, Gu, Rothberg, Wunderling. Mixed Integer Programming: A
Progress Report. Available in Grotschel, M.(editor), The Sharpest Cut.
Klotz, Newman. Practical Guidelines for Solving Difficult Mixed Integer Programs
http://www.sciencedirect.com/science/article/pii/S1876735413000020 or
http://inside.mines.edu/~anewman/MIP_practice120212.pdf
(LP Performance Tuning): Klotz, Newman. Practical Guidelines for Solving
Difficult Linear Programs
http://www.sciencedirect.com/science/article/pii/S1876735412000189
CPLEX MIP parameter settings:
http://www-01.ibm.com/support/docview.wss?uid=swg21400023

46
IBM Software Group
Q&A Reminder
Please write your questions to the Q&A Group via SmartCloud Meeting
– As we are expecting a lot of participants in today’s session, please send your
questions via SmartCloud Meeting Chat:
• This is the conversation balloon tab in the right hand panel of the meeting
center
• Address the question to the Q&A Group option in the drop down box
– If you have further questions or sales enquiries after the meeting please
contact Kitte Knight at kknight@us.ibm.com
46

Practical Guidelines for Solving Difficult Mixed Integer Programs

Recommended

Recommended

More Related Content

What's hot

What's hot (9)

Similar to Practical Guidelines for Solving Difficult Mixed Integer Programs

Similar to Practical Guidelines for Solving Difficult Mixed Integer Programs (20)

Recently uploaded

Recently uploaded (20)

Practical Guidelines for Solving Difficult Mixed Integer Programs