Innovations in CPLEX performance and solver capabilities

Decision Optimization
INFORMS 2013 Workshop - Innovations
INFORMS Fall Conference 2013
Building a Smarter Planet
with

© 2013 IBM Corporation
2 INFORMS 2013 Workshop - Innovations
Workshop presentations
 3:00 - 3:30: Innovations in CPLEX performance and solver capabilities –
Mary Fenelon
 3:30 – 4:15: Expert tips and tricks for the OR practitioner: Everything you
need to know about CPLEX-powered modeling and solving – Ed Klotz
 4:30 - 5:10 From "Good Modeling" to developing and deploying great
enterprise applications – Yianni Gamvros, Yana Ageeva, John Chaves
 5:10 – 5:30 CPLEX-based tools and advanced analytics for Academia –
Dennis Bly

INFORMS 2013 Workshop - Innovations
INFORMS Fall Conference 2013
Innovations in CPLEX
performance and solver
capabilities
Mary Fenelon
Development Manager
IBM ILOG CPLEX Optimization Studio
mfenelon@us.ibm.com

Disclaimer
IBM’s statements regarding its plans, directions, and intent are subject to
change or withdrawal without notice at IBM’s sole discretion. Information
regarding potential future products is intended to outline our general product
direction and it should not be relied on in making a purchasing decision. The
information mentioned regarding potential future products is not a
commitment, promise, or legal obligation to deliver any material, code or
functionality. Information about potential future products may not be
incorporated into any contract. The development, release, and timing of any
future features or functionality described for our products remains at our sole
discretion.
Performance is based on measurements and projections using standard IBM®
benchmarks in a controlled environment. The actual throughput or
performance that any user will experience will vary depending upon many
factors, including considerations such as the amount of multiprogramming in
the user's job stream, the I/O configuration, the storage configuration, and the
workload processed. Therefore, no assurance can be given that an individual
user will achieve results similar to those stated here.

Agenda
 New features in ILOG CPLEX Optimization Studio
 Distributed computing options for CPLEX Optimizer
 Performance

Agenda
 Performance

Model development Tools
Optimization Engines
Math Programming
CPLEX Optimizers
(Simplex, Barrier, Mixed
Integer)
Constraint Programming
Constraint-based
scheduling
CPLEX CP Optimizers
CPLEX Studio (IDE) - OPL Modeling Language
ILOG Concert Technology (C++, .NET, Java) Java client only
CPLEX Optimization Studio
CPLEX
Enterprise
Server
Connectors
•Microsoft Excel
•MATLAB
•Pytthon
•AMPL
Tools & APIs
•CPLEX Interactive
•C Callable Library

New features
 CPLEX 12.5.1 – June 2013
– Remote Object, a distributed algorithm toolkit for Java and C++
– Additional OPL scripting methods and APIs
 CPLEX 12.6 – December 2013
– Additional OPL scripting methods
– Improved model editor
– LP file viewer
– New constraints in CP Optimizer
– CP Optimizer model dump facility
– Global solution to nonconvex QP and MIQP
– Distributed MIP algorithm

Additional OPL scripting methods and API
 12.5.1 APIs
– Relaxation iterator IloOplRelaxationIterator
– Conflict iterator IloOplConflictIterator
 12.5.1 scripting
– Multiple MIP starts
 12.6 scripting
– Lazy constraints and user cuts
– Solution pools
– Priorities
– Problem queries

Improved model editor
 Existing features
– Scope selection
– Syntax coloring
– Search/replace
– Syntax errors marked at run time
– Etc.
 New syntax-aware editing enables
– Highlighting occurrences
– Renaming per entity type
– Refactoring and formatting
– Search/find per entity type
– Auto-completion
– Expand/collapse blocks
– Syntax errors marked while editing

– Search/replace per entity type
– Auto-completion

– Auto-completion

LP file viewer

New constraints in CP Optimizer
 Strong constraint
 SameSequence constraint

 Benefits
– Reinforces the model propagation between variables where the user knows
that the domain reduction is weak
– Avoids the manual enumeration of tuple sets for creating
allowedAssignments
– Creates stronger allowedAssignments than those obtained by hand
 Processing
– Enumerate partial solutions by performing a search on the variables of each
strong constraint, but using domain reduction of the whole model
– Add an allowedAssignments constraint over the variables specified in the
strong constraint, with a tuple set corresponding to the partial solutions
found in the previous search
– Remove constraints which are no longer needed given that the strong
constraint is now present in the model
Strong(x1, x2 … xn)

IloIntTupleSet gha(env, 3);
IloIntArray tuple(env, 3);
for (IloInt i = 0; i < n; i++) {
tuple[0] = i;
for (IloInt j = 0; j < n; j++) {
if (i != j) {
tuple[1] = j;
tuple[2] = Game(i, j, n);
gha.add(tuple);
}
}
}
for (IloInt i = 0; i < nbWeeks; i++) {
for (IloInt j = 0; j < nbGamesPerWeek; j++) {
IloIntVarArray vars(env);
vars.add(home[i][j]);
vars.add(away[i][j]);
vars.add(games[i][j]);
model.add(IloAllowedAssignments(env, vars, gha));
}
}
! Best Branches Non-fixed
* 56 2688 0.53s
* 52 7402 0.99s
* 48 19418 2.38s
* 46 102k 11.87s
* 44 109k 12.81s
* 42 195k 20.22s
* 40 218k 22.43s
* 38 239k 24.47s
for (IloInt i = 0; i < nbWeeks; i++) {
for (IloInt j = 0; j < nbGamesPerWeek; j++) {
model.add(home[i][j] != away[i][j]);
model.add(games[i][j] == Game(home[i][j], away[i][j], n));
IloIntVarArray vars(env);
vars.add(home[i][j]);
vars.add(away[i][j]);
vars.add(games[i][j]);
model.add(IloStrong(env, vars));
}
}
! Best Branches Non-fixed W Branch decision
* 50 520 0.44s 6 -
* 46 2468 0.65s 1 -
* 38 4304 0.88s 1 -
* 36 25689 2.98s 1 -
* 34 86973 8.15s 1 -
The original tuple set has 90 tuples, the generated ones have
between 5 and 66 tuples.
Using the strong constraint in sports.cpp

 CP Optimizer has the concept of an interval sequence. In OPL:
dvar sequence seq1[r in Res] in all(t in tasksOnRes[r]) a[t];
 The sameSequence constraint constrains two different sequences to have the same
order of corresponding intervals
sameSequence(seq1, seq2)
 The correspondence can be specified explicitly or be implicit from the order of intervals
given in the definition of the sequences
 The sameSequence constraint can naturally model constraints like no-bypass whereby
products must be processed by different machines in the same order (although that
order is free at the outset
SameSequence (seq1, seq2)
M
1
M
2
Conveyor belt
seq1 seq2
sameSequence(seq1, seq2)

dvar interval itvs[j in Jobs][m in Mchs] size
OpDurations[j][m];
dvar sequence mchs[m in Mchs] in all(j in Jobs) itvs[j][m]
types all(j in Jobs) j;
minimize max(j in Jobs) endOf(itvs[j][nbMchs-1]);
subject to {
forall (m in Mchs)
noOverlap(mchs[m]);
forall (j in Jobs, o in 0..nbMchs-2)
endBeforeStart(itvs[j][o], itvs[j][o+1]);
forall (m in Mchs, j in Jobs: 0<m)
typeOfPrev(mchs[m],itvs[j][m],nbJobs)
== typeOfPrev(mchs[0],itvs[j][0],nbJobs);
}
Using sameSequence in sched_flowshop.mod
dvar interval itvs[j in Jobs][m in Mchs] size OpDurations[j][m];
dvar sequence mchs[m in Mchs] in all(j in Jobs) itvs[j][m];
minimize max(j in Jobs) endOf(itvs[j][nbMchs-1]);
subject to {
forall (j in Jobs, o in 0..nbMchs-2)
endBeforeStart(itvs[j][o], itvs[j][o+1]);
forall (m in Mchs)
noOverlap(mchs[m]);
forall (m in Mchs: 0<m)
sameSequence(mchs[0], mchs[m]);
}

CP Optimizer model dump facility
 Model instance can be written to a file
– Constraints
– Variables
– Search phases
– Parameter settings
 Easier way to communicate instance to IBM Support
 Aids in debugging formulation
 Parameter to anonymize names

CPLEX Optimizer parameter hierarchy
 Provide consistent naming across APIs
 Traditional names will be maintained
 Easier to find relevant parameters
 Based on hierarchy used in the CPLEX Interactive Optimizer

CPLEX Optimizer parameter hierarchy
API Traditional Hierarchical
Callable
Library
CPX_PARAM_FRACCUTS CPXPARAM_MIP_Cuts_Gomory
Java IloCplex.IntParam.FracCuts IloCplex.Param.MIP.Cuts.Gomory
.NET Cplex.IntParam.FracCuts Cplex.Param.MIP.Cuts.Gomory
C++ IloCplex::FracCuts IloCplex::Param::MIP::Cuts::Gomory
Python cplex.parameters.mip.cuts.gomory cplex.parameters.mip.cuts.gomory
MATLAB cplex.Param.mip.cuts.gomory cplex.Param.mip.cuts.gomory
Example

Global solution to nonconvex QP and MIQP
 When Q is positive-semidefinite, the objective function is convex
 When Q is indefinite, the objective function is nonconvex
min ½ xTQx + cTx
s.t. Ax = b
u ≤ x ≤ l
optionally some x integral
Quadratic program (QP) and
mixed integer quadratic program (MIQP)

 Solution to convex QP and MIQP in CPLEX since v. 4.0 and v. 8.0, resp.
 Solution to first order optimality for nonconvex QP since v. 12.3
– Primal-dual interior point algorithm
– Requires linear algebra routines for indefinite factorization
– CPLEX wrote own parallel factorization and solution routines
 Solution to global optimality for nonconvex QP and MIQP in v. 12.6

 Global solution requires branching on continuous variables
– QP solved as MIQP, even if no integer variables
– NP-complete, can be much more time-consuming than solving to local optimum
 Solution Algorithm
– Spatial branch-and-cut over convex QP relaxation
• Branch on continuous variables
• Tighten linear relaxation in local nodes by locally valid cuts
– Automatic selection of linearization of non-convex part in Q
• McCormick relaxation
• Eigenvector reformulation based on indefinite factorization
– Solve the indefinite problem when an incumbent is found

Agenda
 Performance

Distributed computing*
 Core speed vs. more cores
 Memory bandwidth on a single system limited by single cache and memory bus
 High performance servers
 Clusters of commodity hardware
 Cloud
*Koch, Shinano and Ralphs. What Could a Million Cores Do To Solve Integer Programs? 2012.

Distributed computing
 Client-server
– CPLEX Enterprise Server or ILOG ODM Enterprise Optimization Server
– CPLEX Remote Object
 Distributed algorithms
– Build your own with the CPLEX Remote Object
– For MIP, use the CPLEX distributed MIP algorithm, new in v. 12.6

Distributed computing: Client-server
 Optimization Server in ODM Enterprise or CPLEX Enterprise Server
– Built to enterprise-class standards
– WebSphere Application Server
• Queuing, failover, etc.
• Administration
– Oracle, DB2 or Microsoft SQL database to store results
– J2EE compliant
– Asynchronous, e.g., client can be disconnected
 CPLEX Enterprise Server
– From IDE
• IDE panel to submit/monitor/control jobs
• Results are returned to IDE
– From Java APIs
– OPL .mod and .dat files are sent to server
• Data read on server, not client
• Data can be in .dat file or from a database, flat files, etc.
 ODM Enterprise
– With the Java API for CPLEX, CP Optimizer or OPL via Optimization Server custom task

Distributed computing: CPLEX Remote Object
 Transparent use of remote hardware
 Change single line of code to describe
connection
 All methods are mapped to messages to
remote system
 Transport abstraction hides the details of
using these protocols
– ssh
– TCP/IP
– MPI
– Local
optimize(Data data)
{
IloEnv env;
IloCplex cplex(env, transport
argc, argv);
buildModel(cplex, data);
cplex.solve();
useSolution(cplex);
}
optimize(Data data)
{
IloEnv env;
IloCplex cplex(env);
cplex.solve();
useSolution(cplex);
}

Distributed computing: CPLEX Remote Object
 Synchronous or asynchronous
 Asynchronous calls support a
master/worker distributed parallel
programming paradigm for building
distributed parallel optimization algorithms
 Asynchronous calls return AsyncHandle
objects to join back
 Examples provided:
– iloparbenders.cpp
– iloparmipopt.cpp
– ParBenders.java
– parbenders.c
– parmipopt.c
optimize(Data data)
{
IloEnv env;
AsyncHandle handle;
argc, argv);
handle = cplex.solve();
doOtherThings();
handle.join();
useSolution(cplex);
}
optimize(Data data)
{
IloEnv env;
argc, argv);
cplex.solve();
useSolution(cplex);
}

Distributed computing: CPLEX distributed MIP
 Built on the CPLEX Remote Object technology
 A “master” distributes work to multiple “workers”
 Two phases
– Racing ramp-up*
– Each machine uses different settings, and a winner is selected
• Only incumbent objective values and best bounds are communicated
• Infinite ramp-up allowed (concurrent MIP)
– Distributed tree
• Nodes of the tree created by the winner distributed to workers
• Nodes exchanged at synchronization points
– Deterministic
 Available in the CPLEX Interactive, and the C, C++, Java, Matlab and Python APIs
*Shinano, Achterberg, Berthold, Heinz, Koch. ParaSCIP – a parallel extension of SCIP. 2010.

 Does hardware need to be identical?
– No, but workers should be similar in performance to get best overall performance
– Master does not have to be similar to workers
 When should I use infinite ramp-up?
– To exploit performance variability
– In order to achieve a 1.5x to 2x speed-up for problems which aren’t too trivial
 When should I use distributed search?
– Large number of nodes to enumerate
– Very hard problems
 Do I have to have exclusive access to the machine?
– No, but for best overall performance, the worker should have little load
 Does it work on HPC systems?
– Yes. MPI is the de facto standard for HPC and is one of the supported transport
protocols
– Experiments have been done on an IBM Blue Gene Q with 256 workers
FAQs

 Why would I want determinism?
– The same solution will be reached each time so the downstream
processing will see consistent results
– For tuning and debugging of the application
 Which MPI distributions are supported?
– OpenMPI and MPICH, depending on the platform
 What shared memory parallel MIP features are supported?
– MIPstarts
– Informational callback on the master
– Almost all parameters, e.g.
• Gap
• Time limits
– Terminate function
FAQs

 What shared memory parallel MIP features are not supported?
– Control callbacks
– Restarts
– Global solution to nonconvex MIQP and QP
 What do I need to do to set up a distributed run?
– CPLEX Interactive on each machine with appropriate shared libraries
– Describe the machines in a VMC file or with function arguments
– Other steps depend on the transport
 When does ramp-up stop?
– Controlled via parameters
– “Automatic” value balances number of open nodes and solution quality
FAQs

Agenda
 Performance

Performance – CP Optimizer 12.6
 Contains a new scheduling algorithm
 Increases search robustness on medium-sized problems
 Provides more optimality proofs on smaller instances
 The new search is activated by default
 Also some performance increase on integer problems
Version-to-version average solution time ratios
12.3/12.2 12.4/12.3 12.5/12.4 12.5.1/12.5 12.6/12.5.1
Scheduling
1 worker
1.06 1.01 1.29 1.03 1.69
Scheduling
4 workers
1.22 0.97 1.35 1.03 1.63
Integer prob.
1 worker
1.10 0.99 0.97 1.05 1.10
Integer prob.
4 workers
1.06 1.01 1.01 1.03 1.08

Performance – CPLEX Optimizer 12.6
 Average speedup of MIP optimizer on MILP v. 12.6 compared to v. 12.5.1
– 15% on hard MILP models (more than 100 sec.)
– 5% on non-trivial MILP problems (more than 1 sec.)

0
200
400
600
800
1000
1200
1400
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
numberoftimeouts
0
50
100
150
200
250
totalspeedup
 10 sec
 100 sec
 1000 sec
Date: 28 September 2013
Testset: 3147 models (1792 in  10sec, 1554 in  100sec, 1384 in  1000sec)
Machine: Intel X5650 @ 2.67GHz, 24 GB RAM, 12 threads (deterministic since CPLEX 11.0)
Timelimit: 10,000 sec
v6.0
v6.5.3
v7.0
v8.0
v9.0
v10.0
v11.0 v12.1
v12.2
v12.4
v12.5
v12.6
CPLEX MILP performance

Technical Sessions
 Analyzing 12 Years of Progress in CPLEX Sunday, 16:30 - 18:00 Roland Wunderling
 Recent Developments in CPLEX Monday, 08:00 - 09:30 Tobias Achterberg
 Non-convex Quadratic Programming in CPLEX Tuesday, 11:00 - 12:30 Christian Bliek and
Pierre Bonami
 Tutorial: Performance Variability in Mixed-integer Programming Tuesday, 8:00 – 9:30
Andrea Lodi and Andrea Tramontani
 Software Demonstration: Expert Tips and Tricks for Using CPLEX Optimization
Studio…for Distributed Computing Tuesday Oct 08, 16:30 - 17:15
 Lift-and-Project Cuts in CPLEX 12.5.1 Wednesday, 13:30 - 15:00 Andrea Tramontani
 Concurrent root cut loops to exploit random performance variability Wednesday, 15:30
- 17:00 A. Tramontani, M. Fischetti, A. Lodi, D. Salvignin, M. Monaci
 Interesting Use Cases for the CPLEX Remote Object Wednesday, 15:30 - 17:00 Laszlo
Ladanyi and Daniel Junglas

Innovations in CPLEX performance and solver capabilities

More Related Content

What's hot

Viewers also liked

Similar to Innovations in CPLEX performance and solver capabilities

Recently uploaded

Innovations in CPLEX performance and solver capabilities