The document summarizes a talk on solving large scale linear network flow problems with MOSEK. It outlines the network simplex algorithm, how MOSEK exploits the structure of network problems, and provides computational results comparing MOSEK to other network solvers. MOSEK uses a primal network simplex method that is highly tuned for large problems by exploiting degeneracy and using efficient data structures. Computational tests show MOSEK outperforms earlier network solvers and is competitive with specialized network simplex and cost-scaling implementations.

1. Informs Annual Conf. Denver 2004
1/25
Solving Large Scale Linear Network
Flow Problems with MOSEK
Bo Jensen
MOSEK ApS
C/O Symbion Science Park
Fruebjergvej 3, Box 16
2100 Copenhagen Ø
Denmark
http://www.mosek.com
bo.jensen@mosek.com
Back
Close

2. Talk outline
• Introduction to MOSEK 2/25
• The pure network flow problem
• Network simplex - exploiting structure
• MOSEK network simplex
• Computational results
• The MOSEK network API - design considerations
Back
Close

3. Introduction to MOSEK
An large scale optimization software package. 3/25
Solvers : Interfaces :
• Linear optimization, interior and • MATLAB
simplex
• GAMS
• Mixed integer for linear and
• AMPL
quadratic optimization
• AIMMS
• Conic optimization
• Java
• Generel convex nonlinear optimiza-
tion • C/C++
• .NET
Current Version 3.1. The next major release Ver. 4.0 will contain
many new features and improvements among these network simplex.
Back
Close

4. The pure network flow problem
Given a network G = (N, A) with capacities lij , ux , and costs cij
x
ij
4/25
on its arcs, find flow satisfying node demands/supply at lowest total
cost.
Standard LP formulation:
min cij xij
(i,j)∈A
s.t. lic ≤ xij − xji ≤ uc for all i ∈ N
i
{j:(i,j)∈A} {j:(j,i)∈A}
lij ≤ xij ≤ ux
x
ij for all (i, j) ∈ A .
Problem structure:
Every column has two nonzeroes, with value -1.0 and 1.0.
Back
Close

5. Well known facts
• A large amount of solutions methods 5/25
• Both polynomial simplex and non-simplex algorithms exists
• Practical efficient implementations exists
• Can solve large problems very fast
• Primal network simplex is competitive
Back
Close

6. Network simplex - exploiting structure
• Basis is a permuted triangular matrix 6/25
2 3 2 3
x48 x31 x47 x12 x36 x14 x35 x1r x48 x47 x36 x35 x14 x31 x12 x1r
6
6 r1 1 −1 −1 −1 77
6
6 r8 1 7
7
6 r2 1 7 6 r7 1 7
r3 −1 −1 −1 r6 1
6 7 6 7
6 7 6 7
B=6 r4 −1 −1 1 7L = 6 r5 1
6 7 6 7
7
6
6 r5 1 7
7
6
6 r4 −1 −1 1 7
7
6
6 r6 1 7
7
6
6 r3 −1 −1 −1 7
7
4 r7 1 5 4 r2 1 5
r8 1 r1 −1 1 −1 −1
• Basis can be represented as a spanning tree
Back
Close

7. – Fast FTRAN
∗ We need B −1Aj for entering column j, loop the basis equiv-
alent path (BEP).
7/25
Back
Close

8. – Fast BTRAN
∗ If pivoting xBi out of basis, we need row i of B −1 to update
free duals. Correspond to subtree hanging from xBi
8/25
2 3
1
6 −1 −1 −1 7
1
6 7
6 7
1
6 7
B −1 =6
6 7
6 1 7
7
6
6 1 1 1 7
7
4 1 5
−1 −1 −1 −1 −1 −1 −1 −1
Back
Close

9. MOSEK primal network simplex
9/25
What is implemented ?
• No new theoretical ideas
• In generel a brush up and refinement of known methods
• Highly tuned algorithm for large scale problems
Back
Close

10. • Where is the time spend ?
Problem numcon numvar Price FTRAN BTRAN
big8 50000 400000 52% 3% 44%
8 n12 4096 32768 78% 12% 7%
10/25
16 n12 4096 65536 86% 8% 4%
i n10 1024 32768 90% 4% 4%
hi7 16384 131431 30% 9% 58%
lo7 16384 131431 34% 11% 52%
Conclusion :
Pricing and updating duals are the two key operations. The time
spend in dual update is highly affected by the structure of the basis
tree.
Back
Close

11. • Some problems tend to produce ’cheap’ basis trees
11/25
Wide and ’cheap’ basistree Deep and ’expensive’ basistree
Note : Having many slacks in the basis keeps the tree wide.
Back
Close

12. • Exploiting degeneracy
Flow problems are often highly primal degenerate. Creates ’ties’
in the ratio test, we can choose freely among these. Of course the
question of cycling arise.
12/25
Back
Close

13. • Efficient datastructure for basis tree
Objectives :
1. Easy location of smallest subtree size for better dual update
2. Possible to loop subtree above and below a node 13/25
3. Cheap identification of BEP
4. Easy to update after basis exchange
Mosek uses a version of the Extended Augmented Predecessor
Method (L¨bel,1996 )
o
Back
Close

14. • Obtaining a feasible solution
Most standard simplex algorithms use a two phase method, where
phase one minimize the sum of infeasibility :
x 14/25
min (lij − xij ) + (xij − ux )
ij
(i,j)∈I1 (i,j)∈I2
x
I1 = {(i, j) ∈ A|lij > xij }
I2 = {(i, j) ∈ A|ux < xij }
ij
This does not work well for the network simplex.
Back
Close

15. Network simplex feasibility method:
– Network simplex codes use some kind of penalty approach
– Very sensitive if penalty is set too high, disturbs the pricing
phase 15/25
– MOSEK uses a variant of the Gradual Penalty Method (GPM)
(Grigoriadis, 1986 )
Back
Close

16. • Pricing - a historical perspective
Problem numcon numvar Mulvey Gibby Grigoriadis SMP
8 n11 2048 16384 3.61 1.06 1.43 1.00
16 n11 2048 32768 6.66 1.09 1.76 1.00
16/25
i n11 2048 92682 16.11 1.19 2.03 1.00
long13 8193 188439 3.15 5.01 2.51 1.00
hi6 11585 93033 1.32 1.60 2.96 1.00
lo7 46341 406225 1.49 2.48 2.57 1.00
a
a
Jensen and Berthelsen, 2003
– Older partial pricing strategies is not competitive
– Keeping a sorted queue seems to be method of choice
Back
Close

17. Computational results
• Development in network simplex 17/25
• MOSEK network Vs. other specialized implementations
• MOSEK network Vs. standard LP solvers
• Conclusions
Back
Close

18. First an example of development in network simplex
Netflo (Kennington and Helgason, 1980 )
Vs.
MOSEK network optimizer 2004 18/25
Problem numcon numvar Netflo MOSEK Netflo/MOSEK
8 n11 2048 16384 3.12 2.35 1.33
16 n11 2048 32768 7.12 3.67 1.94
i n11 2048 92682 42.66 10.43 4.09
hi5 8192 65750 11.36 1.5 7.57
lo9 32768 262921 1064.21 59.58 17.86
long13 8193 188439 16.26 1.80 9.03
square15 32762 753526 1452.10 43.41 33.45
wide15 32769 753687 1550.62 40.79 38.01
a b
a
An early version of MOSEK network simplex. Both solvers running on
integer data.
b
Jensen and Berthelsen, 2003
Back
Close

19. Network Simplex and Cost Scaling implementations
MCF Solvers:
• MCF 1.3 an efficient network simplex implementation by L¨bel
o
19/25
• CS2 an efficient Cost-Scaling Push-Relabel algorithm by Goldberg
Test setup :
• In total 77 problems were included in the test, results are shown
for a representive subset
• All test instance were randomly generated
• We used a C++ translation from the MCFClass project at the
University of Pisa
• Double precision data is used for all tested codes
• Tested on a AMD Athlon XP 3200++ 1 GB RAM running Linux
Back
Close

20. NETGEN LO instances generated with netgen
Problem numcon numvar MOSEK network MCF CS2
Iter Time Iter Time Time
lo1 2048 17933 21744 0.12 22294 0.17 0.2
lo2 2896 25355 33279 0.25 35229 0.33 0.40 20/25
lo3 4096 32877 45577 0.41 49049 0.54 0.63
lo4 5793 50760 67927 0.81 76961 1.18 1.22
lo5 8192 65750 89836 1.32 109025 2.33 2.05
lo6 11585 93033 138303 2.65 165839 7.46 3.92
lo7 16384 131431 193585 5.80 196823 13.27 6.68
lo8 23170 202986 276060 11.51 286881 24.35 13.18
lo9 32768 262921 400955 26.93 423743 63.27 18.86
lo10 46341 406225 608275 73.16 639384 133.32 37.60
Problem MCF / MOSEK CS2 / MOSEK
Iter Time Time
lo1 1.03 1.42 1.67
lo2 1.06 1.32 1.60
lo3 1.08 1.32 1.54
lo4 1.13 1.46 1.51
lo5 1.21 1.77 1.55
lo6 1.20 2.82 1.48
lo7 1.02 2.29 1.15
lo8 1.04 2.12 1.15
lo9 1.06 2.35 0.70
lo10 1.05 1.82 0.51
Back
Close

21. GRIDGEN LONG instances generated with gridgen
Problem numcon numvar MOSEK network MCF CS2
Iter Time Iter Time Time
long11 2049 47127 6532 0.10 9844 0.15 0.4
long11 5 2897 66631 8389 0.14 16073 0.27 0.68 21/25
long12 5 5793 133239 15502 0.48 19950 0.82 1.84
long13 8193 188439 22834 1.00 30636 1.97 3.00
long13 5 11585 266455 34172 1.97 48896 5.34 5.29
long14 16385 376855 54242 4.83 87330 17.36 9.52
long14 5 23169 532887 73049 9.67 138079 49.16 16.05
long15 32769 753687 107550 22.02 207611 87.31 30.32
Problem MCF / MOSEK CS2 / MOSEK
Iter Time Time
long11 1.51 1.50 4.00
long11 5 1.92 1.93 4.86
long12 5 1.29 1.71 3.83
long13 1.34 1.97 3.00
long13 5 1.43 2.71 2.69
long14 1.61 3.59 1.97
long14 5 1.89 5.08 1.66
long15 1.93 3.97 1.38
Back
Close

22. GOTO 8 instances generated with GOTO
Problem numcon numvar MOSEK network MCF CS2
Iter Time Iter Time Time
8 n9 512 4096 10375 0.06 11318 0.06 0.04
8 n95 724 5792 15693 0.11 22345 0.15 0.06 22/25
8 n10 1024 8192 24814 0.21 31234 0.26 0.10
8 n105 1448 11584 35607 0.38 39052 0.56 0.16
8 n11 2048 16384 58868 0.98 65258 1.30 0.32
8 n115 2896 23168 87171 2.46 103824 3.43 0.59
8 n12 4096 32768 172425 4.67 178520 8.33 1.27
8 n125 5793 46344 187540 9.69 253379 17.87 1.74
8 n13 8192 65536 266201 26.47 365946 51.09 3.13
8 n135 11585 92680 377822 55.57 561301 154.72 5.73
8 n14 16834 131072 522723 130.61 715647 425.83 8.92
Problem MCF / MOSEK CS2 / MOSEK
Iter Time Time
8 n9 1.09 1.00 0.67
8 n95 1.42 1.36 0.55
8 n10 1.26 1.24 0.48
8 n105 1.10 1.47 0.42
8 n11 1.11 1.33 0.33
8 n115 1.19 1.39 0.24
8 n12 1.04 1.78 0.27
8 n125 1.35 1.84 0.18
8 n13 1.37 1.93 0.12
8 n135 1.49 2.78 0.10
8 n14 1.37 3.26 0.07 Back
Close

23. Standard LP Vs. Network Simplex
Problem numcon numvar Primal network simplex Primal simplex IPM
iter Time iter Time Time
hi1 2048 17933 6384 0.06 9886 1.34 7.37
hi2 2896 25355 7870 0.10 14557 2.67 19.30 23/25
hi3 4096 32877 10998 0.15 24931 7.99 47.73
hi4 5793 50760 16910 0.32 40704 17.43 136.62
hi5 8192 65750 22897 0.63 68643 52.95 329.58
hi6 11585 93033 34545 1.17 124024 165.36 1027.00
hi7 16384 131431 49894 2.49 219899 492.01 3600.42
hi8 23170 202986 76292 5.89 353020 1205.78 7391.12
hi9 32768 262921 118673 13.73 703132 4194.89 M EM
hi10 46341 406225 194499 35.77 1071190 9016.27 M EM
Problem Primal simplex / network IPM / network
Iter Time Time
hi1 1.55 22.33 122.83
hi2 1.85 26.70 193.00
hi3 2.27 53.27 318.20
hi4 2.41 54.47 426.94
hi5 3.00 84.05 523.14
hi6 3.59 141.33 877.78
hi7 4.41 197.59 1445.95
hi8 4.63 204.72 1254.86
hi9 5.92 305.53 ?
hi10 5.51 252.06 ?
a
Back
a
Tested on a AMD Athlon XP 3200++ 1 GB RAM running Linux. Close

24. Numerical conclusions
24/25
Generel conclusions
• Network simplex implementations has improved
• Network simplex is surprisingly faster than standard simplex
• Network simplex and cost scaling algorithms are method of choice
– Network simplex performs well optimizing small to large graphs
and on ’easy’ problems
– For reoptimization is network simplex superior (see Frangioni
and Manca 2004)
MOSEK related conclusions
• MOSEK is the fastest network simplex on 73 of 77 instances
• MOSEK is faster than CS2 on 40 of 77 instances Back
Close

25. The MOSEK network API - design consid-
erations 25/25
• Own network API or standard LP API with network detection
– Specific network API more efficient
• Easy integration with standard API
– Transparent exchange data between network and standard task
– No terms as edge and nodes
– Interface with two non zeroes in each column
– Extendable API to generalized networks
Back
Close