The Ascendance of the Dual Simplex Method: A Geometric View

Robert Fourer, Ascendance of the Dual Simplex Method: A Geometric View
U.S.-Mexico Workshop on Optimization and Its Applications, Huatulco, 8-12 January 2018 1
The Ascendance of the Dual
Simplex Method: A Geometric View
Robert Fourer
4er@ampl.com
AMPL Optimization Inc.
www.ampl.com — +1 773-336-AMPL
U.S.-Mexico Workshop on
Optimization and Its Applications
Huatulco — 8-12 January 2018

The Ascendance of the Dual Simplex
Method: A Geometric View
First described in the 1950s, the dual simplex evolved in
the 1990s to become the method most often used in solving
linear programs. Factors in the ascendance of the dual
simplex method include Don Goldfarb’s proposal for a
steepest-edge variant, and an improved understanding of
the bounded-variable extension. The ways that these come
together to produce a highly effective algorithm are still
not widely appreciated, however. This talk employs a
geometric approach to the dual simplex method to provide
a unified and straightforward description of the factors that
work in its favor.

Motivation

U.S.-Mexico Workshop on Optimization and Its Applications, Huatulco, 8-12 January 2018
Basic variables , nonbasic variables
 Coefficient columns of basic variables
form a nonsingular matrix
Basic solution
 ̅ 0, ̅
Feasible basic solution
 ̅ 0
4
Primal Linear Program
Minimize
Subject to , 0

Binding constraints , nonbinding constraints
 Coefficient rows of binding constraints
Vertex solution

Feasible vertex solution

5
Dual Linear Program
Maximize
Subject to

Binding constraints , nonbinding constraints
Vertex solution
 0,
Feasible vertex solution
 0
6
Dual Linear Program
Maximize
Subject to , 0

Given
 feasible basic solution ̅ and corresponding basis matrix
Choose a nonbasic variable to enter
 solve
 select ∈ : 0
Choose a basic variable to leave
 solve
 select ∈ : Θ ̅ ⁄ min ̅ /
Update
 ̅ ← Θ
 ̅ ← ̅ Θ for all ∈
7
Primal Simplex Method

Given
 feasible vertex solution and corresponding basis matrix
Choose a binding constraint to leave
 solve
 select ∈ : 0
Choose a nonbinding constraint to enter
 solve
 select ∈ : Φ / min /
Update
 ← Φ
 ← Φ for all ∈
8
Dual Simplex Method

Work of · is the same for any
 For each nonzero in column of , add to sum
 Need all in an -vector
Primal simplex
 Select one 0 ( ) for ∈
 inner products, but often ≪
Dual simplex
 Compute min / ( )
 Always inner products
9
Inner Products with : Column-Wise

Store by row as well as by column
 Accumulate inner products together
Work of · depends on sparsity of
 For each nonzero ,
 for each nonzero in row of , add to sum for
Faster in dual simplex
 tends to be especially sparse
Faster in primal if you update them all
 ← /
 ← for all ∈
10
Inner Products with : Row-Wise

Primal Steepest Edge

Simplex step
 If enters, solution ̅ changes by
 Entering variable ̅ increases to
 Basic variables ̅ change by (where )
 Objective is reduced by
Steepness of step
 /
reduction of objective per unit change in solution
Main steepest-edge computations
 Choose largest / over all ∈ with 0
 Update for ∈
 Update for ∈ . . .
12

Updating
 ← /
 known after updating
 not known except for , but is known
 ←
 ← 2
Hard part is
 Solve
 Then compute as for each ∈
 One extra solve and | | extra inner products
13

Dual Steepest Edge

Simplex step
 If constraint ∈ is relaxed,
 Slack variable increases to
 Variables change by (where )
 Objective is reduced by
Steepness of step
 /
reduction of objective per unit change in solution
Main steepest-edge computations
 Choose largest / over all ∈ with 0
 Update for ∈
 Update for ∈ . . .
17
Dual Steepest Edge

Updating
 ← ( /
 known after updating
 not known except for , but is known
 ←
 ← 2
Hard part is
 Solve
 Then is for each ∈
 One extra solve but no extra inner products
18
Dual Steepest Edge

Bounded Variables

Generalize 0 to ℓ
 State simplex methods for ℓ, finite
 Extend to allow some ℓ ∞ and/or ∞
 Check that ℓ 0, ∞ reduces to previous case
Further improve the dual simplex method
 Take longer steps
 Adapt to degeneracy
20
Bounded Variables

Basic variables , nonbasic variables ∪
 Coefficient columns of basic variables
Basic solution
 ̅ ℓ for ∈ , ̅ for ∈
 ̅ ∑ ℓ∈ ∑ ∈ ,
Feasible basic solution
 ℓ ̅
21
Primal LP with Bounded Variables
Minimize
Subject to , ℓ

Dual LP with Bounded Variables

What is : ℓ ??
 Sum of concave piecewise-linear functions : ℓ
 Slope of for 0
 Slope of ℓ for 0
 Example for 0 ℓ ∞ :
25
Maximize : ℓ
Subject to
: ℓ
ℓ

Binding constraints ,
nonbinding constraints ∪
“Vertex” solution
 0,
 ∈ for 0
 ∈ for 0
Always feasible!
26
Maximize : ℓ
Subject to
: ℓ
ℓ

Given
 feasible basic solution ̅ and corresponding basis matrix
Choose a nonbasic variable to enter
 solve
 select ∈ : 0 or
select ∈ : 0
Choose a basic variable to leave
 solve ( ∈ ) or ( ∈ )
 select Θ min Θ , Θ , Θ :
 ∈ : Θ ̅ ℓ min ̅ ℓ ⁄
 ∈ : Θ ̅ ⁄ min ̅ ⁄
 : Θ ℓ
Update . . .
27
Primal Simplex, Bounded Variables

Given
 feasible vertex solution and corresponding basis matrix
Choose a binding constraint to leave
 solve ∑ ℓ∈ ∑ ∈
 select ∈ : ℓ or
Choose a nonbinding constraint to enter
 solve (if ℓ ) or (if )
 select Φ min Φ , Φ ) :
 ∈ : Φ / min /
 ∈ : Φ / min /
Update . . .
28
Dual Simplex, Bounded Variables
: ℓ
ℓ

Some ℓ ∞ and/or ∞ ??
 A basis may be infeasible
 0 but ℓ ∞
 0 but ∞
 Minimize “sum of infeasible variables” to get feasible
 Replace each ℓ ∞ by 1
 Replace each ∞ by 1
 Replace all finite bounds by 0
All ℓ 0 and ∞ ??
 Then 0
 Primal and dual algorithms
reduce to their simpler forms
29
In Principle, Bounds Can Be Infinite
∞ ∞

Decisions are bounded
 Variables are bounded
 Slacks on inequality constraints are bounded
Integer-valued decisions have small values
 Many are zero-one!
Standard presolve routines compute bounds
 Compute bounds where none given by user
 Tighten bounds in multiple passes
30
In Practice, Bounds Tend to be Finite

Standard iteration
 increases to Φ (if ℓ ) or decreases to Φ (if )
 moves to 0
Suppose increases/decreases further . . .
 moves past 0, but solution remains feasible
 Rate of improvement in objective degrades by ℓ
 If increasing, objective slope
changes from to ℓ
 If decreasing, objective slope
changes from ℓ to
 If rate still positive, can continue
until next , ∈ reaches 0
31
Long Steps
: ℓ
ℓ

Replace single ratio test by a loop
 Set initial objective improvement rate to
ℓ (if ℓ ) or (if )
 Form set of all ratios that may be reached:
: ⁄ ∈ , 0 ∪ : ⁄ ∈ , 0
 Repeat for increasing ∈ ⁄ ∶
 Let ← ℓ
 Let ← ∖ ⁄
until 0
Equivalent to “weighted selection”
 In theory, faster than sorting
 In practice, a small part of iteration cost
32
Long Step Iteration

Change bounds for some fractional ̅ , ∈
 Increase ℓ to ̅ , resulting in ̅ ℓ
 Decrease to ̅ , resulting in ̅
 Either way, binding constraint ∈ can leave
 Continue with dual simplex steps until optimal again
Fix some fractional binary ̅ , ∈
 Increase ℓ to 1, resulting in ̅ ℓ
 Decrease to ℓ 0, resulting in ̅
 Either way, binding constraint ∈ can leave
 Since now ℓ , using long steps
the constraint will never become binding again
33
Re-Optimization for MIPs

Choosing a binding constraint to relax
 For 0, place ∈
 For 0, place ∈
 For 0 ???
 Guess ∈ if you think is likely to increase
 Guess ∈ if you think is likely to decrease
. . . can’t be sure though until you choose
 solve ∑ ℓ∈ ∑ ∈
 select ∈ : ℓ or
34
Degeneracy
: ℓ
ℓ

Choosing a nonbinding constraint to add
 Set initial objective improvement rate to
ℓ (if ℓ ) or (if )
 Collect all ratios that may be reached:
: ⁄ ∈ , 0 ∪ : ⁄ ∈ , 0
 Some of these ratios may be zero!
 For ∈ , = 0 and 0
 For ∈ , = 0 and 0
 Repeat for every ⁄ 0 ∈ ∶
 Let ← ℓ
 Let ← ∖ ⁄
while 0
 If 0, iteration is degenerate
 If still 0, continue with
nondegenerate iteration
35
Degeneracy
: ℓ
ℓ

For some 0, you “guess wrong”
 ∈ , but decreases
 ∈ , but increases
As a result, you are overly optimistic about
the objective improvement rate r
Long-step ratio test corrects for your wrong guesses
 If the corrected 0 then
you can take a nondegenerate step after all
None of this changes the optimality condition
 ℓ for all ∈
36
Degeneracy: Benefit of Long Steps

Fast inner products
Dual steepest edge
Bounded-variable extension
 Feasibility for finite bounds
 Long steps
 More nondegenerate steps
37
Ascendance of the Dual Simplex

The Ascendance of the Dual Simplex Method: A Geometric View

Recommended

Recommended

More Related Content

Similar to The Ascendance of the Dual Simplex Method: A Geometric View

Similar to The Ascendance of the Dual Simplex Method: A Geometric View (18)

More from Bob Fourer

More from Bob Fourer (7)

Recently uploaded

Recently uploaded (20)

The Ascendance of the Dual Simplex Method: A Geometric View