1. Outline



relationship among topics
secrets
LP with upper bounds

by Simplex method


by Simplex method for bounded variables


extended basic feasible solution (EBFS)
optimality conditions for bounded variables


basic feasible solution (BFS)
ideas of the proof
examples


Example 1 for ideas but inexact
Example 2 for the exact procedure
1

2. A Depot for Multiple Products
 multi-product
by a fleet of trucks
Possible Formulation:
objective function
common constraints, e.g., trucks,
DC capacity, etc.
network
constraints for
type-1 product
network
constraints for
type-1 product
....
depot
network
constraints for
type-1 product
non-negativity constraints
2

3. A General Type
of Optimization Problems

structure of many problems:




network constraints: easy
other constraints: hard
objective function
network constraints
hard constraints
non-negativity constraints
making use of the easy constraints to solve the problems
solution methods: large-scale optimization


column generation, Lagrangian relaxation, Dantzig-Wolfe
decomposition …
basis: linear programming, network optimization (and also
non-linear optimization, integer optimization, combinatorial
optimization)
3

4. Relationship of Solution Techniques
 two
directions of theoretical development
for network programming
linear prog.
 from
special structures of networks
 from
linear programming
network prog.
 ideal:
understanding
development in both directions
non-linear prog.
dynamic prog.
…
int. prog.
4

5. Relationship of Solution Techniques
minimum cost flow
network
algorithms
shortest-path
algorithms
column generation, DantzigWolfe decomposition
network
simplex
revised simplex method
simplex method
linear algebra
Lagrangian
relaxation
non-linear
optimization
5

6. Our Topics

simplex method for bounded variables



minimum cost algorithms






linkage between LP and network simplex
optimality conditions for minimum cost flow networks
standard, and successive shortest path
equivalence among network and LP optimality conditions
revised simplex
column generation
Dantzig-Wolfe decomposition
Lagrangian relaxation
It takes more than one
semester to cover these
topics in detail! We will
only cover the ideas.
6

7. Secrets
7

8. The Most Beautiful …
8

9. Maybe the Most Beautiful of All…
 linear
algebra
geometric
properties
algebraic
properties
matrix
properties
9

10. LP with Upper Bounds
10

11. LP with Upper Bounds
 upper
bounds: common in network problems,
e.g., an arc with finite capacity
 quite
some theory of network optimization
being from LP
max
s.t.
T
c x
Ax b
0 x u
11

12. To Solve LP with Upper Bounds
 incorporate
the upper-bound constraints into
the set of functional constraints and solve
accordingly
max
s.t.
cT x
Ax b
0 x u
max
s.t.
cT x
A
x
I
0
b
u
x
12

13. To Solve LP with Upper Bounds
 In
the simplex method the lower bound
constraints 0 x do not appear in A.
 Is
it possible to work only with A even with
upper-bound constraints?
 Yes.
max
s.t.
cT x
Ax b
0 x u
max
s.t.
cT x
A
x
I
0
b
u
x
13

14. BFS for Standard LP max
 Am n,
m
 basic
feasible solution (BFS) x of LP, i.e.,
 feasible:
n, of rank m
s.t.
cT x
Ax b
0 x
Ax
b, 0
x
 basic
 non-basic
variables: (at least) n-m variables = 0
 basic
variables: m non-negative variables with linearly
independent columns
14

15. Extended Basic Feasible Solution of
LP with Bounded Variables
 Am n,
m
n, of rank m
 extended
basic feasible solution ( EBFS ) x of
LP with bounded variables, i.e., max cT x
 feasible:
 basic
Ax
b, 0
x
u
s.t.
Ax b
0 x u
solution
 non-basic
variables: (at least) n-m variables = 0, or =
their upper bounds
 Basic
variables: m variables of the form 0
linearly independent columns
xi
ui, with
15

16. Optimality Conditions
of Standard LP

Maximum Conditions: BFS x is maximal if


0 for all non-basic variable xj = 0
Minimum Conditions: BFS x is minimal if


cj
cj
0 for all non-basic variable xj = 0
intuition

c j : increase of the objective function by unit increase
in xj
 maximum condition: no good to increase non-basic xj
 minimum condition: no good to decrease non-basic xj
16

17. Optimality Conditions
of LP with Bounded Variables
 Maximum
Conditions: EBFS x is maximal if
 cj
0 for all non-basic variable xj = 0, and
 cj
0 for all non-basic variable xj = uj
 Minimum
Conditions: EBFS x is minimal if
 cj
0 for all non-basic variable xj = 0, and
 cj
0 for all non-basic variable xj = uj
17

18. How to Prove?
18

19. General Idea
 optimality
conditions of the EBFS
 from
duality theory and complementary slackness
conditions
19

20. Complementary Slackness
Conditions
 primal-dual
pair
max
s.t.
cT x
Ax b
0 x
min
bT y
s.t.
y T A cT
y
 Theorem
1 (Complementary Slackness
Conditions)
 if
x primal feasible and y dual feasible
 then x primal optimal and y dual optimal iff
xj(yTA j cj) = 0 for all j, and yi(bi Ai x) = 0 for all i
20

21. Complementary Slackness
Conditions
 primal-dual
pair
max
s.t.
cT x
Ax b
0 x
min
bT y
s.t.
y T A cT
y
 Theorem
2 (Necessary and Sufficient
Condition)
 if
x primal feasible
 then x primal optimal iff there exists dual feasible
y such that x and y satisfy the Complementary
Slackness Conditions
21

22. Complementary Slackness Conditions
for LP with Bounded Variables
max
s.t.
cT x
Ax b
x u
0 x
bT y + uT
min
yT A +
s.t.
T
cT
y
 by
Theorem 2, primal feasible x and dual
feasible (yT, T) are optimal iff
 xj(yTA j
 yi(bi

j(uj
+
j
- cj-) = 0,
j
- Ai x) = 0,
i
- xj-) = 0,
j
22

23. General Idea of the Proof

optimality conditions of the EBFS
 from
duality theory and complementary slackness
conditions

ideas of the proof
 given
an EBFS x satisfying the upper-bound optimality
conditions
possible to find dual feasible variables (yT, T)T
such that x and (yT, T)T satisfy the complementary
slackness conditions
 then
23

24. Example 1. Upper-Bound Constraints
as Functional Constraints
 max
2x + 5y,
min
2x
5y,
 s.t.
x + 2y



20,
2x + y
0
x
16,
2, 0
y
8.
24

25. Examples of LP with
Bounded Variables
25

26. Example 1. Upper-Bound Constraints
as Functional Constraints

min

2x
5y,
s.t.

x + 2y
20,

2x + y
16,

0
x
2, 0
y
8.

max. value = 44

x* = 2 and y* = 8
26

27. The following procedure is not exactly
the Simplex Method for Bounded
Variables. It primarily brings out the
ideas of the exact method.
27

28. Example 1. Upper-Bound Constraints
by Optimality Conditions of Bounded Variables
-5

y as the entering variable
 2y
+ s1 = 20
 y + s2 = 16
y 8
min 2x 5y,
s.t.
x + 2y 20,
2x + y 16,
0 x 2, 0 y
8.
28

29. Example 1. Upper-Bound Constraints
by Optimality Conditions of Bounded Variables


mark the non-basic variable y at its upper bound
for y = 8



obj. fun.: -2x – 5y – z = 0
eqt. (1): x + 2y + s1 = 20
eqt. (2): 2x + y + s2 = 16
-2x - z = 40
x + s1 = 4
2x + s2 = 8
29

30. Example 1. Upper-Bound Constraints
by Optimality Conditions of Bounded Variables

x as the entering variable
x
+ s1 = 4
 2x
x
+ s2 = 8
2
min 2x 5y,
s.t.
x + 2y 20,
2x + y 16,
0 x 2, 0 y
8.
30

31. Example 1. Upper-Bound Constraints
by Optimality Conditions of Bounded Variables

for x at its upper bound 2, mark x, and
 obj.
fun.: -2x – z = 40
-z = 44
 eqt.
(1): x + s1 = 4
s1 = 2
 eqt.
(2): 2x + s2 = 8
s2 = 4
min 2x 5y,
s.t.
x + 2y 20,
2x + y 16,
0 x 2, 0 y
8.
31

32. Example 1. Upper-Bound Constraints
by Optimality Conditions of Bounded Variables

satisfying the optimality condition for bounded
variables

0 for all non-basic variable xj = 0, and


cj
cj
0 for all non-basic variable xj = uj
z* = -44, with x* = 2 and y* = 8
32

33. Example 1 Being Too Specific

in general, variables swapping among all sorts
of status
 non-basic
at 0
 basic
at 0
 basic between 0 and upper bound
 basic at upper bound
 non-basic at upper bound

Simplex method for bounded variables: a
special algorithm to record all possibilities
33

34. The following example follows the
exact procedure of the Simplex
Method for Bounded Variables.
34

35. Example 2
 max
3x1 + 5x2 + 2x3
min 3x1
5x2
2x3,
 s.t.

x1 + x2 + 2x3
7,

2x1 + 4x2 + 3x3
15,

0
x1
4, 0
x2
3, 0
x3
3.
35

36. Example 2 by Simplex Method
for Bounded Variables
 potential
entering variable: x2
 bounded
by upper bound 3
 define x2
= u2-x2 = 3-x2
min 3x1 5x2 2x3,
s.t.
x1 + x2 + 2x3
2x1 + 4x2 + 3x3
0 x1 4, 0 x2
7,
15,
3, 0
x3
36
3.

37. Example 2 by Simplex Method
for Bounded Variables
37

38. Example 2 by Simplex Method
for Bounded Variables
 x1
 s2

as the (potential) entering variable
as the leaving variable
min 3x1 5x2 2x3,
s.t.
x1 + x2 + 2x3
2x1 + 4x2 + 3x3
0 x1 4, 0 x2
3.
7,
15,
3, 0
a pivot operation as in standard Simplex Method
38
x3

39. Example 2 by Simplex Method
for Bounded Variables
 which
can be an entering variable? x2
 can s1
be a leaving variable? Yes
 can x1
be a leaving variable? Yes
min 3x1 5x2 2x3,
s.t.
x1 + x2 + 2x3
2x1 + 4x2 + 3x3
0 x1 4, 0 x2
3.
7,
15,
3, 0
39
x3

40. Example 2 by Simplex Method
for Bounded Variables



when x2 = 1.25, x1 reaches its upper bound 4
replace x1 by x1 , and x1 is a basic variable = 0
min 3x
result x1 2 x2 1.5 x3 0.5s2 1.5
s.t.
1
(u1 x1 ) 2 x2 1.5 x3 0.5s2 1.5
5x2
2x3,
x1 + x2 + 2x3
2x1 + 4x2 + 3x3
0 x1 4, 0 x2
7,
15,
3, 0
x1 2 x2 1.5 x3 0.5s2 1.5 u1
40
x3
3.

41. Example 2 by Simplex Method
for Bounded Variables
x
 . 2 entering and x1 leaving
a
min 3x1 5x2 2x3,
s.t.
x1 + x2 + 2x3
2x1 + 4x2 + 3x3
0 x1 4, 0 x2
3.
7,
15,
3, 0
x3
“normal” pivot operation with aij < 0
41

42. Example 2 by Simplex Method
for Bounded Variables
 minimum

min 3x1 5x2 2x3,
s.t.
x1 + x2 + 2x3
2x1 + 4x2 + 3x3
0 x1 4, 0 x2
3.
7,
15,
3, 0
x3
z* = -20.75, x1* = 4, x2* = 1.75, x3* = 0
42