2. 2
Optimization Methods
One-Dimensional Unconstrained Optimization
Golden-Section Search
Quadratic Interpolation
Newton's Method
Multi-Dimensional Unconstrained Optimization
Non-gradient or direct methods
Gradient methods
Linear Programming (Constrained)
Graphical Solution
Simplex Method
3. 3
• Basic linear programming (LP) problem consists
of two major parts:
– The objective function
– A set of constraints
• For maximization problem, the objective function
is generally expressed as
n
n x
c
x
c
x
c
2
2
1
1
Z
Maximize
cj= payoff of each unit of the jth activity that is undertaken
xj= magnitude of the jth activity
Z= total payoff due to the total number of activities
Standard Form Linear Programming Problem
4. 4
• The constraints can be represented generally as
where
aij= amount of the ith resource that is consumed for each
unit of the jth activity
bi = amount of the ith resource that is available
• The general second type of constraints specifies
that all activities must have a positive value. i.e,
i
n
in
i
i b
x
a
x
a
x
a
2
2
1
1
Standard Form Linear Programming Problem
0
i
x
5. 5
Problem
Products
Resource Regular Premium Resource
Availability
Raw gas 7 m3/tonne 11 m3/tonne 77 m3/week
Production
time
10 hr/tonne 8 hr/tonne 80 hr/week
Storage 9 tonne 6 tonne
Profit $150/tonne $175/tonne
How many tonnes of regular and premium gas to
produce in order to maximize weekly profit?
6. 6
Developing LP Formulation
Let x1 = 1 tonne of regular gas
x2 = 1 tonne of premium gas
Maximize Z = 150x1 + 175x2
subject to
7x1 + 11x2 ≤ 77 (material constraint) (1)
10x1 + 8x2 ≤ 80 (time constraint) (2)
x1 ≤ 9 (regular gas storage) (3)
x2 ≤ 6 (premium gas storage) (4)
x1, x2 ≥ 0 (positivity) (5, 6)
7. 7
Graphical Solution
Feasible solution space
(the shaded area) contains
all points that obey the
constraints or points that
represent the feasible
solutions.
Points located outside of
this area are known as
infeasible solution.
8. 8
Graphical Solution
We can adjust the line
representing the objective
function Z = 150x2 + 175x2
to locate the optimum
point.
One of the corner points
(points where two or more
lines intersect) will be an
optimum point.
– Corner points can be
feasible or infeasible
9. 9
Graphical Solution
Some insights:
• Increasing storages (
constraints (3) and (4))
does not improve profit.
• Raising either materials or
production time can
improve profit.
10. 10
Possible outcomes that can generally be
obtained in a LP problem
1. Unique solution. The maximum objective
function intersects a single point.
2. Alternate solutions. Problem has an infinite
number of optima corresponding to a line
segment.
3. No feasible solution.
4. Unbounded problems. Problem is under-
constrained and therefore open-ended.
11. 11
(a)Unique solution – represented by a corner point
Alternate solutions – represented by points on one
of the constraint line
(b) No feasible solution
(c) Unbounded problems
12. 12
Key properties of LP that drives the design of
Simplex Method (An algorithm for solving LP
problems)
1. The optimum point is always at a feasible corner
point. (Why?)
– We only need to check the corner points.
2. If a corner point feasible solution has an objective
function value that is better that or equal to all
adjacent corner point feasible solutions, then it is
optimal.
– We don't necessarily need to search all corner
points.
13. 13
Key properties of LP that drives the design
of Simplex Method
3. There are finite number of corner point feasible
solutions.
–Any method that checks only corner points will
terminate eventually.
14. 14
Basic idea behind the Simplex method
1. Starting at a feasible corner point solution.
2. Repeatedly move to a better adjacent corner point
feasible solution until an optimum point is found.
15. 15
Finding Corner Points Algebraically
• How to find corner points algebraically?
• How to locate adjacent corner points?
• How to decide which adjacent point to
move to next?
• Simplex method offers a neat solution
(involves the use of slack variables) to
address these questions.
16. 16
First, constraint equations are reformulated as
equalities by introducing slack variables – variables
that measures how much of a constrained resource is
available.
0
,
,
,
,
,
6
9
80
8
10
77
11
7
to
subject
175
150
Maximize
4
3
2
1
2
1
4
2
3
1
2
2
1
1
2
1
2
1
S
S
S
S
x
x
S
x
S
x
S
x
x
S
x
x
x
x
Z
0
,
6
9
80
8
10
77
11
7
to
subject
175
150
Maximize
2
1
2
1
2
1
2
1
2
1
x
x
x
x
x
x
x
x
x
x
Z
Slack variables
Si > 0 means the corresponding resource is not fully consumed.
Si < 0 means the corresponding resource is over-consumed.
17. 17
• Adding slack variables results in an augmented
system of linear equations which is under specified
(has more unknowns than equations).
– 6 unknowns (= 2 original variables + 4 slack variables)
– 4 equations
• In general, if a LP problem has n variables and m
constraints, the resulting augmented system will
have a total of (n original variables + m slack
variables) and m equations.
Augmented System
Continue …
18. 18
• To solve an augmented system with
– 6 unknowns (= 2 original variables + 4 slack variables)
– 4 equations
We need to set 2 of the variables constants before
we can solve the system of equations.
e.g., Setting x1 = 0, x2 = 0 yields
Augmented System
Continue …
6
9
80
77
4
3
2
1
S
S
S
S
6
9
80
8
10
77
11
7
4
2
3
1
2
2
1
1
2
1
S
x
S
x
S
x
x
S
x
x
19. 19
Solving the resulting system yields x1=0, x2=0, S1=77, S2=80,
S3=9, S4=6, which represent one of the augmented solutions –
values of all (original + slack) variables are given.
For this example, the values tell us that if we don't produce any regular (x1)
or premium (x2) gas, we would have the following amount of unconsumed
resources:
– 77 m3 of raw gas,
– 80 production hours
– 9 tonne of regular gas storage
– 6 tonne of premium gas storage
6
9
80
77
4
3
2
1
S
S
S
S
Augmented System
20. 20
• An augmented solution which corresponds to a
corner point is also known as a basic solution.
– The augmented solution x1=0, x2=0, S1=77, S2=80, S3=9,
S4=6 is a basic solution. It corresponds to corner point A.
– In addition, it is a basic feasible solution because all
variables ≥ 0.
• So far, you know how to find an augmented
solution.
– How should we set the variables so that the augmented
solution is a basic solution?
Augmented System
Continue …
21. 21
• The equations (or constraints) have a one-to-one
relationship with the slack variables.
– For example, S1 relates to the 1st equation, S4 relates to
the 4th equation.
• If a point is on the line 7x1 + 11x2 = 77 (line labeled
"1"), what's the value of S1?
• If a point is on the line 10x1 + 8x2 = 80 (line labeled
"2"), which variable has the value zero?
• At the corner point D, which variables have the
value zero?
Characteristics of Slack Variables
Show Graph & Equations
22. 22
• Setting two variables to zero has the effect of
"selecting two lines", and solving the resulting
system means finding the intersecting point (a
corner point) of the selected lines.
Characteristics of Slack Variables
6
9
80
8
10
77
11
7
4
2
3
1
2
1
2
1
S
x
S
x
x
x
x
x
The value of x1 and x2 of this system is the coordinate of
the corner point C (the point where line "1" and line "2"
intersect.)
For example, let S1=0, S2=0. Then the system becomes
23. 23
Finding Corner Points Algebraically
We now know how to find the corner points.
Next
• How to find an adjacent corner point?
• How to decide which adjacent corner point
to move to?
24. 24
Basic and Non-Basic Variables
Corner
Point
Non-Basic
Variables
Basic
Variables
Some of the
Adjacent
Corner Points
A x1, x2 S1, S2, S3, S4 B, E
B x2, S2 x1, S1, S3, S4 A, C
C S1, S2 x1, x2, S3, S4 B, D
D S1, S4 x1, x2, S2, S3 C, E
E x1, S4 x2, S1, S2, S3 A, D
F x1, S2 x2, S1, S3, S4
Variables in which their values are set to zero are
called non-basic variables.
Remember: Non-basic, variable set to zero,
corresponding constraint is active
Do you see a
pattern between
any pair of
adjacent corner
points?
25. 25
Basic and Non-Basic Variables
• If two points are adjacent corner points, then
– Their basic sets are identical except for one member.
– Their non-basic sets are identical except for one
member.
• The reverse of the above statement is not always
true. (e.g., F and A)
• Questions:
– To find an adjacent corner point, can we just switch one
variable between the basic and non-basic sets?
– How should we decide which corner point to move to
next?
26. 26
Basic and Non-Basic Variables
How do we make sure the following
conditions are satisfied?
– The corner points are adjacent.
– The corner points are feasible.
– The new corner point gives better value
for the objective function than the current
corner point.
27. 27
Which variables to switch?
All three conditions from the previous slide can be
satisfied if the following steps are taken:
• Selecting an entering basic variable – a non-
basic variable that will increase the objective
function value the most if allowed to take on a
positive value. Move this variable from the non-
basic set to the basic set.
• Finding the leaving basic variable – a basic
variable that is forced to a value zero first after
allowing the entering basic variable to increase.
Move the variable that is forced to zero from the
basic set to the non-basic set.
28. 28
Which variables to switch?
Suppose we are currently at point A.
Basic set = { S1, S2, S3, S4 }, non-basic set = { x1, x2 }
Which variable should be the entering basic variable?
Which variable should be the leaving basic variable?
0
,
,
,
,
,
6
9
80
8
10
77
11
7
to
subject
175
150
Maximize
4
3
2
1
2
1
4
2
3
1
2
2
1
1
2
1
2
1
S
S
S
S
x
x
S
x
S
x
S
x
x
S
x
x
x
x
Z
29. 29
Which variables to switch?
x2 has a larger coefficient in the objective function, so
we should make x2 the entering basic variable. (x1
remains a non-basic variable and has the value zero)
If we keep increasing x2, other basic variables will
decrease. Which basic variable will become zero
first?
0
,
,
,
,
,
6
9
80
8
77
11
to
subject
175
150
Maximize
4
3
2
1
2
1
4
2
3
2
2
1
2
2
1
S
S
S
S
x
x
S
x
S
S
x
S
x
x
x
Z
30. 30
Which variables to switch?
If we take the ratio of the RHS value to the coefficient
of the entering basic variable (x2), the smallest non-
negative ratio can tell us which basic variable will
reach zero first if we keep increasing the value of x2.
So S4 should be made the leaving basic variable.
0
,
,
,
,
,
)
6
1
/
6
(
6
N/A
9
)
10
8
/
80
(
80
8
)
7
11
/
77
(
77
11
to
subject
175
150
Maximize
4
3
2
1
2
1
4
2
3
2
2
1
2
2
1
S
S
S
S
x
x
S
x
S
S
x
S
x
x
x
Z
Show Graph & Equations
Minimum
ratio test
31. 31
Which variables to switch?
So setting x1=0, S4=0 and solve the system of
equations would yield an augmented solution
correspond to point E.
Solving the system of equations at every corner point
is not very efficient. Simplex method offers an
efficient way to compute the value of each variable
by updating the equations incrementally.
