7.3_Nonlinear Programming-LagrangeExamples.pptx

Nonlinear Programming:
Lagrange Multiplier Examples
PROF. MICHAEL HYLAND
WINTER 2020
CEE 111
Methods IV: Systems Analysis and Decision-Making

Today’s Outline
Review
Lagrangian Method
◦ Introduction
◦ Formulation
New Material
Lagrangian Method
◦ Examples

Announcements
Homework 4
◦ Due: Last day of Class: March 13th
Design Project
◦ Due: Wednesday March 18
Final Exam
◦ Monday March 16th
Self + Peer Evaluation on Group Assignments
◦ Due: Wednesday

Design Problem
Goals & Objectives
Constraints in Achieving Goals & Objectives
Linear Problems
Nonlinear
Problems
Continuous
Variables
Binary/Integer
Variables
Branch &
Bound
Single
Objective
Multiple
Objectives
Weighted
Single Goal
Primary Goal
+ Bounded
Secondary
Goals
Single Objective Linear Programming Problem
Exact Solutions
Approximate
Solutions
Lagrangian
+ Calculus
Nonlinear System of
Algebraic Equations
Kuhn-
Tucker
Conditions
Linearization
Search
Techniques
Taylor
Series
Piecewise
Linearization
Gradient
Search
Single Objective Linear
Programming Problem
4

Lagrange Multipliers
A technique to handle constrained
nonlinear programs
Basic Idea:
◦ Take original nonlinear constrained
program with objective function
and constraints
◦ Move all the constraints into the
objective function
◦ Multiply each constraint by Lagrangian
multiplier
◦ Obtain one objective function with
no constraints
Example with 3 variables and 2
constraints
◦ Ideas extend to more variables and
more constraints
𝑀𝑖𝑛 𝑜𝑟 𝑀𝑎𝑥 𝑍 = 𝑓 𝑥1, 𝑥2, 𝑥3
Subject to:
𝑔1 𝑥1, 𝑥2, 𝑥3 = 𝑏1
𝑔2 𝑥1, 𝑥2, 𝑥3 = 𝑏2

𝑀𝑖𝑛 𝑜𝑟 𝑀𝑎𝑥 𝑍 =
𝑓 𝑥1, 𝑥2, 𝑥3
Subject to:
𝑔1 𝑥1, 𝑥2, 𝑥3 = 𝑏1
𝑔2 𝑥1, 𝑥2, 𝑥3 = 𝑏2
Now, remember, we
want one objective
function equation
and no constraints
Let 𝜆1and 𝜆2 be the Lagrange Multipliers associated with
the first and second constraints, respectively.
Then:
𝐿 = 𝑓 𝑥1, 𝑥2, 𝑥3 + 𝜆1 𝑔1 𝑥1, 𝑥2, 𝑥3 − 𝑏1 + 𝜆2 𝑔2 𝑥1, 𝑥2, 𝑥3 − 𝑏2
Where, 𝐿 is the Lagrangian Function

𝐿 = 𝑓 𝑥1, 𝑥2, 𝑥3 + 𝜆1 𝑔1 𝑥1, 𝑥2, 𝑥3 − 𝑏1 + 𝜆2 𝑔2 𝑥1, 𝑥2, 𝑥3 − 𝑏2
We now have one objective function and no constraints
◦ Therefore, we can apply calculus methods from unconstrained optimization
We need to take five partial derivatives to determine necessary
conditions of a stationary point
◦ 𝑥1, 𝑥2, 𝑥3, 𝜆1, 𝜆2
𝜆1, 𝜆2 are now decision variables in the unconstrained optimization
problem

Lagrangian Method
Set the 5 Partial Derivatives equal to zero
𝜕𝐿
𝜕𝑥1
=
𝜕𝑓 𝑥1, 𝑥2, 𝑥3
𝜕𝑥1
+ 𝜆1
𝜕 𝑔1 𝑥1, 𝑥2, 𝑥3 − 𝑏1
𝜕𝑥1
+ 𝜆2
𝜕 𝑔2 𝑥1, 𝑥2, 𝑥3 − 𝑏2
𝜕𝑥1
= 0
𝜕𝐿
𝜕𝑥2
=
𝜕𝑓 𝑥1, 𝑥2, 𝑥3
𝜕𝑥2
+ 𝜆1
𝜕 𝑔1 𝑥1, 𝑥2, 𝑥3 − 𝑏1
𝜕𝑥2
+ 𝜆2
𝜕 𝑔2 𝑥1, 𝑥2, 𝑥3 − 𝑏2
𝜕𝑥2
= 0
𝜕𝐿
𝜕𝑥3
=
𝜕𝑓 𝑥1, 𝑥2, 𝑥3
𝜕𝑥3
+ 𝜆1
𝜕 𝑔1 𝑥1, 𝑥2, 𝑥3 − 𝑏1
𝜕𝑥3
+ 𝜆2
𝜕 𝑔2 𝑥1, 𝑥2, 𝑥3 − 𝑏2
𝜕𝑥3
= 0
𝜕𝐿
𝜕𝜆1
= 𝑔1 𝑥1, 𝑥2, 𝑥3 − 𝑏1 = 0
𝜕𝐿
𝜕𝜆2
= 𝑔2 𝑥1, 𝑥2, 𝑥3 − 𝑏2 = 0

Review
LAGRANGIAN METHOD: PROPERTIES + EXAMPLES

Solution to Constrained Non-linear Optimization Problems using the Lagrangian
Consider the general constrained nonlinear optimization problem below:
1 2
max (or min) ( , , , )
n
Z f x x x

 
1 2 1 2 1 2 1 2
1
, , , ; , , , ) ( , , , ) ( , , , )
m
n m n i i i n
i
x x x f x x x b g x x x
   

  

L(
Form the Langrangian, L, as:
1 1 2 1
2 1 2 2
1 2
subject to:
( , , , )
( , , , )
( , , , )
n
n
m n m
g x x x b
g x x x b
g x x x b



Notice
1 1 1 2
2 2 1 2
1 2
( , , , ) 0
( , , , ) 0
( , , , ) 0
n
n
m m n
b g x x x
b g x x x
b g x x x
 
 
 
0
Called Lagrange Multipliers
1 1 2 1
2 1 2 2
1 2
( , , , ) 0
( , , , ) 0
( , , , ) 0
n
n
m n m
g x x x b
g x x x b
g x x x b
 
 
 
 
1 2 1 2 1 2 1 2
1
, , , ; , , , ) ( , , , ) ( , , , )
m
n m n i i n i
i
x x x f x x x g x x x b
   

  

L(
Doesn’t matter
11

 
1 2 1 2 1 2 1 2
1
min (or max) , , , ; , , , ) ( , , , ) ( , , , )
m
n m n i i i n
i
   

  

L(
Next, consider the unconstrained optimization problem:
The stationary (extreme) points of L are determined by
* * * * * *
1 2 1 2
, , , ; , , , )
n m
x x x   
(
* * * * * *
1 2 1 2
* * * * * *
1 2 1 2
, , , ; , , , )
, , , ; , , , )
0 ; 1,2, ,
0 ; 1,2, ,
i
j
n m
n m
x x x
x x x
i n
x
j m
  
  


 


 

(
(
L
L
Note that:
1 2
1 1 2 1
2 1 2 2
1 2
max (or min) ( , , , )
subject to:
( , , , )
( , , , )
( , , , )
n
n
n
m n m
Z f x x x
g x x x b
g x x x b
g x x x b




* * *
1 2
* * *
1 2
* * * * * *
1 2 1 2
, , , ; , , , )
0 ( , , , ); 1,2, ,
or
( , , , ) ; 1,2, ,
i i n
j
i n i
n m
x x x
b g x x x j m
g x x x b j m
  


   

 
(
L
Conclusion: Solution to satisfies the constraints of the original problem.
12

 
1 2 1 2 1 2 1 2
1
min (or max) , , , ; , , , ) ( , , , ) ( , , , )
m
n m n i i i n
i
   

  

L(
Next, consider the unconstrained optimization problem:
The stationary (extreme) points of L are determined by
* * * * * *
1 2 1 2
, , , ; , , , )
n m
x x x   
(
* * * * * *
1 2 1 2
* * * * * *
1 2 1 2
, , , ; , , , )
, , , ; , , , )
0 ; 1,2, ,
0 ; 1,2, ,
i
j
n m
n m
x x x
x x x
i n
x
j m
  
  


 


 

(
(
L
L
Note also:
 
1 2 1 2 1 2 1 2
1
, , , ; , , , ) ( , , , ) ( , , , )
m
n m n i i i n
i
   

  

L(
= 0 at optimum
1 2 1 2 1 2
, , , ; , , , ) ( , , , )
n m n
x x x f x x x
   
L(
1 2
( , , , )
n
Z f x x x

But
1 2
max (or min) ( , , , )
n
Z f x x x
 
Conclusion: solving either produces the same value
min (or max) or min (or max) Z
L
guaranteed by
13

1 2
1 1 2 1
2 1 2 2
1 2
max (or min) ( , , , )
subject to:
( , , , )
( , , , )
( , , , )
n
n
n
m n m
Z f x x x
g x x x b
g x x x b
g x x x b




 
1 2 1 2 1 2 1 2
1
min (or max) , , , ; , , , ) ( , , , ) ( , , , )
m
n m n i i i n
i
   

  

L(
Conclusion:
are equivalent representations of the
same problem
Constrained Optimization Problem
Unconstrained Lagrangian Optimization Problem
14

Second Order Conditions
2 2
2 2
0 and 0 , 1,2, , ; 0 , 1,2, Relative Mini u
, m m
i j
i n j m
x 
 
     
 

D H
L L
2 2
2 2
0 and 0 , 1,2, , ; 0 , 1,2, Relative Maxi u
, m m
i j
i n j m
x 
 
     
 

D H
L L
Saddle Po nt
0 i
  
D H
Inconclusive; need higher order t st
0 e s
  
D H
15

Select Location of a Waste Processing Plant
Linear Example
1 2
1 2
1 2
3 5 3000
3 1500
3 2 2100
x x
x x
x x
 
 
 
1 2
0, 0
x x
 
Constraints
1 2
Minimize 750,000 200 180
Z x x
  
Objective
16

Minimize: 1 2
750,000 200 180
  
Z x x
subject to:
1 2
1 2
1 2
3 5 3000
3 1500
3 2 2100
x x
x x
x x
 
 
 
1 2
0, 0
x x
 
Example Model
1 2 1
1 2 2
1 2 3
3 5 3000
3 1500
3 2 2100
x x S
x x S
x x S
  
  
  
1 2 1 2 3
0, 0, 0, 0, 0
x x S S S
    
17

Minimize: 1 2
750,000 200 180
  
Z x x
subject to:
1 2 1
1 2 2
1 2 3
3 5 3000
3 1500
3 2 2100
x x S
x x S
x x S
  
  
  
1 2 1 2 3
0, 0, 0, 0, 0
x x S S S
    
2 2 2
1 2 1
2 2 2
1 2 2
2 2 2
1 2 3
3 5 3000
3 1500
3 2 2100
x x S
x x S
x x S
  
  
  
2 2
1 2
750,000 200 180
Z x x
  
subject to:
Minimize:
Not needed, since by definition
2 2 2 2 2
1 1 2 2 1 1 2 2 3 3
, , , ,
x x x x S S S S S S
    
Let
2
1
2
2
2
1
2
2
2
3
0
0
0
0
0
x
x
S
S
S





Example Model
Can’t have these inequalities in a Lagrangian formulation
18

Lagrangian
Consider the following function:
1 2
Min 750,000 200 180
Z x x
  
1 2 1
1 2 2
1 2 3
3 5 3000
3 1500
3 2 2100
x x S
x x S
x x S
  
  
  
1 2 1 2 3
0, 0, 0, 0, 0
x x S S S
    
1 2 1
1 2 2
1 2 3
3 5 3000 0
3 1500 0
3 2 2100 0
x x S
x x S
x x S
   
   
   
 
 
 
2 2
1 2
2 2 2
1 1 2 1
2 2 2
2 1 2 2
2 2 2
3 1 2 3
2 2 2 2 2
1 1 2 2 1 1 2 2 3 3
Min 750,000 200 180
3 5 3000
3 1500
3 2 2100
, , , ,
x x
x x S
x x S
x x S
x x x x S S S S S S



  
    
    
    
    
L
Note that:
Min L = Min Z 2
1
x
 
 
 
2 2
1 2
1
2
3
2 2
1 2
Min 750,000 200 180
0
0
0
750,000 200 180
x x
x x



  
 
 
 
  
L
2 2 2
1 2 1
2 2 2
1 2 2
2 2 2
1 2 3
3 5 3000 0
3 1500 0
3 2 2100 0
x x S
x x S
x x S
   
   
   
2 2
1 2
Min 750,000 200 180
Z x x
  
19

Lagrangian
1 2 1 2 3
1 2 3
0, 0, 0, 0, 0, 0, 0, 0
x x S S S   
       
       
    
  
L L L L L L L L
To find the values of 1 2 1 2 3 1 2 3
, , , , , , ,
x x S S S    that minimize L
Set
 
 
 
2 2
1 2
2 2 2
1 1 2 1
2 2 2
2 1 2 2
2 2 2
3 1 2 3
Min 750,000 200 180
3 5 3000
3 1500
3 2 2100
x x
x x S
x x S
x x S



  
    
    
    
L
Unconstrained optimization
problem
20

1 1 1 2 1 3 1
1
2 1 2 2 2 3 2
2
1 1
1
2 2
2
3 3
3
2 2 2
1 2 1
1
2 2 2
1 2 2
2
2 2 2
1 2 3
3
400 6 2 6 0
360 10 6 4 0
2 0
2 0
2 0
3 5 3000 0
3 1500 0
3 2
x x x x
x
x x x x
x
S
S
S
S
S
S
x x S
x x S
x x S
  
  







     


     


 


 


 


    


    


   

L
L
L
L
L
L
L
L
2100 0

Lagrangian
 
 
 
2 2
1 2
2 2 2
1 1 2 1
2 2 2
2 1 2 2
2 2 2
3 1 2 3
Min 750,000 200 180
3 5 3000
3 1500
3 2 2100
x x
x x S
x x S
x x S



  
    
    
    
L
Factor out 1
x
Factor out 2
x
21

 
 
1 2 3 1
1
1 2 3 2
2
1 1
1
2 2
2
3 3
3
2 2 2
1 2 1
1
2 2 2
1 2 2
2
2 2 2
1 2 3
3
400 6 2 6 0
360 10 6 4 0
2 0
2 0
2 0
3 5 3000 0
3 1500 0
3 2 2100 0
x
x
x
x
S
S
S
S
S
S
x x S
x x S
x x S
  
  







     


     


 


 


 


    


    


    

L
L
L
L
L
L
L
L
Lagrangian
 
 
 
2 2
1 2
2 2 2
1 1 2 1
2 2 2
2 1 2 2
2 2 2
3 1 2 3
Min 750,000 200 180
3 5 3000
3 1500
3 2 2100
x x
x x S
x x S
x x S



  
    
    
    
L
22

Lagrangian
 
 
 
2 2
1 2
2 2 2
1 1 2 1
2 2 2
2 1 2 2
2 2 2
3 1 2 3
Min 750,000 200 180
3 5 3000
3 1500
3 2 2100
x x
x x S
x x S
x x S



  
    
    
    
L
Solving these 8 equations in 8 unknowns
minimizes
 
 
1 2 3 1
1
1 2 3 2
2
1 1
1
2 2
2
3 3
3
2 2 2
1 2 1
1
2 2 2
1 2 2
2
2 2 2
1 2 3
3
400 6 2 6 0
360 10 6 4 0
2 0
2 0
2 0
3 5 3000 0
3 1500 0
3 2 2100 0
x
x
x
x
S
S
S
S
S
S
x x S
x x S
x x S
  
  







     


     


 


 


 


    


    


    

L
L
L
L
L
L
L
L
23

Lagrangian
 
 
 
2 2
1 2
2 2 2
1 1 2 1
2 2 2
2 1 2 2
2 2 2
3 1 2 3
Min 750,000 200 180
3 5 3000
3 1500
3 2 2100
x x
x x S
x x S
x x S



  
    
    
    
L
These 3 equations ensure that
original constraints are satisfied
1 2 1
1 2 2
1 2 3
3 5 3000
3 1500
3 2 2100
x x S
x x S
x x S
  
  
  
 
 
1 2 3 1
1
1 2 3 2
2
1 1
1
2 2
2
3 3
3
2 2 2
1 2 1
1
2 2 2
1 2 2
2
2 2 2
1 2 3
3
400 6 2 6 0
360 10 6 4 0
2 0
2 0
2 0
3 5 3000 0
3 1500 0
3 2 2100 0
x
x
x
x
S
S
S
S
S
S
x x S
x x S
x x S
  
  







     


     


 


 


 


    


    


    

L
L
L
L
L
L
L
L
24

Lagrangian
 
 
 
2 2
1 2
2 2 2
1 1 2 1
2 2 2
2 1 2 2
2 2 2
3 1 2 3
Min 750,000 200 180
3 5 3000
3 1500
3 2 2100
x x
x x S
x x S
x x S



  
    
    
    
L
These definitions
ensure that the solution to the
unconstrained problem satisfies
2 2 2 2 2
1 1 2 2 1 1 2 2 3 3
, , , ,
x x x x S S S S S S
    
1 2 1 2 3
0, 0, 0, 0, 0
x x S S S
    
 
 
1 2 3 1
1
1 2 3 2
2
1 1
1
2 2
2
3 3
3
2 2 2
1 2 1
1
2 2 2
1 2 2
2
2 2 2
1 2 3
3
400 6 2 6 0
360 10 6 4 0
2 0
2 0
2 0
3 5 3000 0
3 1500 0
3 2 2100 0
x
x
x
x
S
S
S
S
S
S
x x S
x x S
x x S
  
  







     


     


 


 


 


    


    


    

L
L
L
L
L
L
L
L
25

Lagrangian
 
 
 
2 2
1 2
2 2 2
1 1 2 1
2 2 2
2 1 2 2
2 2 2
3 1 2 3
Min 750,000 200 180
3 5 3000
3 1500
3 2 2100
x x
x x S
x x S
x x S



  
    
    
    
L
These 3 equations ensure that
1 2
Min 750,000 200 180
Z x x
  
Min Min Z
L =
 
 
1 2 3 1
1
1 2 3 2
2
1 1
1
2 2
2
3 3
3
2 2 2
1 2 1
1
2 2 2
1 2 2
2
2 2 2
1 2 3
3
400 6 2 6 0
360 10 6 4 0
2 0
2 0
2 0
3 5 3000 0
3 1500 0
3 2 2100 0
x
x
x
x
S
S
S
S
S
S
x x S
x x S
x x S
  
  







     


     


 


 


 


    


    


    

L
L
L
L
L
L
L
L
26

Lagrangian
Solution to these equations solves
Minimize: 1 2
750,000 200 180
  
Z x x
subject to:
1 2
1 2
1 2
3 5 3000
3 1500
3 2 2100
x x
x x
x x
 
 
 
1 2
0, 0
x x
 
original problem
 
 
1 2 3 1
1
1 2 3 2
2
1 1
1
2 2
2
3 3
3
2 2 2
1 2 1
1
2 2 2
1 2 2
2
2 2 2
1 2 3
3
400 6 2 6 0
360 10 6 4 0
2 0
2 0
2 0
3 5 3000 0
3 1500 0
3 2 2100 0
x
x
x
x
S
S
S
S
S
S
x x S
x x S
x x S
  
  







     


     


 


 


 


    


    


    

L
L
L
L
L
L
L
L
27

Microsoft Excel 10.0 Answer Report
Worksheet: [CEE111_Waste_Plant_Location_Dual.xls]Primal
Report Created: 1/31/2006 9:44:11 AM
Target Cell (Min)
Cell Name Original Value Final Value
$E$15 Mn Z' 0 -154000
Adjustable Cells
Cell Name Original Value Final Value
$C$14 Value x1 0 500
$D$14 Value x2 0 300
Constraints
Cell Name Cell Value Formula Status Slack
$E$10 Prop Line -3000 $E$10>=$G$10 Binding 0
$E$11 d<=16 -1400 $E$11>=$G$11 Not Binding 100
$E$12 Green Belt -2100 $E$12>=$G$12 Binding 0
$C$14 Value x1 500 $C$14>=0 Not Binding 500
$D$14 Value x2 300 $D$14>=0 Not Binding 300
Microsoft Excel 10.0 Sensitivity Report
Worksheet: [CEE111_Waste_Plant_Location_Dual.xls]Primal
Report Created: 1/31/2006 9:44:11 AM
Adjustable Cells
Final Reduced
Cell Name Value Gradient
$C$14 Value x1 500 0
$D$14 Value x2 300 0
Constraints
Final Lagrange
Cell Name Value Multiplier
$E$10 Prop Line -3000 15.55555556
$E$11 d<=16 -1400 0
$E$12 Green Belt -2100 51.11111111
1
2
500
300
x
x


1
2
3
0
100
0
S
S
S



1
2
3
15.55555
0
51.11111






Original Problem: Excel Solver Solution
28

 
 
1
2
1
2
3
1
2
400 6(15.55555) 2(0) 6(51.11111) 22.36 0
360 10(15.55555) 6(0) 4(51.11111) 17.32 0
2(15.55555)(0) 0
2(0)(10) 0
2(51.11111)(0) 0
3(500) 5(300) 0 3000 0
500 3(
x
x
S
S
S



      


      


 


 


 


    


 

L
L
L
L
L
L
L
3
300) 100 1500 0
3(500) 2(300) 0 2100 0

  

    

L
Lagrangian
Solution to these equations solves
original problem
1
2
1
2
3
1
2
3
500
300
0
100
0
15.55555
0
51.11111
x
x
S
S
S











2 2 2 2 2
1 1 2 2 1 1 2 2 3 3
, , , ,
x x x x S S S S S S
    
1
2
1
2
3
1
2
3
500 22.36
300 17.32
0
100 10
0
15.55555
0
51.11111
x
x
S
S
S



 
 

 




29

Interpretation
30
Without proof, we note that the Lagrange multiplier i
represent the change in the objective function at the
optimum caused by a change in the right-hand side
constant of constraint “i”:
The Lagrange multiplier has therefore the same
meaning as the dual variable in linear programming.
i
i
f
b





Based on Problem 13.13 in the textbook
Using the Lagrangian approach, solve the following problem:
Solve again using calculus with substitution – as the textbook does
31
1 1
Subject to: 1
Min
x y
x y

 
Another Example

32
S
x
x
y y
z
z
z
z
S
Example 2: Sheet metal forming

Example 2: Sheet metal forming
Suppose you have a square sheet of metal with side S that
is to be cut and folded into a topless box of maximum
volume (see Figure 13.10 page 358; shown on previous
page). It is clear that we must have x=y but we keep them
separate here. Problem: find the dimensions of cutting x, y,
and z.
The problem can be written:
Maximize xyz
Subject to: x + 2z = S
y + 2z = S
33

Maximize xyz
Subject to: x + 2z = S
y + 2z = S
The Lagrangian function is:
L= xyz + 1 (S - x - 2z) + 2 (S - y - 2z)
Write the first order necessary conditions and solve this
problem.
34
Example 2: Sheet metal
forming

7.3_Nonlinear Programming-LagrangeExamples.pptx

Recommended

Recommended

More Related Content

Similar to 7.3_Nonlinear Programming-LagrangeExamples.pptx

Similar to 7.3_Nonlinear Programming-LagrangeExamples.pptx (20)

Recently uploaded

Recently uploaded (20)

7.3_Nonlinear Programming-LagrangeExamples.pptx