### SlideShare for iOS

by Linkedin Corporation

FREE - On the App Store

more justifications, more examples, more dimensions, more constraints, more better!

more justifications, more examples, more dimensions, more constraints, more better!

- Total Views
- 7,158
- Views on SlideShare
- 7,138
- Embed Views

- Likes
- 1
- Downloads
- 118
- Comments
- 0

http://www.slideshare.net | 19 |

http://translate.googleusercontent.com | 1 |

Uploaded via SlideShare as Adobe PDF

© All Rights Reserved

- 1. Lesson 28 (Sections 18.2–5) Lagrange Multipliers II Math 20 November 28, 2007 Announcements Problem Set 11 assigned today. Due December 5. next OH: Today 1–3 (SC 323) Midterm II review: Tuesday 12/4, 7:30-9:00pm in Hall E Midterm II: Thursday, 12/6, 7-8:30pm in Hall A
- 2. Outline A homework problem Restating the Method of Lagrange Multipliers Statement Justiﬁcations Second order conditions Compact feasibility sets Ad hoc arguments Analytic conditions Example: More than two variables More than one constraint
- 3. Problem 17.1.10 Problem Maximize the quantity f (x, y , z) = Ax a y b z c subject to the constraint that px + qy + rz = m. (Here A, a, b, c, p, q, r , m are positive constants.)
- 4. Problem 17.1.10 Problem Maximize the quantity f (x, y , z) = Ax a y b z c subject to the constraint that px + qy + rz = m. (Here A, a, b, c, p, q, r , m are positive constants.) Solution (By elimination) Solving the constraint for z in terms of x and y , we get m − px − qy z= r So we optimize the unconstrained function Aab x y (m − px − qy )c f (x, y ) = rc
- 5. We have ∂f (x, y ) A = c ax a−1 y b (m − px − qy )c + x a y b c(m − px − qy )c−1 (−p) x r A = c x a−1 y b (m − px − qy )c−1 [a(m − px − qy ) − cpx] r Likewise ∂f (x, y ) A = c x a y b−1 (m − px − qy )c−1 [b(m − px − qy ) − cqy ] y r So throwing out the critical points where x = 0, y = 0, or z = 0 (these give minimal values of f , not maximal), we get (a + c)px + aqy = am bpx + (b + c)qy = bm
- 6. This is a fun exercise in Cramer’s Rule: am aq 1 1 amq bm (b + c)q b b+c x= = (a + c)p aq a+c a pq bp (b + c)q b b+c amqc m a = = pq(ac + bc + c 2 ) p a+b+c It follows that m b m c y= z= q a+b+c r a+b+c If this is a utility-maximization problem subject to ﬁxed budget, the portion spent on each good ( px , for instance) is the relative m a degree to which that good multiplies utility ( a+b+c ).
- 7. Outline A homework problem Restating the Method of Lagrange Multipliers Statement Justiﬁcations Second order conditions Compact feasibility sets Ad hoc arguments Analytic conditions Example: More than two variables More than one constraint
- 8. Theorem (The Method of Lagrange Multipliers) Let f (x1 , x2 , . . . , xn ) and g (x1 , x2 , . . . , xn ) be functions of several variables. The critical points of the function f restricted to the set g = 0 are solutions to the equations: ∂f ∂g (x1 , x2 , . . . , xn ) = λ (x1 , x2 , . . . , xn ) for each i = 1, . . . , n ∂xi ∂xi g (x1 , x2 , . . . , xn ) = 0. Note that this is n + 1 equations in n + 1 variables x1 , . . . , xn , λ.
- 9. Graphical Justiﬁcation In two variables, the critical points of f restricted to the level curve g = 0 are found when the tangent to the the level curve of f is parallel to the tangent to the level curve g = 0.
- 10. These tangents have slopes dy fx dy gx =− =− and dx fy dx gy f g
- 11. These tangents have slopes dy fx dy gx =− =− and dx fy dx gy f g So they are equal when fy fx g f = x =⇒ x = fy gy gx gy or fx = λgx fy = λgy
- 12. Symbolic Justiﬁcation Suppose that we can use the relation g (x1 , . . . , xn ) = 0 to solve for xn in terms of the the other variables x1 , . . . , xn−1 , after making some choices. Then the critical points of f (x1 , . . . , xn ) are unconstrained critical points of f (x1 , . . . , xn (x1 , . . . , xn−1 )). f x1 x2 xn ··· xn−1 x1 x2 ···
- 13. Now for any i = 1, . . . , n − 1, ∂f ∂f ∂f ∂xn = + ∂xi ∂xi ∂xn ∂xi g g ∂f ∂f ∂g /∂xi − = ∂xi ∂xn ∂g /∂xn ∂f If = 0, then ∂xi g ∂f /∂xi ∂g /∂xi ∂f /∂xi ∂f /∂xn ⇐⇒ = = ∂f /∂xn ∂g /∂xn ∂g /∂xi ∂g /∂xn ∂f ∂g So as before, =λ for all i. ∂xi ∂xi
- 14. Another perspective To ﬁnd the critical points of f subject to the constraint that g = 0, create the lagrangian function L = f (x1 , x2 , . . . , xn ) − λg (x1 , x2 , . . . , xn ) If L is restricted to the set g = 0, L = f and so the constrained critical points are unconstrained critical points of L . So for each i, ∂L ∂f ∂g = 0 =⇒ =λ . ∂xi ∂xi ∂xi But also, ∂L = 0 =⇒ g (x1 , x2 , . . . , xn ) = 0. ∂λ
- 15. Outline A homework problem Restating the Method of Lagrange Multipliers Statement Justiﬁcations Second order conditions Compact feasibility sets Ad hoc arguments Analytic conditions Example: More than two variables More than one constraint
- 16. Second order conditions The Method of Lagrange Multipliers ﬁnds the constrained critical points, but doesn’t determine their “type” (max, min, neither). So what then?
- 17. A dash of topology Cf. Sections 17.2–3 Deﬁnition A subset of Rn is called closed if it includes its boundary.
- 18. A dash of topology Cf. Sections 17.2–3 Deﬁnition A subset of Rn is called closed if it includes its boundary. x2 + y2 ≤ 1 x2 + y2 ≤ 1 y ≥0 not closed closed closed Basically, if a subset is described by ≤ or ≥ inequalities, it is closed.
- 19. Deﬁnition A subset of Rn is called bounded if it is contained within some ball centered at the origin. x2 + y2 ≤ 1 x2 + y2 ≤ 1 y ≥0 bounded not bounded bounded
- 20. Deﬁnition A subset of Rn is called compact if it is closed and bounded. x2 + y2 ≤ 1 x2 + y2 ≤ 1 y ≥0 not compact not compact compact
- 21. Optimizing over compact sets Theorem (Compact Set Method) To ﬁnd the extreme values of function f on a compact set D of Rn , it suﬃces to ﬁnd the (unconstrained) critical points of f “inside” D the (constrained) critical points of f on the “boundary” of D.
- 22. Ad hoc arguments If D is not compact, sometimes it’s still easy to argue that as x gets farther away, f becomes larger, or smaller, so the critical points are “obviously” maxes, or mins.
- 23. Ad hoc arguments If D is not compact, sometimes it’s still easy to argue that as x gets farther away, f becomes larger, or smaller, so the critical points are “obviously” maxes, or mins. (Example later)
- 24. Analytic conditions Recall Equation 16.13, cf. Section 18.4 For the two-variable constrained optimization problem, we have (look in the book if you want the gory details): Lλλ Lλx Lλy 0 gx gy d 2f fxy − λgxy = Lxλ Lxx Lxy − λgxx = gx fxx dx 2 Ly λ Lyx Lyy g − λgyx gyy − λgyy gy fyx The critical point is a local max if this determinant is negative, and a local min if this is positive. The matrix on the right is the Hessian of the Lagrangian. But there is still a distinction between this and the unconstrained case. The constrained extrema are critical points of the Lagrangian, not extrema. Don’t worry too much about this!
- 25. Outline A homework problem Restating the Method of Lagrange Multipliers Statement Justiﬁcations Second order conditions Compact feasibility sets Ad hoc arguments Analytic conditions Example: More than two variables More than one constraint
- 26. Problem 17.1.10 Problem Maximize the quantity f (x, y , z) = Ax a y b z c subject to the constraint that px + qy + rz = m. (Here A, a, b, c, p, q, r , m are positive constants.)
- 27. Problem 17.1.10 Problem Maximize the quantity f (x, y , z) = Ax a y b z c subject to the constraint that px + qy + rz = m. (Here A, a, b, c, p, q, r , m are positive constants.) Solution The Lagrange equations are Aax a−1 y b z c = λp Abx a y b−1 z c = λq Acx a y b z c−1 = λr We rule out any solution with x, y , z, or λ equal to 0 (they will minimize f , not maximize it).
- 28. Dividing the ﬁrst two equations gives ay p bp = =⇒ y = x bx q aq Dividing the ﬁrst and last equations gives az p cp = =⇒ z = x cx r ar Plugging these into the equation of constraint gives bp cp m a px + x + x = m =⇒ x = a a p a+b+c
- 29. Outline A homework problem Restating the Method of Lagrange Multipliers Statement Justiﬁcations Second order conditions Compact feasibility sets Ad hoc arguments Analytic conditions Example: More than two variables More than one constraint
- 30. General method for more than one constraint If we are optimizing f (x1 , . . . , xn ) subject to gj (x1 , . . . , xn ) ≡ 0, j = 1, . . . , m we need multiple lambdas for them. The new Lagrangian is m L (x1 , . . . , xn ) = f (x1 , . . . , xn ) − λj gj (x1 , . . . , xn ) j=1 ∂L ∂L The conditions are that = 0 and = 0 for all i and j. In ∂xi ∂λj other words, ∂f ∂g1 ∂gm + · · · + λm = λ1 (all i) ∂xi ∂xi ∂xi gj (x1 , . . . , xn ) = 0 (all j)
- 31. Example Find the minimum distance between the curves xy = 1 and x + 2y = 1.
- 32. Example Find the minimum distance between the curves xy = 1 and x + 2y = 1. Reframing this, we can minimize f (x, y , u, v ) = (x − u)2 + (y − v )2 subject to the constraints xy − 1 = 0 u + 2v = 1.
- 33. • • • • xy = 1 • • • • x + 2y = 1
- 34. The Lagrangian is L = (x − u)2 + (y − v )2 − λ(xy − 1) − µ(u + 2v − 1) So the Lagrangian equations are 2(x − u) = λy −2(x − u) = µ 2(y − v ) = λx −2(y − v ) = 2µ Dividing the two λ equations and the two µ equations gives x −u x −u y 1 = =. y −v y −v x 2 Since the left-hand-sides are the same, we have 2y =√ Since x. √ 1 1 √ , or x = − 2, y = − √ xy = 1, we can say either x = 2, y = 2 2
- 35. √ 1 Suppose x = 2, y = √. Then 2 √ √ 2−u 1 32 =⇒ 2u − v = = 1 2 2 √ −v 2 This along with u + 2v = 1 gives √ √ 1 1 4−3 2 u= 1+3 2 v= 5 10 √ 1 If we instead choose x = − 2, y = − √2 , we get √ 1 1 3 2+ √ 1−3 2 u= v= 5 5 2
- 36. √ 1 9−4 2 5 • • xy = 1 • • x + 2y = 1 √ 1 9+4 2 5
- 37. Because f gets larger as x, y , u, and v get larger, the absolute minimum is the smaller of these two critical values. So the √ 1 minimum distance is 5 9 − 4 2 .

Full NameComment goes here.Chuan-Yi Kuoat Nationa Tsing Hua University Tagslagrange2 years ago