Upcoming SlideShare
×

# An Introduction to Linear Programming

4,875 views

Published on

My talk about linear programming in NTU's APEX Club in NTU, Singapore in 2007. The club is for people who are keen on participating in ACM International Collegiate Programming Contests organized by IBM annually.

9 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

Views
Total views
4,875
On SlideShare
0
From Embeds
0
Number of Embeds
36
Actions
Shares
0
453
0
Likes
9
Embeds 0
No embeds

No notes for slide

### An Introduction to Linear Programming

1. 1. An Introduction to Linear Programming Pham Minh Tri Ph.D. Candidate and Project Officer School of Computer Engineering 18 Jan 2008 presented by
2. 2. An example <ul><li>Suppose a candy manufacturer produces chocolate covered peanuts called Choconuts , and chocolate and caramel covered peanuts called Carmonuts . </li></ul><ul><ul><li>Choconuts require 3 pounds of chocolate and 1.5 pounds of peanuts to manufacture a case and sell for \$9.00 per case . </li></ul></ul><ul><ul><li>Carmonuts require 2 pounds of chocolate and 3 pounds of peanuts to manufacture a case and sell for \$12.00 per case . </li></ul></ul><ul><li>There are 300 pounds of chocolate and 250 pounds of peanuts available in the current shipment </li></ul><ul><li>How many cases of each should the company produce to maximize their profit? </li></ul>
3. 3. An example <ul><li>First, let’s define our variables: </li></ul><ul><ul><li>X = number of cases of Choconuts to produce </li></ul></ul><ul><ul><li>Y = number of cases of Carmonuts to produceSuppose a candy </li></ul></ul>
4. 4. An example <ul><li>The Choconuts require 3 pounds of chocolate and the Carmonuts require 2 pounds of chocolate and we only have 300 pounds of chocolate . So, the inequality that represents </li></ul><ul><li>3 X + 2 Y <= 300 </li></ul><ul><li>The Choconuts require 1.5 pounds of peanuts and the Carmonuts require 3 pounds of peanuts and we only have 250 pounds of peanuts . So, the inequality that represents </li></ul><ul><li>1.5 X + 3 Y <= 250 </li></ul>
5. 5. An example <ul><li>Now, since we cannot produce a negative amount of either candy, we also have these constraints: </li></ul><ul><ul><li>X >= 0 </li></ul></ul><ul><ul><li>Y >= 0 </li></ul></ul><ul><li>Let’s graph these inequalities; </li></ul>
6. 6. An example
7. 7. An example <ul><li>X >= 0 </li></ul>
8. 8. An example <ul><li>X >= 0 </li></ul><ul><li>Y >= 0 </li></ul>
9. 9. An example <ul><li>X >= 0 </li></ul><ul><li>Y >= 0 </li></ul><ul><li>3 X + 2 Y <= 300 </li></ul>(0,83.3) (166.7,0)
10. 10. An example <ul><li>X >= 0 </li></ul><ul><li>Y >= 0 </li></ul><ul><li>3 X + 2 Y <= 300 </li></ul><ul><li>1.5 X + 3 Y <= 250 </li></ul>(0,83.3) (100,0) (0,150) (166.7,0)
11. 11. An example <ul><li>X >= 0 </li></ul><ul><li>Y >= 0 </li></ul><ul><li>3 X + 2 Y <= 300 </li></ul><ul><li>1.5 X + 3 Y <= 250 </li></ul>(0,83.3) (100,0) (0,150) (166.7,0) Feasible region Feasible region is always a convex hull
12. 12. An example <ul><li>A Choconut sells for \$9.00 and a Carmonut sells for \$12.00 . How many cases of each should the company produce to maximize their profit? </li></ul><ul><li>Profit: P( X , Y ) = 9 X + 12 Y </li></ul>
13. 13. An example <ul><li>Profit: P( X , Y ) = 9 X + 12 Y </li></ul><ul><li>Can be applied to points in region F only </li></ul><ul><li>Problem: find point ( X , Y ) in region F such that </li></ul><ul><ul><li>P ( X , Y )  max </li></ul></ul>(0,83.3) (100,0) (0,150) (166.7,0) Feasible Region F
14. 14. An example <ul><li>Profit: P( X , Y ) = 9 X + 12 Y </li></ul>(0,83.3) (100,0) (0,150) (166.7,0) Feasible Region F Direction Of profit increase
15. 15. An example <ul><li>Profit: P( X , Y ) = 9 X + 12 Y </li></ul>(0,83.3) (100,0) (0,150) (166.7,0) Feasible Region F Direction Of profit increase P = 0
16. 16. An example <ul><li>Profit: P( X , Y ) = 9 X + 12 Y </li></ul>(0,83.3) (100,0) (0,150) (166.7,0) Feasible Region F Direction Of profit increase
17. 17. An example <ul><li>Profit: P( X , Y ) = 9 X + 12 Y </li></ul>(0,83.3) (100,0) (0,150) (166.7,0) Feasible Region F Direction Of profit increase
18. 18. An example <ul><li>Profit: P( X , Y ) = 9 X + 12 Y </li></ul>(0,83.3) (100,0) (0,150) (166.7,0) Feasible Region F Direction Of profit increase
19. 19. An example <ul><li>Profit: P( X , Y ) = 9 X + 12 Y </li></ul>(0,83.3) (100,0) (0,150) (166.7,0) Feasible Region F Direction Of profit increase
20. 20. An example <ul><li>Profit: P( X , Y ) = 9 X + 12 Y </li></ul><ul><li>Optimal solution: </li></ul><ul><ul><li>At ( X , Y )=(66,50) </li></ul></ul><ul><ul><li>P(66,50) = 1192 </li></ul></ul>(0,83.3) (100,0) (0,150) (166.7,0) Feasible Region F Direction Of profit increase P = 1192 (66,50) An optimal solution, always locates at one of the corners
21. 21. 2-variable Linear Programming <ul><li>Find (x 1 ,x 2 ) to: </li></ul><ul><li>Maximize: c 1 x 1 + c 2 x 2 </li></ul><ul><li>Constraints: </li></ul><ul><ul><li>x 1 >= 0 and x 2 >= 0 </li></ul></ul><ul><ul><li>a 1,1 x 1 + a 1,2 x 2 <= b 1 </li></ul></ul><ul><ul><li>a 2,1 x 1 + a 2,2 x 2 <= b 2 </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><ul><li>a M,1 x 1 + a M,2 x 2 <= b M </li></ul></ul>Simple Algorithm: - Find the feasible region F (convex hull) - Choose the corner that has the highest profit
22. 22. What about over 2 variables? <ul><li>Same idea, except that feasible region F lives in a higher dimensional space: </li></ul><ul><ul><li>2-variable linear programming: F = 2D convex hull </li></ul></ul><ul><ul><li>3-variable linear programming: F = 3D convex hull </li></ul></ul><ul><ul><li>4-variable linear programming: F = 4D convex hull </li></ul></ul><ul><ul><li>etc </li></ul></ul>
23. 23. Linear Programming in general <ul><li>Find (x 1 , x 2 , …, x N ) to: </li></ul><ul><li>Maximize: c 1 x 1 + c 2 x 2 + … + c N x N </li></ul><ul><li>Constraints: </li></ul><ul><ul><li>x 1 >= 0, x 2 >= 0, …, x N >= 0 </li></ul></ul><ul><ul><li>a 1,1 x 1 + a 1,2 x 2 + … + a 1,N x N <= b 1 </li></ul></ul><ul><ul><li>a 2,1 x 1 + a 2,2 x 2 + … + a 2,N x N <= b 2 </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><ul><li>a M,1 x 1 + a M,2 x 2 + … + a M,N x N <= b M </li></ul></ul>
24. 24. Linear Programming, short form <ul><li>Find (x 1 , x 2 , …, x N ) to: </li></ul><ul><li>Maximize: c 1 x 1 + c 2 x 2 + … + c N x N </li></ul><ul><li>Constraints: </li></ul><ul><ul><li>x 1 >= 0, x 2 >= 0, …, x N >= 0 </li></ul></ul><ul><ul><li>a 1,1 x 1 + a 1,2 x 2 + … + a 1,N x N <= b 1 </li></ul></ul><ul><ul><li>a 2,1 x 1 + a 2,2 x 2 + … + a 2,N x N <= b 2 </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><ul><li>a M,1 x 1 + a M,2 x 2 + … + a M,N x N <= b M </li></ul></ul>Find x to: Maximize: c T x Constraints: x >= 0 A x <= b
25. 25. Linear Programming <ul><li>If we multiply each constraint with a non-negative number, </li></ul><ul><li>and sum them up: </li></ul><ul><ul><li>y 1 >= 0, y 2 >= 0, …, y M >= 0 </li></ul></ul><ul><ul><li>y 1 (a 1,1 x 1 + a 1,2 x 2 + … + a 1,N x N ) <= y 1 b 1 </li></ul></ul><ul><ul><li>y 2 (a 2,1 x 1 + a 2,2 x 2 + … + a 2,N x N ) <= y 2 b 2 </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><ul><li>y M (a M,1 x 1 + a M,2 x 2 + … + a M,N x N )<= y M b M </li></ul></ul><ul><li>The equation y T A x <= y T b holds for every non-negative vector y </li></ul>y T A x <= y T b
26. 26. Linear Programming <ul><li>Let us limit the choices for y further. Suppose we are interested in </li></ul><ul><li>only y such that: </li></ul><ul><ul><li>y 1 >= 0, y 2 >= 0, …, y M >= 0 </li></ul></ul><ul><ul><li>y 1 a 1,1 + y 2 a 2,1 + … + y M a M,1 >= c 1 </li></ul></ul><ul><ul><li>y 1 a 1,2 + y 2 a 2,2 + … + y M a M,2 >= c 2 </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><ul><li>y 1 a 1,N + y 2 a 2,N + … + y M a M,N >= c N </li></ul></ul><ul><li>This leads to: y T A x >= c T x </li></ul>y T A >= c T Let’s call this feasible region G for y
27. 27. Linear Programming - Duality <ul><li>Hence, we have this nice property: </li></ul><ul><ul><li>For all: vector x in F, and y in G: </li></ul></ul><ul><li>c T x <= y T A x <= y T b </li></ul><ul><li>This leads to the following Duality theorem; </li></ul>
28. 28. Linear Programming - Duality <ul><li>The following two problems are equivalent: </li></ul><ul><ul><li>Primal problem: </li></ul></ul><ul><ul><ul><li>Maximize c T x </li></ul></ul></ul><ul><ul><ul><li>Subject to A x <= b , x >= 0 </li></ul></ul></ul><ul><ul><li>Dual problem: </li></ul></ul><ul><ul><ul><li>Minimize b T y </li></ul></ul></ul><ul><ul><ul><li>Subject to A T y >= c , y >= 0 </li></ul></ul></ul>
29. 29. Linear Programming - Duality <ul><li>Furthermore, at the optimal solution, we have the following complementary slackness theorem : (no proof, sorry!) </li></ul><ul><ul><li>y i (a i,1 x 1 + a i,2 x 2 + … + a i,N x N -b i ) = 0 </li></ul></ul><ul><ul><li>x i (a 1,i y 1 + a 2,i y 2 + … + a M,i y N - c i ) = 0 </li></ul></ul>Using this duality, we can convert a problem with many variables but a few constraints (hard to solve) into a problem of multiple constraints with a few variables (easy to solve).
30. 30. Another example <ul><li>A burglar breaks into a bank’s storeroom. He notices that the items in the room are in one of ten different types. Each item type has its own value , volume , and weight . The items are homogeneous and can be broken into pieces while still preserving their values. There are so many valuable items for each type that he cannot even count. </li></ul><ul><li>The burglar brings along with him a small bag, which can help him carry at most M kilograms with maximum volume of N meter cubed . </li></ul><ul><li>Help him decide how many items per type he should steal to maximize the total value . </li></ul>
31. 31. Another example <ul><li>Suppose the i-th item type has value a i , weight w i , and volume v i </li></ul><ul><li>Let’s define our variables: </li></ul><ul><ul><li>x 1 = number of items of type 1 </li></ul></ul><ul><ul><li>x 2 = number of items of type 2 </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><ul><li>x 10 = number of items of type 10 </li></ul></ul>
32. 32. Another example <ul><li>There is no way the number of items is negative, therefore: </li></ul><ul><ul><li>x 1 >= 0 </li></ul></ul><ul><ul><li>x 2 >= 0 </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><ul><li>x 10 >= 0 </li></ul></ul><ul><li>Weight constraint: </li></ul><ul><ul><li>w 1 x 1 + w 2 x 2 + … + w 10 x 10 <= M </li></ul></ul><ul><li>Volume constraint: </li></ul><ul><ul><li>v 1 x 1 + v 2 x 2 + … + v 10 x 10 <= N </li></ul></ul><ul><li>Objective function: </li></ul><ul><ul><li>a 1 x 1 + a 2 x 2 + … + a 10 x 10 </li></ul></ul>
33. 33. Another example <ul><li>The primal problem has 10 variables and 2 constraints. To solve it, we need to work with a 10D convex hull!!! </li></ul><ul><li>If we convert into the dual problem, we have 2 variables and 10 constraints as follows: </li></ul><ul><ul><li>Minimize: M y 1 + N y 2 </li></ul></ul><ul><ul><li>Subject to: </li></ul></ul><ul><ul><ul><li>y 1 >= 0, y 2 >= 0 </li></ul></ul></ul><ul><ul><ul><li>w 1 y 1 + v 1 y 2 >= a 1 </li></ul></ul></ul><ul><ul><ul><li>w 2 y 1 + v 2 y 2 >= a 2 </li></ul></ul></ul><ul><ul><ul><li>… </li></ul></ul></ul><ul><ul><ul><li>w 10 y 1 + v 10 y 2 >= a 10 </li></ul></ul></ul>
34. 34. Another example <ul><li>Once we have found optimal y =( y 1 , y 2 ), we can use the complementary slackness theorem to find x =( x 1 , x 2 , …, x 10 ): </li></ul><ul><ul><li>y 1 (w 1 x 1 + w 2 x 2 + … + w 10 x 10 -M) = 0 </li></ul></ul><ul><ul><li>y 2 (v 1 x 1 + v 2 x 2 + … + v 10 x 10 -N) = 0 </li></ul></ul><ul><ul><li>x i (w i y 1 + v i y 2 - a i ) = 0 for i = 1 to 10 </li></ul></ul><ul><li>Thus: </li></ul><ul><ul><li>In the last 10 equations, if w i y 1 + v i y 2 > a i then x i must be 0 </li></ul></ul><ul><ul><li>The remaining x i ’s can be found by solving the remaining system of linear equations. </li></ul></ul>