Est

1. 348 Statistical Computing with R
11.8 Linear Programming – The Simplex Method
The simplex method is a widely applied optimization method for a special
class of constrained optimization problems with linear objective functions and
linear constraints. The constraints usually include inequalities, and therefore
the region over which the objective function is to be optimized (the feasible
region) can be described by a simplex. Linear programming methods include
the simplex method and interior point methods, but here we illustrate the
simplex method only. See Nocedal and Wright [206, Ch. 13] for a summary
of the simplex method.
Given m linear constraints in n variables, let A be the m × n matrix of
coefficients, so that the constraints are given by Ax ≥ b, where b ∈ Rm
. Here
we suppose that m < n. An element x ∈ Rn
of the feasible set satisfies the
constraint Ax ≥ b. The objective function is a linear function of n variables
with coefficients given by vector c. Hence, the objective is to minimize cT
x
subject to the constraint Ax ≥ b.
The problem as stated above is the primal problem. The dual problem is:
maximize bT
y subject to the constraint AT
y ≤ c, where y ∈ Rn
. The duality
theorem states that if either the primal or the dual problem has an optimal
solution with a finite objective value, then the primal and the dual problems
have the same optimal objective value.
The vertices of the simplex are called the basic feasible points of the fea-
sible set. When the optimal value of the objective function exists, it will be
achieved at one of the basic feasible points. The simplex algorithm evaluates
the objective function at the basic feasible points, but selects the points at
each iteration in such a way that an optimal solution is found in relatively
few iterations. It can be shown (see e.g. [206, Thm. 13.4]) that if the linear
program is bounded and not degenerate, the simplex algorithm will terminate
after finitely many iterations at one of the basic feasible points.
The simplex method is implemented by the simplex function in the boot
package [34]. The simplex function will maximize or minimize the linear
function ax subject to the constraints A1x ≤ b1, A2x ≥ b2, A3x = b3, and
x ≥ 0. Either the primal or dual problem is easily handled by the simplex
function.
Example 11.16 (Simplex algorithm)
Use the simplex algorithm to solve the following problem.
Maximize 2x + 2y + 3z subject to
−2x + y + z ≤ 1
4x − y + 3z ≤ 3
x ≥ 0, y ≥ 0, z ≥ 0.

2. Numerical Methods in R 349
For such a small problem, it would not be too difficult to solve it directly, be-
cause the theory implies that if there is an optimal solution, it will be achieved
at one of the vertices of the feasible set. Hence, we need only evaluate the
objective function at each of the finitely many vertices. The vertices are de-
termined by the intersection of the linear constraints. The simplex method
also evaluates the objective function as it moves from one vertex to another,
usually changing the coordinates in one vertex only at each step. The trick
is to decide which vertex to check next by moving in the direction of greatest
increase/decrease in the objective function. Eventually, for bounded, nonde-
generate problems, the value of the objective function cannot be improved and
the algorithm terminates with the solution. The simplex function implements
the algorithm.
library(boot) #for simplex function
A1 <- rbind(c(-2, 1, 1), c(4, -1, 3))
b1 <- c(1, 3)
a <- c(2, 2, 3)
simplex(a = a, A1 = A1, b1 = b1, maxi = TRUE)
Optimal solution has the following values
x1 x2 x3
2 5 0
The optimal value of the objective function is 14.
11.9 Application: Game Theory
In the linear program of Example 11.16, the constraints are inequalities.
Equality constraints are also possible. Equality constraints might arise if, for
example, the sum of the variables is fixed. If the variables represent a discrete
probability mass function, the sum of the probabilities must equal one. We
solve for a probability mass function in the next problem. It is a classical
problem in game theory.
Example 11.17 (Solving the Morra game)
One of the world’s oldest known games of strategy is the Morra game. In the
3-finger Morra game, each player shows 1, 2, or 3 fingers, and simultaneously
each calls his guess of the number of fingers his opponent will show. If both
players guess correctly, the game is a draw. If exactly one player guesses
correctly, he wins an amount equal to the sum of the fingers shown by both