This document contains lecture slides on nonlinear programming from lectures given at MIT. It discusses two main issues in nonlinear programming: 1) characterizing solutions through necessary and sufficient conditions using concepts like Lagrange multipliers and sensitivity analysis, and 2) computational methods for finding solutions through iterative algorithms. It provides examples of application areas for nonlinear programming like data networks, production planning, and engineering design. It outlines topics covered in the first lecture, including duality theory and the relationship between linear and nonlinear programming.

1. LECTURE SLIDES ON NONLINEAR PROGRAMMING
BASED ON LECTURES GIVEN AT THE
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
CAMBRIDGE, MASS
DIMITRI P. BERTSEKAS
These lecture slides are based on the book:
“Nonlinear Programming,” Athena Scientific,
by Dimitri P. Bertsekas; see
http://www.athenasc.com/nonlinbook.html
The slides are copyrighted but may be freely
reproduced and distributed for any noncom-
mercial purpose.

2. 6.252 NONLINEAR PROGRAMMING
LECTURE 1: INTRODUCTION
LECTURE OUTLINE
• Nonlinear Programming
• Application Contexts
• Characterization Issue
• Computation Issue
• Duality
• Organization

3. NONLINEAR PROGRAMMING
min f(x),
x∈X
where
• f : n → is a continuous (and usually differ-
entiable) function of n variables
• X = n or X is a subset of n with a “continu-
ous” character.
• If X = n, the problem is called unconstrained
• If f is linear and X is polyhedral, the problem
is a linear programming problem. Otherwise it is
a nonlinear programming problem
• Linear and nonlinear programming have tradi-
tionally been treated separately. Their method-
ologies have gradually come closer.

4. TWO MAIN ISSUES•
• Characterization of minima
− Necessary conditions
− Sufficient conditions
− Lagrange multiplier theory
− Sensitivity
− Duality
• Computation by iterative algorithms
− Iterative descent
− Approximation methods
− Dual and primal-dual methods

5. APPLICATIONS OF NONLINEAR PROGRAMMING•
• Data networks – Routing
• Production planning
• Resource allocation
• Computer-aided design
• Solution of equilibrium models
• Data analysis and least squares formulations
• Modeling human or organizational behavior

6. CHARACTERIZATION PROBLEM
• Unconstrained problems
− Zero 1st order variation along all directions
• Constrained problems
− Nonnegative 1st order variation along all fea-
sible directions
• Equality constraints
− Zero 1st order variation along all directions
on the constraint surface
− Lagrange multiplier theory
• Sensitivity

7. COMPUTATION PROBLEM
• Iterative descent
• Approximation
• Role of convergence analysis
• Role of rate of convergence analysis
• Using an existing package to solve a nonlinear
programming problem

8. POST-OPTIMAL ANALYSIS
• Sensitivity
• Role of Lagrange multipliers as prices

9. DUALITY
• Min-common point problem / max-intercept prob-
lem duality
0 0
Min Common Point
Max Intercept Point Max Intercept Point
Min Common Point
S S
(a) (b)
Illustration of the optimal values of the min common point
and max intercept point problems. In (a), the two optimal
values are not equal. In (b), the set S, when “extended
upwards” along the nth axis, yields the set
S¯ = {¯ xn ≥ xn, ¯x | for some x ∈ S, ¯ xi = xi, i = 1, . . . , n − 1}
which is convex. As a result, the two optimal values are
equal. This fact, when suitably formalized, is the basis for
some of the most important duality results.