Optimization Methods

1. OPTIMIZATION METHODSOPTIMIZATION METHODS
Svetlana Novikova
Master's program
"Information and software of the automated systems"
3 semester - the fall semester of the second year master degree

2. Information about courseInformation about course
Number of groups - 1
Number of people in group - 7-12.
Of these males - 90%, females – 10%
Preliminary courses:
Calculus
Linear Algebra
Computational Mathematics
Mathematic modeling
2

3. Distribution of working timeDistribution of working time
3

4. Types of Extrema
Figure 1.1. A strong global
minimizer.
Figure 1.2. Types of minima.
IntroductionIntroduction
4

5. Necessary and Sufficient ConditionsNecessary and Sufficient Conditions
for Local Minimafor Local Minima
Necessary conditions.
If X∗
is a local minimizer in the
interior of R then gradient
g(X∗
) =f’(X*) = 0.
Sufficient conditions.
If X*
is located in the interior of R, then the conditions
1) Gradient g(X*
) =f’(X*) = 0
2) Hessian H(X*
) is positive definite
are sufficient for X*
to be a strong local minimizer.
5

6. ONE-DIMENSIONAL OPTIMIZATIONONE-DIMENSIONAL OPTIMIZATION

7. Classical optimization techniques

8. • Golden-Section Search
• Passive search method
• Blindman-Walking search
• Dichotomous Search
NUMERICAL METHODS WITHOUTNUMERICAL METHODS WITHOUT
THE USE OF DERIVATIVES.THE USE OF DERIVATIVES.
8

9. NUMERICAL METHODS WITH THENUMERICAL METHODS WITH THE
USE OF DERIVATIVES.USE OF DERIVATIVES.
• Mid-point method
• Newton's method
9

10. MULTI-DIMENSIONAL OPTIMIZATION

11. The classic methodThe classic method
Step 1. Solving the system of equations (1),
find all the stationary points of the
function.
Step 2. Using the sufficient conditions for a
minimum of stationary points find
local minimum points X0
, and by
comparing the values of them,​​
determine the points of global
minimum.
11

12. Numerical methods of
multidimensional optimization
Methods used to optimize the
value of the function only at
certain points (direct methods)
Methods that use derivatives of
the function (gradients)
12

13. • Search method on a
simplex
• Method of cyclic
coordinate descent (CCD)
Direct methods.
X0
X1
13

14. METHODS WITH USING DERIVATIVES.METHODS WITH USING DERIVATIVES.
• Gradient descent method
• Method of steepest descent
• Newton's Method
14

15. CONSTRAINED OPTIMIZATION
• The classic method
• The method of Lagrange multipliers
• The Penalty Function Method
15

16. Settlement-graphic work
LINEAR PROGRAMMING

17. • Graphical method of solving the LPP
• First Simplex method
• Methods of construction of the initial support
plan
 L-problem
 Extended M-problem
17

18. Thank you for your attention.
18