Optimization toolbox presentation

MATLAB Optimization
Toolbox
Presented by
Chin Pei
February 28, 2003

Presentation Outline
 Introduction
 Function Optimization
 Optimization Toolbox
 Routines / Algorithms available
 Minimization Problems
 Unconstrained
 Constrained

Example

The Algorithm Description
 Multiobjective Optimization
 Optimal PID Control Example

Function Optimization
 Optimization concerns the minimization
or maximization of functions
 Standard Optimization Problem
IntroductionIntroduction Unconstrained MinimizationUnconstrained Minimization Constrained MinimizationConstrained Minimization
Multiobjective OptimizationMultiobjective Optimization ConclusionConclusion
( )~
~
min
x
f x
( )~
0jg x ≤
( )~
0ih x =
L U
k k kx x x≤ ≤
Equality ConstraintsSubject to:
Inequality Constraints
Side Constraints

( )~
f x is the objective function, which measure
and evaluate the performance of a system.
In a standard problem, we are minimizing
the function.
For maximization, it is equivalent to minimization
of the –ve of the objective function.
~
x is a column vector of design variables, which can
affect the performance of the system.

Constraints – Limitation to the design space.
Can be linear or nonlinear, explicit or implicit functions
( )~
0jg x ≤
( )~
0ih x =
L U
k k kx x x≤ ≤
Equality Constraints
Inequality Constraints
Side Constraints
Most algorithm require less than!!!

Optimization Toolbox
 Is a collection of functions that extend the capability of
MATLAB. The toolbox includes routines for:
 Unconstrained optimization
 Constrained nonlinear optimization, including goal
attainment problems, minimax problems, and semi-
infinite minimization problems
 Quadratic and linear programming
 Nonlinear least squares and curve fitting
 Nonlinear systems of equations solving
 Constrained linear least squares
 Specialized algorithms for large scale problems

Minimization Algorithm

Minimization Algorithm (Cont.)

Equation Solving Algorithms

Least-Squares Algorithms

Implementing Opt. Toolbox
 Most of these optimization routines require
the definition of an M-file containing the
function, f, to be minimized.
 Maximization is achieved by supplying the
routines with –f.
 Optimization options passed to the routines
change optimization parameters.
 Default optimization parameters can be
changed through an options structure.

Unconstrained Minimization
 Consider the problem of finding a set of
values [x1 x2]T
that solves
IntroductionIntroduction Unconstrained Minimization Constrained MinimizationConstrained Minimization
( ) ( )1
~
2 2
1 2 1 2 2
~
min 4 2 4 2 1x
x
f x e x x x x x= + + + +
[ ]1 2
~
T
x x x=

Steps
 Create an M-file that returns the
function value (Objective Function)
 Call it objfun.m
 Then, invoke the unconstrained
minimization routine
 Use fminunc

Step 1 – Obj. Function
function f = objfun(x)
f=exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1);
[ ]1 2
~
T
x x x=
Objective function

Step 2 – Invoke Routine
x0 = [-1,1];
options = optimset(‘LargeScale’,’off’);
[xmin,feval,exitflag,output]=
fminunc(‘objfun’,x0,options);
Output arguments
Input arguments
Starting with a guess
Optimization parameters settings

Results
xmin =
0.5000 -1.0000
feval =
1.3028e-010
exitflag =
1
output =
iterations: 7
funcCount: 40
stepsize: 1
firstorderopt: 8.1998e-004
algorithm: 'medium-scale: Quasi-Newton line search'
Minimum point of design variables
Objective function value
Exitflag tells if the algorithm is converged.
If exitflag > 0, then local minimum is found
Some other information

More on fminunc – Input
[xmin,feval,exitflag,output,grad,hessian]=
fminunc(fun,x0,options,P1,P2,…)
 fun: Return a function of objective function.
 x0: Starts with an initial guess. The guess must be a vector of size of
number of design variables.
 option: To set some of the optimization parameters. (More after few
slides)
 P1,P2,…: To pass additional parameters.
Ref. Manual: Pg. 5-5 to 5-9

More on fminunc – Output
[xmin,feval,exitflag,output,grad,hessian]=
fminunc(fun,x0,options,P1,P2,…)
 xmin: Vector of the minimum point (optimal point). The size is the
number of design variables.
 feval: The objective function value of at the optimal point.
 exitflag: A value shows whether the optimization routine is
terminated successfully. (converged if >0)
 output: This structure gives more details about the optimization
 grad: The gradient value at the optimal point.
 hessian: The hessian value of at the optimal point

Options Setting – optimset
Options =
optimset(‘param1’,value1, ‘param2’,value2,…)
 The routines in Optimization Toolbox has a set of default optimization
parameters.
 However, the toolbox allows you to alter some of those parameters,
for example: the tolerance, the step size, the gradient or hessian
values, the max. number of iterations etc.
 There are also a list of features available, for example: displaying the
values at each iterations, compare the user supply gradient or
hessian, etc.
 You can also choose the algorithm you wish to use.

Options Setting (Cont.)
Options =
 Type help optimset in command window, a list of options setting available will be displayed.
 How to read? For example:
LargeScale - Use large-scale algorithm if
possible [ {on} | off ]
The default is with { }
Parameter (param1)
Value (value1)

Options Setting (Cont.)
LargeScale - Use large-scale algorithm if
possible [ {on} | off ]
 Since the default is on, if we would like to turn off, we just type:
Options =
optimset(‘LargeScale’, ‘off’)
Options =
and pass to the input of fminunc.

Useful Option Settings
 Display - Level of display [ off | iter |
notify | final ]
 MaxIter - Maximum number of iterations
allowed [ positive integer ]
 TolCon - Termination tolerance on the
constraint violation [ positive scalar ]
 TolFun - Termination tolerance on the
function value [ positive scalar ]
 TolX - Termination tolerance on X [ positive
scalar ]
Highly recommended to use!!!

fminunc and fminsearch
 fminunc uses algorithm with gradient
and hessian information.
 Two modes:
 Large-Scale: interior-reflective Newton
 Medium-Scale: quasi-Newton (BFGS)
 Not preferred in solving highly
discontinuous functions.
 This function may only give local solutions.

fminunc and fminsearch
 fminsearch is generally less efficient than
fminunc for problems of order greater than
two. However, when the problem is highly
discontinuous, fminsearch may be more
robust.
 This is a direct search method that does not
use numerical or analytic gradients as in
fminunc.
 This function may only give local solutions.

Constrained Minimization
IntroductionIntroduction Unconstrained MinimizationUnconstrained Minimization Constrained Minimization
[xmin,feval,exitflag,output,lambda,grad,hessian]
=
fmincon(fun,x0,A,B,Aeq,Beq,LB,UB,NONLCON,options,
P1,P2,…)
Vector of Lagrange
Multiplier at optimal
point

Example
( )~
1 2 3
~
min
x
f x x x x= −
2
1 22 0x x+ ≤
1 2 3
1 2 3
2 2 0
2 2 72
x x x
x x x
− − − ≤
+ + ≤
1 2 30 , , 30x x x≤ ≤
Subject to:
1 2 2 0
,
1 2 2 72
A B
− − −   
= =   
   
0 30
0 , 30
0 30
LB UB
   
   = =   
      
function f = myfun(x)
f=-x(1)*x(2)*x(3);

Example (Cont.)
2
1 22 0x x+ ≤For
Create a function call nonlcon which returns 2 constraint vectors [C,Ceq]
function [C,Ceq]=nonlcon(x)
C=2*x(1)^2+x(2);
Ceq=[];
Remember to return a null
Matrix if the constraint does
not apply

Example (Cont.)
x0=[10;10;10];
A=[-1 -2 -2;1 2 2];
B=[0 72]';
LB = [0 0 0]';
UB = [30 30 30]';
[x,feval]=fmincon(@myfun,x0,A,B,[],[],LB,UB,@nonlcon)
1 2 2 0
,
1 2 2 72
A B
− − −   
= =   
   
Initial guess (3 design variables)
CAREFUL!!!
fmincon(fun,x0,A,B,Aeq,Beq,LB,UB,NONLCON,options,P1,P2,…)
0 30
0 , 30
0 30
LB UB
   
   = =   
      

Example (Cont.)
Warning: Large-scale (trust region) method does not currently solve this type of problem, switching to
medium-scale (line search).
> In D:ProgramsMATLAB6p1toolboxoptimfmincon.m at line 213
In D:usrCHINTANGOptToolboxmin_con.m at line 6
Optimization terminated successfully:
Magnitude of directional derivative in search direction less than 2*options.TolFun and maximum
constraint violation is less than options.TolCon
Active Constraints:
2
9
x =
0.00050378663220
0.00000000000000
30.00000000000000
feval =
-4.657237250542452e-035
2
1 22 0x x+ ≤
1 2 3
1 2 3
2 2 0
2 2 72
x x x
x x x
− − − ≤
+ + ≤
1
2
3
0 30
0 30
0 30
x
x
x
≤ ≤
≤ ≤
≤ ≤
Const. 1
Const. 2
Const. 3
Const. 4
Const. 5
Const. 6
Const. 7
Sequence: A,B,Aeq,Beq,LB,UB,C,Ceq
Const. 8
Const. 9

Multiobjective Optimization
 Previous examples involved problems
with a single objective function.
 Now let us look at solving problem with
multiobjective function by lsqnonlin.
 Example is taken by designing an
optimal PID controller for an plant.
Multiobjective Optimization ConclusionConclusion

Simulink Example
Goal: Optimize the control parameters in Simulink model optsim.mdl
in order to minimize the error between the output and input.
Plant description:
• Third order under-damped with actuator limits.
• Actuation limits are a saturation limit and a slew rate limit.
• Saturation limit cuts off input: +/- 2 units
• Slew rate limit: 0.8 unit/sec

Simulink Example (Cont.)
Initial PID Controller Design

Solving Methodology
 Design variables are the gains in PID
controller (KP, KI and KD) .
 Objective function is the error between
the output and input.

Solving Methodology (Cont.)
 Let pid = [Kp Ki Kd]T
 Let also the step input is unity.
 F = yout - 1
 Construct a function tracklsq for
objective function.

Objective Function
function F = tracklsq(pid,a1,a2)
Kp = pid(1);
Ki = pid(2);
Kd = pid(3);
% Compute function value
opt = simset('solver','ode5','SrcWorkspace','Current');
[tout,xout,yout] = sim('optsim',[0 100],opt);
F = yout-1;
Getting the simulation
data from Simulink
The idea is perform nonlinear least squares
minimization of the errors from time 0 to 100
at the time step of 1.
So, there are 101 objective functions to minimize.

The lsqnonlin
[X,RESNORM,RESIDUAL,EXITFLAG,OUTPUT,LAMBDA,JACOBIAN]
= LSQNONLIN(FUN,X0,LB,UB,OPTIONS,P1,P2,..)

Invoking the Routine
clear all
Optsim;
pid0 = [0.63 0.0504 1.9688];
a1 = 3; a2 = 43;
options =
optimset('LargeScale','off','Display','iter','TolX',0.001,'TolFun
',0.001);
pid = lsqnonlin(@tracklsq,pid0,[],[],options,a1,a2)
Kp = pid(1); Ki = pid(2); Kd = pid(3);

Results
Optimal gains

Results (Cont.)
Initial Design
Optimization Process
Optimal Controller Result

Conclusion
 Easy to use! But, we do not know what is happening
behind the routine. Therefore, it is still important to
understand the limitation of each routine.
 Basic steps:
 Recognize the class of optimization problem
 Define the design variables
 Create objective function
 Recognize the constraints
 Start an initial guess
 Invoke suitable routine
 Analyze the results (it might not make sense)
Multiobjective OptimizationMultiobjective Optimization Conclusion

Thank You!
Questions & Suggestions?

Optimization toolbox presentation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Optimization toolbox presentation

Similar to Optimization toolbox presentation (20)

Recently uploaded

Recently uploaded (20)

Optimization toolbox presentation