ok

1. HARAMAYA UNIVERSITY
POST GRADUATE PROGRAM DIRECTORATE
COLLEGE OF NATURALAND COMPUTATIONAL SCIENCE
MSC PROGRAM IN MATHEMATICAL MODELING
Prepared by: -
1. Abdella Kereme Ahmed ID No: PGP/646/14
2. Alemseged Kifle ID No: PGP/646/14
Presentation Assignment of Optimization and Optimal Control
Submitted to: - Getinet A. (Ph.D.)
June, 2021 GC.
Haramaya University, Ethiopia

2. 1. FIBONACCI AND GOLDEN SECTION METHOD
1.1 INTRODUCTION
Optimization problem involves the objective function and/or constraints that are not stated as
explicit functions of the design variables or which are too complicated to manipulate, we cannot
solve it by using the classical analytical methods.
So, we consider now algorithms for locating a local minimum in the nonlinear optimization
problem with no constraints. In such cases we need to use the numerical methods of
optimization for solution.
Like that Fibonacci method, golden section method

3. 1.2 Fibonacci method
Fibonacci method can be used to find the minimum of a function of one variable even if the
function is not continuous. This method, like many other elimination methods, has the following
limitations:
The initial interval of uncertainty, in which the optimum lies, has to be known.
The function being optimized has to be unimodal in the initial interval of uncertainty.
The exact optimum cannot be located in this method
The number of function evaluations to be used in the search or the resolution required has to be
specified beforehand.

4. Cont.…
This method makes use of the sequence of Fibonacci numbers, 𝐹𝑛 for placing the experiments.
These numbers are defined as
𝐹0 = 𝐹1 = 1.
𝐹𝑛 = 𝐹𝑛−1 + 𝐹𝑛−2 , 𝑛 = 2,3,4 …
which yield the sequence 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, …
Procedure. Let L0 be the initial interval of uncertainty defined by 𝑎 ≤ 𝑥 ≤ 𝑏 and n be the total
number of experiments to be conducted. Define
𝐿2
∗
=
𝐹𝑛−2
𝐹𝑛
𝐿0 …………………… (1)

5. CONT.…
and place the first two experiments at points x1 and x2, which are located at a
distance of 𝐿2
∗
from each end of L0. This gives
𝑋1 = 𝑎 + 𝐿2
∗
= 𝑎 +
𝐹𝑛−2
𝐹𝑛
𝐿0 ……………… (2)
𝑋2 = 𝑏 − 𝐿2
∗
= 𝑏 −
𝐹𝑛−2
𝐹𝑛
𝐿0 = 𝑎 +
𝐹𝑛−1
𝐹𝑛
𝐿0 ………………….. (3)
Discard part of the interval by using the unimodality assumption.

6. Cont.'s….
Then there remains a smaller interval of uncertainty 𝐹2 given by
𝐿2 = 𝐿0 − 𝐿2
∗
= 𝐿0 1 −
𝐹𝑛−2
𝐹𝑛
=
𝐹𝑛−1
𝐹𝑛
𝐿0 ……………………… (4)
and with one experiment left in it. This experiment will be at a distance of
𝐿2
∗
=
𝐹𝑛−2
𝐹𝑛
𝐿0 =
𝐹𝑛−2
𝐹𝑛−1
𝐿2 ………………………………….. (5)
from one end and
𝐿2 − 𝐿2
∗
=
𝐹𝑛−3
𝐹𝑛
𝐿0 =
𝐹𝑛−3
𝐹𝑛−1
𝐿2 …………………………… (6)
from the other end.

7. Cont.'s….
Now place the third experiment in the interval 𝐿2 so that the current two experiments are
located at a distance of
𝐿3
∗
=
𝐹𝑛−3
𝐹𝑛
𝐿0 =
𝐹𝑛−3
𝐹𝑛−1
𝐿2 =
𝐿0
𝛾3 …………………………………. (7)
from each end of the interval 𝐿2 .
Again, the unimodality property will allow us to reduce the interval of uncertainty to 𝐿3 given
by 𝐿3 = 𝐿2 − 𝐿3
∗
= 𝐿2 −
𝐹𝑛−3
𝐹𝑛−1
𝐿2 =
𝐹𝑛−2
𝐹𝑛−1
𝐿2 =
𝐹𝑛−2
𝐹𝑛
𝐿0 =
𝐿0
𝛾2 ………………. (8
This process of discarding a certain interval and placing a new experiment in the remaining
interval can be continued, so that the location of the j th experiment and the interval of
uncertainty at the end of j experiments are, respectively, given by

8. CONT…
𝐿𝑗
∗
=
𝐹𝑛−𝑗
𝐹𝑛−(𝑗−1)
𝐿𝑗−1 =
𝐹𝑛−𝑗
𝐹𝑛
𝐿0 =
𝐿0
𝛾𝑗 ………………………………… (9)
𝐿𝑗 =
𝐹𝑛−(𝑗−1)
𝐹𝑛
𝐿0 =
𝐿0
𝛾𝑗−1 ……………………………. (10)
The ratio of the interval of uncertainty remaining after conducting j of the n
predetermined experiments to the initial interval of uncertainty becomes
𝐿𝑗
𝐿0
=
𝐹𝑛−(𝑗−1)
𝐹𝑛
………………………… (11)
and for 𝑗 = 𝑛, we obtain
𝐿𝑛
𝐿0
=
𝐹1
𝐹𝑛
=
1
𝐹𝑛
………………………. (12)

9. CONT’S …
The ratio 𝐿𝑛
𝐿0
will permit us to determine n, the required number of experiments, to achieve
any desired accuracy in locating the optimum point. Table 1 gives the reduction ratio in the
interval of uncertainty obtainable for different number of experiments.
The symbol 𝐿𝑗 is used to denote the interval of uncertainty remaining after conducting 𝑗
experiments,
while the symbol 𝐿𝑗
∗
𝑗 is used to define the position of the 𝑗 𝑡ℎ experiment.

10. Table 1 Reduction Ratios
Value of n Fibonacci number,𝐹𝑛 Reduction ratio,
𝐿𝑛
𝐿0
0 1 1.0
1 1 1.0
2 2 0.5
3 3 0.3333
4 5 0.2
5 8 0.1250
6 13 0.07692
7 21 0.04762
8 34 0.02941
9 55 0.01818
10 89 0.01124
11 144 0.006944
12 233 0.004292
13 377 0.002653
14 610 0.001639
15 987 0.001013
16 1597 0.0006406
17 2584 0.0003870
18 4181 0.0002392
19 6765 0.0001479
20 10946 0.00009135

11. 1.3 GOLDEN SECTION METHOD
 The golden section method is same as the Fibonacci method except that in the Fibonacci
method the total number of experiments to be conducted has to be specified before beginning
the calculation, whereas this is not required in the golden section method.
 In the Fibonacci method, the location of the first two experiments is determined by the total
number of experiments, N.
 In the golden section method we start with the assumption that we are going to conduct a large
number of experiments. Of course, the total number of experiments can be decided during the
computation.
 Set 𝐿 = 𝑎 and 𝑅 = 𝑏
 Pick two points, 𝑥1 and 𝑥2, with 𝑥1 < 𝑥2, and both points in 𝐿, 𝑅
 Find 𝑓(𝑥1) and 𝑓(𝑥2)

12. Cont.'s …
 If 𝑓(𝑥1) > 𝑓(𝑥2), then the minimizer must be to the right of 𝑥1.
Redefine the range as 𝑅 = 𝑥1, 𝐿 = 𝑏 i.e. the minimizer must be in the interval [𝑥1, 𝑏].
 Repeat the algorithm from Step 2 until the interval length is smaller than some pre-set
tolerance.
 Otherwise (i.e. 𝑓(𝑥1) < 𝑓(𝑥2)), the minimizer must be to the left of 𝑥2.
 Redefine the range as 𝑅 = 𝑎, 𝐿 = 𝑥2 i.e. the minimizer must be in the interval [𝑎, 𝑥2].
 Repeat the algorithm from Step 2 until the interval length is smaller than some pre-set tolerance

13. Cont.…
____________ 𝐿__________________
______𝐿1 ___________
_____𝐿2 _____
L__________________________________ R
𝑥1 𝑥2
The intervals of uncertainty remaining at the end of different number of experiments can be
computed as follows:
𝐿2 = lim
𝑁→∞
𝐹𝑁−1
𝐹𝑁
𝐿0 ………………………… (13)

14. CONT….
𝐿3 = lim
𝑁→∞
𝐹𝑁−2
𝐹𝑁
𝐿0 = lim
𝑁→∞
𝐹𝑁−2
𝐹𝑁−1
𝐹𝑁−1
𝐹𝑁
𝐿0 ………………….. (14)
𝐿3 ≃ lim
𝑁→∞
𝐹𝑁−1
𝐹𝑁
2
𝐿0 ……………………………………………. (15)
This result can be generalized to obtain
𝐿𝐾 = lim
𝑁→∞
𝐹𝑁−1
𝐹𝑁
𝐾−1
𝐿0 ……………………………………….. (16)
Using the relation
𝐹𝑛 = 𝐹𝑛−1 + 𝐹𝑛−2 ………………………………………. (17)
we obtain, after dividing both sides by 𝐹𝑛−1,

15. Cont.….

𝐹𝑁
𝐹𝑁−1
= 1 +
𝐹𝑁−2
𝐹𝑁−1
……………………………………………… (18)
By defining a ratio 𝛾 as
𝛾 = lim
𝑁→∞
𝐹𝑁
𝐹𝑁−1
…………………………………………………. (19)
Equation (18) can be expressed as
𝛾2 ≃
1
𝛾
+ 1 ………………………………………….. (20)
that is,
𝛾2
− 𝛾 − 1 = 0 ………………………………… (21)

16. CONT…
Solve for 𝛾 ( 𝛾 is called the Golden Ratio, named by ancient Greeks)
This gives the root 𝛾 = 1.618, and hence Eq. (16) yields
𝐿𝐾 =
1
𝛾
𝐾−1
𝐿0 = 0.618 𝐾−1
𝐿0 ………………………….. (22)
In Eq. (15) the ratios
𝐹𝑁−2
𝐹𝑁−1
and
𝐹𝑁−1
𝐹𝑁
have been taken to be same for large values of 𝑁.
 Procedure. The procedure is same as the Fibonacci method except that the location of the first
two experiments is defined by

17. Cont.….
 𝐿2
∗
=
𝐹𝑁−2
𝐹𝑁−1
𝐹𝑁−1
𝐹𝑁
𝐿0 =
1
𝛾2 = 0.382𝐿0 ……………………….. (23)
The desired accuracy can be specified to stop the procedure.
So, for the range [a , b],
 𝑋1 = 𝑎 + 𝐿2
∗
= 𝑎 + 0.382𝐿0
= a + 0.382(b − a) = 0.6183a + 0.38197b
 𝑋2 = 𝑏 − 𝐿2
∗
= 𝑏 − 0.382𝐿0
= b − 0.382(b − a) = 0.6183b + 0.38197a

18. Example 1:
Find an approximation to the minimum of 𝑓 𝑥 = 4𝑥3
+ 𝑥2
− 7𝑥 + 14 𝑤𝑖𝑡ℎ𝑖𝑛 𝑡ℎ𝑒 interval [0,1]
using Golden ratio method with and iterate until the width of the interval is less than 0.15
Solution
Step1: 𝐿0 = 0,1 , stopping tolerance ∈= 0.15
Step2: 𝐿2
∗
=
1
𝛾2 = 0.6182 2 = 0.3819
0_______________________________1
____𝐿2
∗
____ 𝑥1 𝑥2____𝐿2
∗
___

19. Cont.…
𝑥1 = 0 + 0.3819 = 0.3819 , 𝑓 𝑥1 = 11. 6953
𝑥2 = 1 − 𝐿2
∗
= 0.6181 , 𝑓 𝑥2 = 11. 9999
Here, 𝑥1 < 𝑥2 but 𝑓 𝑥1 > 𝑓 𝑥1
Thus, minimizer must be to the right of 𝑥1
Therefore, the interval of uncertainty is [ 𝑥1, 1] => [0.3819,1]
𝐿2 = 𝑅 − 𝐿 = 1 − 𝑥1 = 1 − 0.3819 = 0.6181
Step3: to generate 𝑥3 𝐿3
∗
=
1
𝛾3 = 0.6182 3 = 0.2361
𝑥1 _______________________________1
____𝐿3
∗
____ 𝑥2 𝑥3____𝐿3
∗
___

20. CONT….
𝑥3 = 1 − 𝐿3
∗
= 0.7639, 𝑓 𝑥3 = 11.093
Now 𝑥2 < 𝑥3 𝑎𝑛𝑑 𝑓 𝑥2 < 𝑓 𝑥3
Thus, minimizer must be to the left of 𝑥1
Therefore, the interval of uncertainty is [ 𝑥1, 𝑥3] => [0.3819,0.7639]
𝐿3 = 𝑅 − 𝐿 = 𝑥3 − 𝑥1 = 0.7639 − 0.3819 = 0.382

21. Cont.….
Step4: to generate 𝑥4 then define 𝐿4
∗
=
1
𝛾4 = 0.6182 4 = 0.1459
𝑥1 _______________________________𝑥3
____𝐿4
∗
____ 𝑥4 𝑥2____𝐿4
∗
___
𝑥4 = 𝑥1 + 𝐿4
∗
= 0.5278 , 𝑓 𝑥4 = 11.174
Now 𝑥4 < 𝑥2 and 𝑓 𝑥4 > 𝑓 𝑥2
Thus, minimizer must be to the right of 𝑥4
Therefore, the interval of uncertainty is [ 𝑥4, 𝑥3] => [0.5278,0.7639]
𝐿4 = 𝑅 − 𝐿 = 𝑥4 − 𝑥3 = 0.7639 − 0.5278 = 0.2361

22. CONT…
Step5: to generate 𝑥5 then define 𝐿5
∗
=
1
𝛾5 = 0.6182 5
= 0.0902
𝑥4 _______________________________𝑥3
____𝐿5
∗
____ 𝑥2 𝑥5____𝐿5
∗
___
𝑥5 = 𝑥3 − 𝐿5
∗
= 0.6737 , 𝑓 𝑥5 = 10.9639
Now 𝑥2 < 𝑥5 and 𝑓 𝑥2 > 𝑓 𝑥5
Thus, minimizer must be to the right of 𝑥2
Therefore, the interval of uncertainty is [ 𝑥2, 𝑥3] => [0.6181,0.7639]
𝐿5 = 𝑅 − 𝐿 = 𝑥3 − 𝑥2 = 0.7639 − 0.6182 = 0.1457 < 0.15

23. Cont.….
L R x1 x2 f(x1) f(x2) Update R – L
0 1 0.3819 0.6181 11. 6953 11. 9999 L=𝑥1 0.6181
0.3819 1 0.3819 0.7639 11.6953 11.093 L = 𝑥4 0.382
0.5278 0.7639 0.5278 0.7639 11.174 11.093 R= 𝑥2 0.2361
0.6182 0.7639 0.6181 0.7639 11. 9999 11.093 L = x2 0.1457

24. SPECIAL THANKS FOR LISTENING