Comparative study of algorithms of nonlinear optimization

429 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
429
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Comparative study of algorithms of nonlinear optimization

  1. 1. Presented byPritam BhadraPranamesh ChakrabortyIndian Institute of Technology, Kanpur11 May 2013Comparative study of algorithms ofNonlinear Optimization
  2. 2. Methods of Nonlinear OptimizationThe methods for nonlinear optimization are:1. Conjugate gradient methodsa) Fletcher-Reevesb) Polak- Ribiere2. Powell’s conjugate direction method3. Quasi-Newton methodsa) Davidon-Fletcher-Powell (DFP) methodb) Broyden-Fletcher-Goldfarb-Shanno (BFGS) method
  3. 3. Functions used for comparison of algorithms1. Booth function2. Himmelblau function3. Beale function4. Ackley function5. Goldstein function6. Cross-in-tray function7. Bukin function8. Rosenbrock function
  4. 4. For each function(two-dimensional), local minima is obtainedfrom 6 initial points:a) (0,0)b) (1,1)c) (3,3)d) (5,5)e) (-2,-2)f) (-4,-4)g) (-6,-6)for 3 ranges of α (step-size) (0,10) (-5,20) (-50,50)
  5. 5. Booth function2 2( , ) ( 2 7) (2 5)f x y x y x yQuadratic function of 2 variables3d plot of Booth function
  6. 6. Booth functionContour plot of Booth function
  7. 7. Booth function Exact same results for F-R and P-R since the function is quadraticInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# offunctionevaluations# ofiterationsFinalPoint# offunctionevaluations# ofiterationsFinalPoint# offunctionevaluations# ofiterationsFinalPoint# offunctionevaluations# ofiterationsFinalPoint# offunctionevaluations# ofiterationsFinalPoint# offunctionevaluations# ofiterations(0,0) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2(1,1) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2(3,3) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2(5,5) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2(-2,-2) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2(-4,-4) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2(-6,-6) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 The algorithm converges in 2 steps as it is a 2-d quadratic problem
  8. 8. Booth functionInitialPointDavidon-Fletcher-Powell (DFP)methodBroyden-Fletcher-Goldfarb-Shanno(BFGS) method Powells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)(1,3) 2,36 (1,3) 2,29 (1,3) 2,23 (1,3) 2,40 (1,3) 2,31 (1,3) 2,26(3.4,1.08)(1,3) 1 (1,3) 1(1,1)(1,3) 2,43 (1,3) 2,29 (1,3) 2,38 (1,3) 2,42 (1,3) 2,28 (1,3) 2,32(2.6,1.72)(1,3) 1 (1,3) 1(3,3)(1,3) 2,35 (1,3) 2,29 (1,3) 2,32 (1,3) 2,44 (1,3) 2,29 (1,3) 2,31 (1,3) 1 (1,3) 1(5,5)(1,3) 2,32 (1,3) 2,27 (1,3) 2,22 (1,3) 2,37 (1,3) 2,28 (1,3) 2,24NotWorkingreaching toan(1,3) 2 (1,3) 2(-2,-2)(1,3) 2,44 (1,3) 2,28 (1,3) 2,39 (1,3) 2,35 (1,3) 2,28 (1,3) 2,31arbitrarypointandoscillates(1,3) 1 (1,3) 1(-4,-4)(1,3) 2,36 (1,3) 2,27 (1,3) 2,26 (1,3) 2,38 (1,3) 2,26 (1,3) 2,27around thepoint.(1,3) 1 (1,3) 1(-6,-6)(1,3) (2,38) (1,3) 2,31 (1,3) 2,36 (1,3) 2,36 (1,3) 2,25 (1,3) 2,31 (1,3) 1 (1,3) 1DFP and BFGS works very well for Booth functionPowell’s method works nice for proper range ofalpha. A bit large range of alpha can be taken asthere is only a few local minima of the function.For range of alpha between 0 and 10 it did not workbecause in some steps for ensuring the step to bedescent the value of alpha is coming out to benegative.
  9. 9. Himmelblau function2 2 2 2( , ) ( 11) ( 7)f x y x y x y3d plot of Himmelblau function
  10. 10. Himmelblau functionContour plot of Himmelblau function
  11. 11. Himmelblau functionInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)Final Point# ofiterationsFinalPoint# ofiterationsFinal Point# ofiterationsFinal Point# ofiterationsFinal Point# ofiterationsFinal Point# ofiterations(0,0) (3,2) 8 (3,2) 8 (3,2) 8 (3,2) 6 (3,2) 6 (3,2) 6(1,1)(-3.7793,-3.2832)28 (3,2) 13 (3,2) 13 (3,2) 7(-3.7793,-3.2832)7(-3.7793,-3.2832)7(3,3) (3,2) 12 (3,2) 12 (3,2) 12 (3,2) 7 (3,2) 6 (3,2) 6(5,5)(-3.7793,-3.2832)12 (3,2) 11 (3,2) 11(-3.7793,-3.2832)8 (3,2) 6 (3,2) 7(-2,-2)(-2.8051,3.1313)12 (3,2) 18 (3,2) 8(-3.7793,-3.2832)12(-2.8051,3.1313)8 (3,2) 7(-4,-4)(-2.8051,3.1313)11 (3,2) 17 (3,2) 50(-2.8051,3.1313)12(-2.8051,3.1313)7(3.5844,-1.8481)7(-6,-6)(-3.7793,-3.2832)29 (3,2) 36(-3.7793,-3.2832)5(-2.8051,3.1313)18(-3.7793,-3.2832)9(-3.7793,-3.2832)5 P-R works better than F-R (as per # of iterations).α has no significant effect on # of iterations.
  12. 12. Himmelblau functionInitialPointDavidon-Fletcher-Powell (DFP) methodBroyden-Fletcher-Goldfarb-Shanno (BFGS)method Powells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterations Final Point# ofiterations Final Point# ofiterations Final Point# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)(3,2) 9 (3,2) 9 (3,2) 9 (3,2) 9 (3,2) 9 (3,2) 9 (3.02,1.996)1(1,1)(3,2) 7(-3.7793, -3.2832)6 (3,2) 13 (3,2) 7(-3.7793,-3.2832)6 (3,2) 13 (3.00,1.99)1(3,3)(3,2) 6 (3,2) 6 (3,2) 6 (3,2) 6 (3,2) 6 (3,2)6notworking(3.01,2.00)1(5,5)(-3.7793,-3.2832)7 (3,2) 7 (3,2) 7(-3.7793, -3.2832)7 (3,2) 7 (3,2) 7 (3.00,2.00)2(-2,-2)(-3.7793,-3.2832)6(-3.7793, -3.2832)6 (3,2) 8(-3.7793,-3.2832)6 (3,2) 11(-2.8051,3.1313)12 (3.5844, -1.8481)2(-4,-4)(-2.8051,3.1313)7(-2.8051,3.1313)7(3.5844,-1.8481)7(-2.8051,3.1313)7(-2.8051,3.1313)7(3.5844,-1.8481)16(-3.7789, -3.2832)2(-6,-6)(3,2) 8 (3,2) 8(-3.7793,-3.2832)5 (3,2) 8 (3,2) 8(-3.7793,-3.2832)5(-3.7789, -3.2832)2For range of alpha -50 to 50 the algorithm may somehow help reach near local minimstill does not converge, rather it gets distracted to another arbitrary point. For range ofdoes not provide descent direction at each step.
  13. 13. For alpha -50,50 starting from(3,3) ..though there is local minima in the vicinistill it oscillates and does not convergeiterationsx
  14. 14. Beale function2 2 2 3 2( , ) (1.5 ) (2.25 ) (2.625 )f x y x xy x xy x xy3d plot of Beale function
  15. 15. Beale functionContour plot of Beale function
  16. 16. Beale functionInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0) (3,0.5) 15(3.0001,0.5)17(2.9999,0.5)15 (3,0.5) 9 (3,0.5) 9 (3,0.5) 9(1,1) (3,0.5) 14 (3,0.5) 14 (3,0.5) 11 (3,0.5) 17(3.0001,0.5)16 (3,0.5) 11(3,3) Infinite(2.9999,0.5)11 Does not converge Does not converge (3,0.5) 10 Does not converge(5,5) Infinite Infinite Does not converge Does not converge Infinite (3,0.5) 12(-2,-2) Does not converge (3,0.5) 10 (3,0.5) 10 Does not converge(3.0001,0.5)8(3.0001,0.5)8(-4,-4) Infinite Does not converge Does not converge (3,0.5) 10 (3,0.5) 10 (3,0.5) 22(-6,-6) Infinite (3,0.5) 20 (3,0.5) 21 Infinite(3.0001,0.5)38 Does not converge
  17. 17. Beale functionx1x2Beale function for (3,3) initial point and α=(0,10) for P-R method
  18. 18. Beale functionx1x2Beale function for (3,3) initial point and α=(-50,50) for F-R method
  19. 19. Beale functionInitialPointDavidon-Fletcher-Powell (DFP) method Broyden-Fletcher-Goldfarb-Shanno (BFGS) method Powells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)(3,0.5) 8 (3,0.5) 8 (3,0.5) 8 (3,0.5) 8 (3,0.5) 8 (3,0.5) 8 (3,0.5)4(3,0.5)4(3,0.5)3(1,1)(3,0.5) 10 (3,0.5) 9 (3,0.5) 9 (3,0.5) 9 (3,0.5) 13 (3,0.5) 15(11,1) 2(-4.04,1.21)7Convergingveryslowly(3,3)NaN 100 (3,0.5) 10 NaN 100 NaN 100 (3,0.5) 10 varying 100notvarying(3,3)__(-0.1198,3.0003) 3(5,5)(3,0.5) 118 (3,0.5) 12 (3,0.5) 947.93,.8650 (3,0.5) 12 (3,0.5) 40notvarying(5,5)may begetstuckto asaddlepoint(0,5) 3(-2,-2)NaN 100 (3,0.5) 10 (3,0.5) 8 varying 100 (3,0.5) 7 (3,0.5) 7 (3,0.5)4(3,0.5)4(-4,-4)(3,0.5) 117 (3,0.5) 10 (3,0.5) 15 (3,0.5) 10 (3,0.5) 10 (3,0.5) 12(4.7703,0.7383) 1(4.7188,0.7384) 4(-6,-6)(20.66,0.95)50 (3,0.5) 21 NaN 100(-113.86,1.0074)8 (3,0.5) 21 varying 100(6.842,.832)(6.79,0.83)3For range of alpha between -50 to 50 alpha as well as x oscillates between large range probably because it gets distractedtoo much in some iteration steps and converges very slowly sometimes.
  20. 20. Ackley function2 20.2 0.5( ) 0.5(cos(2 ) cos(2 ))( , ) 20 20x y x yf x y e e e3d plot of Ackley function
  21. 21. Ackley functionContour plot of Ackley function
  22. 22. Ackley functionInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)Infinite Infinite Infinite Infinite Infinite Infinite(1,1)(0.9685,0.9685)4(-0.9685,-0.9685)3 (-57,-57) 2(0.9685,0.9685)4(-0.9685,-0.9685)3 (-57,-57) 2(3,3)(0.9685,0.9685)4(-1.9745,-1.97453 (-82,-82) 4(0.9685,0.9685)4(-1.9745,-1.97453 (-82,-82) 4(5,5)(-0.9685,-0.9685)4(-1.9745,-1.97454 (83,83) 4(-0.9685,-0.9685)4(-1.9745,-1.97454 (83,83) 4(-2,-2)(-0.9685,-0.9685)4(-1.9745,-1.97453 (83,83) 4(-0.9685,-0.9685)4(-1.9745,-1.97453 (83,83) 4(-4,-4)(-0.9685,-0.9685)4(1.9745,1.9745)4 (-83,-83) 4(-0.9685,-0.9685)4(1.9745,1.9745)4 (-83,-83) 4(-6,-6)(-0.9685,-0.9685)2(-1.9745,-1.97454 (-83,-83) 4(-0.9685,-0.9685)2(-1.9745,-1.97454 (-83,-83) 4At(0,0) gradient of f(x,y) in not defined and hence the result shows Infiniteα=(-50,50) gives local minima which are far away from the initial point.P-R and F-R gives exactly same results for all α and all initial points.
  23. 23. Ackley functionInitialPointDavidon-Fletcher-Powell (DFP) methodBroyden-Fletcher-Goldfarb-Shanno (BFGS)method Powells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)NaN 100 NaN 100 NaN 100 NaN 100 NaN 100 NaN 100oscillatingoscillatingoscillatingbut it hasbeenidentified that(1,1)(.96885,.9685)4(-.96885,-.9685)3(-57,-57)2(.96885,.9685)4(-.96885,-.9685)3(-57,-57)2oscillatingoscillatingoscillatingit hasfunction(3,3)(.96885,.9685)4(5.9887,5.9887)4(.96885,.9685)4(.96885,.9685)4(5.9887,5.9887)4(.96885,.9685)4oscillatingoscillatingoscillatingvaluezero(5,5)(49,49) 4 (0,0) 6(-295,-295)4 (49,49) 4 (0,0) 6(-274,-274)4oscillatingoscillatingoscillatingor nearlyzero(-2,-2)(-.96885,-.9685)4 (0,0) 6 (0,0) 5(-.96885,-.9685)3 (0,0) 6 (0,0) 5oscillatingoscillatingoscillatingatseveralpoints(-4,-4)(.96885,.9685)4(.96885,.9685)4(.96885,.9685)4(-17,-17)4(.96885,.9685)4(.96885,.9685)4oscillatingoscillatingoscillating(-6,-6)(-.96885,-.9685)3(-.96885,-.9685)4 (0,0) 6(-.96885,-.9685)2(-.96885,-.9685)4 (0,0) 5oscillatingvarying varying
  24. 24. It can be perceived from Powell’s method that Ackley fn has local minima atseveral pointsBut as the algorithm reaches that point algorithm does not stop, it just oscillatesaround it and one can guess that it happens possibly due to large range of alpha(the function has several local minima within a very short distance) and thisperception was validated when the algorithm reached to local or globalminimum points when we ran the algorithm for small range of alpha (-2,3)starting from several starting points!For alpha (-5,20) the value sometimes almost reaches the minima butthen bounce back to another point not close to the previous pointAckley function
  25. 25. In Powell method for Ackley function for range of alpha (-2,3) it happily reachedA minima at (0,0) within 2 iterations though powell’s method did not work wellfor large range of alphaxyAckley function
  26. 26. Goldstein-Price function2 2 2 2 2 2( , ) (1 ( 1) (19 14 3 14 6 3 ))(30 (2 3 ) (18 32 12 48 36 27 ))f x y x y x x y xy y x y x x y xy y3d plot of Goldstein-Price function
  27. 27. Goldstein-Price functionContour plot of Goldstein-Price function
  28. 28. Goldstein-Price functionInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)Infinite (0,-1) 11 (0,-1) 25 Infinite (0,-1) 8 (0,-1) 9(1,1)Infinite (0,-1) 15 (0,-1) 19 Infinite (0,-1) 10 (0,-1) 26(3,3)Infinite Infinite (0,-1) 18 Infinite Infinite (0,-1) 34(5,5)Infinite Infinite (0,-1) 39 Infinite Infinite (0,-1) 22(-2,-2)Infinite Infinite (0,-1) 33 Infinite Infinite (0,-1) 10(-4,-4)Infinite Infinite (0,-1) 49 Infinite Infinite (0,-1) 11(-6,-6)Infinite InfiniteDoes notconvergeInfinite InfiniteDoes notconverge α=(-50,50) works better compared to other α ranges.Changing α to(-100,100) results in convergence for (-6,-6) initial point also.
  29. 29. Goldstein-Price functionDFP and BFGS gives better result (less # of iterations) compared to F-R and P-R .InitialPointDavidon-Fletcher-Powell (DFP) methodBroyden-Fletcher-Goldfarb-Shanno (BFGS)methodPowells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0) (0,-1) 12 (0,-1) 6 (0,-1) 5 (0,-1) 12 (0,-1) 6 (0,-1) 5oscillating(0,-1) 1(0,-.999)1(1,1)(1.8,0.2)12 (1.8,2) 6 (0,-1) 7(1.8,0.2)12 (1.8,2) 6 (0,-1) 7 (2.986,1) 2 (0,-1) 4(0.003,-.99)1(3,3) NaN 100 NaN 100 (0,-1) 11 NaN 100 (0,-1) 17 (0,-1) 11oscillating(0,-1) 2 (0,-1) 5(5,5) NaN 100 NaN 100 (0,-1) 11 NaN 100 (0,-1) 17 (0,-1) 12 (0,-1) 3 (0,-1) 3(-2,-2)(0,-1) 16 (0,-1) 32(-.6,-.4)7 (0,-1) 16 (0,-1) 12(-.6,-.4)7(2.497,.675)4(-.6,-.4)3 (0,-1) 4(-4,-4)NaN 100(-.6,-.4)22 (0,-1) 16 NaN 100 (0,-1) 14 (0,-1) 11(2.99330.9896)3 (0,-1) 3oscillating(-6,-6)NaN 100 NaN 100 (0,-1) 10 NaN 100 (0,-1) 17 (0,-1) 10(1.84340.2297)3 (0,-1) 3*3 (0,-1) 3
  30. 30. Cross-in-tray function2 20.1100( , ) 0.0001 sin( )sin( ) 1x yf x y x y e3d plot of Cross-in -Tray function
  31. 31. Cross-in-tray functionContour plot of Cross-in -Tray function
  32. 32. Cross-in-tray functionInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0) Infinite Infinite Infinite Infinite Infinite Infinite(1,1)(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(3,3)(1.3494,1.3494)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(1.3494,1.3494)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(5,5)(4.4910,4.4910)2(4.4910,4.4910)2(4.4910,4.4910)2(4.4910,4.4910)2(4.4910,4.4910)2(4.4910,4.4910)2(-2,-2)(-1.3494,-1.3494)2(-1.3494,-1.3494)2(1.3494,1.3494)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(1.3494,1.3494)2(-4,-4)(-4.4910,-4.4910)2(-4.4910,-4.4910)2(-4.4910,-4.4910)2(-4.4910,-4.4910)2(-4.4910,-4.4910)2(-4.4910,-4.4910)2(-6,-6)(-4.4910,-4.4910)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(-4.4910,-4.4910)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2At(0,0) gradient of f(x,y) in not defined and hence the result shows InfiniteFor α=(0,10), positive initial point gives positive local minima only whichis not the case for other α.
  33. 33. Cross-in-tray functionAll methods give local minima closest to initial point.Davidon-Fletcher-Powell (DFP) method Broyden-Fletcher-Goldfarb-Shanno (BFGS) method Powells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsNaN NaN NaN NaNNaN NaNoscillating(0,-1) 1 oscillating(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2 oscillating(0,-1) 4 oscillating(1.3494,1.3494)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(1.3494,1.3494)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2oscillating(0,-1) 2oscillating(4.491,4.491)2(4.491,4.491)2(4.491,4.491)2(4.491,4.491)2(4.491,4.491)2(4.491,4.491)2 oscillating(0,-1) 3 oscillating(-1.3494,-1.3494)2(-1.3494,-1.3494)2(1.3494,1.3494)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(1.3494,1.3494)2oscillating(-.6,-.4) 3oscillating(-4.491,-4.491)2(-4.491,-4.491)2(-4.491,-4.491)2(-4.491,-4.491)2(-4.491,-4.491)2(-4.491,-4.491)2 oscillating(0,-1) 3 oscillating(-4.491,-4.491)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(-4.491,-4.491)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2oscillating(0,-1) 3*3oscillating
  34. 34. Bukin function2( , ) 100 | 0.01 | 0.01| 10|f x y y x x3d plot of Bukin function
  35. 35. Bukin function3d plot of Bukin function
  36. 36. Bukin functionInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)Infinite Infinite Infinite Infinite Infinite Infinite(1,1)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(3,3)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(5,5)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(-2,-2)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(-4,-4)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(-6,-6)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge
  37. 37. Bukin functionx1x2Bukin’s function for (-2,-2) initial point and α=(-50,50) for F-R method
  38. 38. Bukin functionInitialPointDavidon-Fletcher-Powell (DFP) method Broyden-Fletcher-Goldfarb-Shanno (BFGS) method Powells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinal Point# ofiterationsFinal Point# ofiterationsFinal Point# ofiterations(0,0)(-0.01620.0000) 1(-0.01620.0000) 2(-0.01620.0000) 1(1,1)(10,1) notconvergingproperly 1(10,1) notconvergingproperly 1varying oroscillating(3,3)NOTWORKINGvarying oroscillatingvaryingvarying oroscillating(5,5)varying oroscillatingoscillatingvarying oroscillating(-2,-2)varying oroscillating(0,0)1varying oroscillating(-4,-4)varying oroscillatingconvergesto(14.32482.0520)but rherewere somepointshaving lessfunctionvalue 6varying oroscillating(-6,-6)varying oroscillating(-0.01620.0000)1varying oroscillating
  39. 39. For Powell method alpha (-50,50) starting from (3,3) the ultimate point just oscillates fro(-17.02,3) to (17.02,3)xy
  40. 40. Rosenbrock function12 2 211( ) {100( ) ( 1) }ni i iif X x x x
  41. 41. Initial PointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0,0,0,0,0,0,0)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(1,1,1,1,1,1,1,1)(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(3,3,3,3,3,3,3,3)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(5,5,5,5,5,5,5,5)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(-2,-2,-2,-2,-2,-2,-2,-2)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(-4,-4,-4,-4,-4,-4,-4,-4)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(-6,-6,-6,-6,-6,-6,-6,-6)Does not converge Does not converge Does not converge Does not converge Does not converge Does not convergeThe point (1,1,1,1,1,1,1,1) is itself a minimum point and hence itconverges for that initial point onlyRosenbrock (8 variable) function
  42. 42. Rosenbrock (8 variable) functionF-R very slowly converges 10 the minimum point.F-R method for α=(0,10) and (-2, -2, -2, -2, -2, -2, -2, -2) initial point
  43. 43. InitialPointDavidon-Fletcher-Powell (DFP) method Broyden-Fletcher-Goldfarb-Shanno (BFGS) methodα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)Final Point# ofiterationsand # offunctionevaluations Final Point# ofiterationsand # offunctionevaluations Final Point# ofiterationsand # offunctionevaluations Final Point# ofiterationsand # offunctionevaluations Final Point# ofiterationsand # offunctionevaluations Final Point# ofiterationsand # offunctionevaluations(0,0,0,0,0,0,0,0)(1,1,1,1,1,1,1,1)59(1,1,1,1,1,1,1,1)45(1,1,1,1,1,1,1,1)45(1,1,1,1,1,1,1,1)46(1,1,1,1,1,1,1,1)45(1,1,1,1,1,1,1,1)45(1,1,1,1,1,1,1,1)(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(3,3,3,3,3,3,3,3)slowlyconverging(1,1,1,1,1,1,1,1)44(1,1,1,1,1,1,1,1)39(1,1,1,1,1,1,1,1)31(1,1,1,1,1,1,1,1)34(1,1,1,1,1,1,1,1)37(5,5,5,5,5,5,5,5)notconvergingor veryslowlyconverging(1,1,1,1,1,1,1,1)44(1,1,1,1,1,1,1,1)271(1,1,1,1,1,1,1,1)75(1,1,1,1,1,1,1,1)48(1,1,1,1,1,1,1,1)72(-2,-2,-2,-2,-2,-2,-2,-2)slowlyconverging/oscillating(1,1,1,1,1,1,1,1)69(1,1,1,1,1,1,1,1)65(1,1,1,1,1,1,1,1)52(1,1,1,1,1,1,1,1)54(1,1,1,1,1,1,1,1)51(-4,-4,-4,-4,-4,-4,-4,-4)slowlyconverging/oscillating(1,1,1,1,1,1,1,1)slowlyconverging/oscillating(1,1,1,1,1,1,1,1)64(1,1,1,1,1,1,1,1)56(1,1,1,1,1,1,1,1)41(1,1,1,1,1,1,1,1)56(-6,-6,-6,-6,-6,-6,-6,-6)(1,1,1,1,1,1,1,1)30(1,1,1,1,1,1,1,1)126(1,1,1,1,1,1,1,1)182(1,1,1,1,1,1,1,1)23(1,1,1,1,1,1,1,1)37(1,1,1,1,1,1,1,1)62Rosenbrock(8 variable) function
  44. 44. Conclusions For quadratic problems, all methods gives satisfactory results.Powell’s method is working satisfactorily for small (i.e.lower-dimensional)problems [particularly for α=(-5,20)]DFP-BFGS works very good for bad functions (e.g Goldstein Pricefunction) where F-R and P-R does not work well.Suitability of DFP, BFGS for higher dimensional problems need to bestudied more.Range of step size if chosen small and searches on both side of the givenpoint [α=(-5,20)], it works well for all problems.

×