• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Comparative study of algorithms of nonlinear optimization
 

Comparative study of algorithms of nonlinear optimization

on

  • 143 views

 

Statistics

Views

Total Views
143
Views on SlideShare
143
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Comparative study of algorithms of nonlinear optimization Comparative study of algorithms of nonlinear optimization Presentation Transcript

    • Presented byPritam BhadraPranamesh ChakrabortyIndian Institute of Technology, Kanpur11 May 2013Comparative study of algorithms ofNonlinear Optimization
    • Methods of Nonlinear OptimizationThe methods for nonlinear optimization are:1. Conjugate gradient methodsa) Fletcher-Reevesb) Polak- Ribiere2. Powell’s conjugate direction method3. Quasi-Newton methodsa) Davidon-Fletcher-Powell (DFP) methodb) Broyden-Fletcher-Goldfarb-Shanno (BFGS) method
    • Functions used for comparison of algorithms1. Booth function2. Himmelblau function3. Beale function4. Ackley function5. Goldstein function6. Cross-in-tray function7. Bukin function8. Rosenbrock function
    • For each function(two-dimensional), local minima is obtainedfrom 6 initial points:a) (0,0)b) (1,1)c) (3,3)d) (5,5)e) (-2,-2)f) (-4,-4)g) (-6,-6)for 3 ranges of α (step-size) (0,10) (-5,20) (-50,50)
    • Booth function2 2( , ) ( 2 7) (2 5)f x y x y x yQuadratic function of 2 variables3d plot of Booth function
    • Booth functionContour plot of Booth function
    • Booth function Exact same results for F-R and P-R since the function is quadraticInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# offunctionevaluations# ofiterationsFinalPoint# offunctionevaluations# ofiterationsFinalPoint# offunctionevaluations# ofiterationsFinalPoint# offunctionevaluations# ofiterationsFinalPoint# offunctionevaluations# ofiterationsFinalPoint# offunctionevaluations# ofiterations(0,0) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2(1,1) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2(3,3) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2(5,5) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2(-2,-2) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2(-4,-4) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2(-6,-6) (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 (1,3) 20 2 (1,3) 15 2 (1,3) 15 2 The algorithm converges in 2 steps as it is a 2-d quadratic problem
    • Booth functionInitialPointDavidon-Fletcher-Powell (DFP)methodBroyden-Fletcher-Goldfarb-Shanno(BFGS) method Powells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)(1,3) 2,36 (1,3) 2,29 (1,3) 2,23 (1,3) 2,40 (1,3) 2,31 (1,3) 2,26(3.4,1.08)(1,3) 1 (1,3) 1(1,1)(1,3) 2,43 (1,3) 2,29 (1,3) 2,38 (1,3) 2,42 (1,3) 2,28 (1,3) 2,32(2.6,1.72)(1,3) 1 (1,3) 1(3,3)(1,3) 2,35 (1,3) 2,29 (1,3) 2,32 (1,3) 2,44 (1,3) 2,29 (1,3) 2,31 (1,3) 1 (1,3) 1(5,5)(1,3) 2,32 (1,3) 2,27 (1,3) 2,22 (1,3) 2,37 (1,3) 2,28 (1,3) 2,24NotWorkingreaching toan(1,3) 2 (1,3) 2(-2,-2)(1,3) 2,44 (1,3) 2,28 (1,3) 2,39 (1,3) 2,35 (1,3) 2,28 (1,3) 2,31arbitrarypointandoscillates(1,3) 1 (1,3) 1(-4,-4)(1,3) 2,36 (1,3) 2,27 (1,3) 2,26 (1,3) 2,38 (1,3) 2,26 (1,3) 2,27around thepoint.(1,3) 1 (1,3) 1(-6,-6)(1,3) (2,38) (1,3) 2,31 (1,3) 2,36 (1,3) 2,36 (1,3) 2,25 (1,3) 2,31 (1,3) 1 (1,3) 1DFP and BFGS works very well for Booth functionPowell’s method works nice for proper range ofalpha. A bit large range of alpha can be taken asthere is only a few local minima of the function.For range of alpha between 0 and 10 it did not workbecause in some steps for ensuring the step to bedescent the value of alpha is coming out to benegative.
    • Himmelblau function2 2 2 2( , ) ( 11) ( 7)f x y x y x y3d plot of Himmelblau function
    • Himmelblau functionContour plot of Himmelblau function
    • Himmelblau functionInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)Final Point# ofiterationsFinalPoint# ofiterationsFinal Point# ofiterationsFinal Point# ofiterationsFinal Point# ofiterationsFinal Point# ofiterations(0,0) (3,2) 8 (3,2) 8 (3,2) 8 (3,2) 6 (3,2) 6 (3,2) 6(1,1)(-3.7793,-3.2832)28 (3,2) 13 (3,2) 13 (3,2) 7(-3.7793,-3.2832)7(-3.7793,-3.2832)7(3,3) (3,2) 12 (3,2) 12 (3,2) 12 (3,2) 7 (3,2) 6 (3,2) 6(5,5)(-3.7793,-3.2832)12 (3,2) 11 (3,2) 11(-3.7793,-3.2832)8 (3,2) 6 (3,2) 7(-2,-2)(-2.8051,3.1313)12 (3,2) 18 (3,2) 8(-3.7793,-3.2832)12(-2.8051,3.1313)8 (3,2) 7(-4,-4)(-2.8051,3.1313)11 (3,2) 17 (3,2) 50(-2.8051,3.1313)12(-2.8051,3.1313)7(3.5844,-1.8481)7(-6,-6)(-3.7793,-3.2832)29 (3,2) 36(-3.7793,-3.2832)5(-2.8051,3.1313)18(-3.7793,-3.2832)9(-3.7793,-3.2832)5 P-R works better than F-R (as per # of iterations).α has no significant effect on # of iterations.
    • Himmelblau functionInitialPointDavidon-Fletcher-Powell (DFP) methodBroyden-Fletcher-Goldfarb-Shanno (BFGS)method Powells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterations Final Point# ofiterations Final Point# ofiterations Final Point# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)(3,2) 9 (3,2) 9 (3,2) 9 (3,2) 9 (3,2) 9 (3,2) 9 (3.02,1.996)1(1,1)(3,2) 7(-3.7793, -3.2832)6 (3,2) 13 (3,2) 7(-3.7793,-3.2832)6 (3,2) 13 (3.00,1.99)1(3,3)(3,2) 6 (3,2) 6 (3,2) 6 (3,2) 6 (3,2) 6 (3,2)6notworking(3.01,2.00)1(5,5)(-3.7793,-3.2832)7 (3,2) 7 (3,2) 7(-3.7793, -3.2832)7 (3,2) 7 (3,2) 7 (3.00,2.00)2(-2,-2)(-3.7793,-3.2832)6(-3.7793, -3.2832)6 (3,2) 8(-3.7793,-3.2832)6 (3,2) 11(-2.8051,3.1313)12 (3.5844, -1.8481)2(-4,-4)(-2.8051,3.1313)7(-2.8051,3.1313)7(3.5844,-1.8481)7(-2.8051,3.1313)7(-2.8051,3.1313)7(3.5844,-1.8481)16(-3.7789, -3.2832)2(-6,-6)(3,2) 8 (3,2) 8(-3.7793,-3.2832)5 (3,2) 8 (3,2) 8(-3.7793,-3.2832)5(-3.7789, -3.2832)2For range of alpha -50 to 50 the algorithm may somehow help reach near local minimstill does not converge, rather it gets distracted to another arbitrary point. For range ofdoes not provide descent direction at each step.
    • For alpha -50,50 starting from(3,3) ..though there is local minima in the vicinistill it oscillates and does not convergeiterationsx
    • Beale function2 2 2 3 2( , ) (1.5 ) (2.25 ) (2.625 )f x y x xy x xy x xy3d plot of Beale function
    • Beale functionContour plot of Beale function
    • Beale functionInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0) (3,0.5) 15(3.0001,0.5)17(2.9999,0.5)15 (3,0.5) 9 (3,0.5) 9 (3,0.5) 9(1,1) (3,0.5) 14 (3,0.5) 14 (3,0.5) 11 (3,0.5) 17(3.0001,0.5)16 (3,0.5) 11(3,3) Infinite(2.9999,0.5)11 Does not converge Does not converge (3,0.5) 10 Does not converge(5,5) Infinite Infinite Does not converge Does not converge Infinite (3,0.5) 12(-2,-2) Does not converge (3,0.5) 10 (3,0.5) 10 Does not converge(3.0001,0.5)8(3.0001,0.5)8(-4,-4) Infinite Does not converge Does not converge (3,0.5) 10 (3,0.5) 10 (3,0.5) 22(-6,-6) Infinite (3,0.5) 20 (3,0.5) 21 Infinite(3.0001,0.5)38 Does not converge
    • Beale functionx1x2Beale function for (3,3) initial point and α=(0,10) for P-R method
    • Beale functionx1x2Beale function for (3,3) initial point and α=(-50,50) for F-R method
    • Beale functionInitialPointDavidon-Fletcher-Powell (DFP) method Broyden-Fletcher-Goldfarb-Shanno (BFGS) method Powells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)(3,0.5) 8 (3,0.5) 8 (3,0.5) 8 (3,0.5) 8 (3,0.5) 8 (3,0.5) 8 (3,0.5)4(3,0.5)4(3,0.5)3(1,1)(3,0.5) 10 (3,0.5) 9 (3,0.5) 9 (3,0.5) 9 (3,0.5) 13 (3,0.5) 15(11,1) 2(-4.04,1.21)7Convergingveryslowly(3,3)NaN 100 (3,0.5) 10 NaN 100 NaN 100 (3,0.5) 10 varying 100notvarying(3,3)__(-0.1198,3.0003) 3(5,5)(3,0.5) 118 (3,0.5) 12 (3,0.5) 947.93,.8650 (3,0.5) 12 (3,0.5) 40notvarying(5,5)may begetstuckto asaddlepoint(0,5) 3(-2,-2)NaN 100 (3,0.5) 10 (3,0.5) 8 varying 100 (3,0.5) 7 (3,0.5) 7 (3,0.5)4(3,0.5)4(-4,-4)(3,0.5) 117 (3,0.5) 10 (3,0.5) 15 (3,0.5) 10 (3,0.5) 10 (3,0.5) 12(4.7703,0.7383) 1(4.7188,0.7384) 4(-6,-6)(20.66,0.95)50 (3,0.5) 21 NaN 100(-113.86,1.0074)8 (3,0.5) 21 varying 100(6.842,.832)(6.79,0.83)3For range of alpha between -50 to 50 alpha as well as x oscillates between large range probably because it gets distractedtoo much in some iteration steps and converges very slowly sometimes.
    • Ackley function2 20.2 0.5( ) 0.5(cos(2 ) cos(2 ))( , ) 20 20x y x yf x y e e e3d plot of Ackley function
    • Ackley functionContour plot of Ackley function
    • Ackley functionInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)Infinite Infinite Infinite Infinite Infinite Infinite(1,1)(0.9685,0.9685)4(-0.9685,-0.9685)3 (-57,-57) 2(0.9685,0.9685)4(-0.9685,-0.9685)3 (-57,-57) 2(3,3)(0.9685,0.9685)4(-1.9745,-1.97453 (-82,-82) 4(0.9685,0.9685)4(-1.9745,-1.97453 (-82,-82) 4(5,5)(-0.9685,-0.9685)4(-1.9745,-1.97454 (83,83) 4(-0.9685,-0.9685)4(-1.9745,-1.97454 (83,83) 4(-2,-2)(-0.9685,-0.9685)4(-1.9745,-1.97453 (83,83) 4(-0.9685,-0.9685)4(-1.9745,-1.97453 (83,83) 4(-4,-4)(-0.9685,-0.9685)4(1.9745,1.9745)4 (-83,-83) 4(-0.9685,-0.9685)4(1.9745,1.9745)4 (-83,-83) 4(-6,-6)(-0.9685,-0.9685)2(-1.9745,-1.97454 (-83,-83) 4(-0.9685,-0.9685)2(-1.9745,-1.97454 (-83,-83) 4At(0,0) gradient of f(x,y) in not defined and hence the result shows Infiniteα=(-50,50) gives local minima which are far away from the initial point.P-R and F-R gives exactly same results for all α and all initial points.
    • Ackley functionInitialPointDavidon-Fletcher-Powell (DFP) methodBroyden-Fletcher-Goldfarb-Shanno (BFGS)method Powells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)NaN 100 NaN 100 NaN 100 NaN 100 NaN 100 NaN 100oscillatingoscillatingoscillatingbut it hasbeenidentified that(1,1)(.96885,.9685)4(-.96885,-.9685)3(-57,-57)2(.96885,.9685)4(-.96885,-.9685)3(-57,-57)2oscillatingoscillatingoscillatingit hasfunction(3,3)(.96885,.9685)4(5.9887,5.9887)4(.96885,.9685)4(.96885,.9685)4(5.9887,5.9887)4(.96885,.9685)4oscillatingoscillatingoscillatingvaluezero(5,5)(49,49) 4 (0,0) 6(-295,-295)4 (49,49) 4 (0,0) 6(-274,-274)4oscillatingoscillatingoscillatingor nearlyzero(-2,-2)(-.96885,-.9685)4 (0,0) 6 (0,0) 5(-.96885,-.9685)3 (0,0) 6 (0,0) 5oscillatingoscillatingoscillatingatseveralpoints(-4,-4)(.96885,.9685)4(.96885,.9685)4(.96885,.9685)4(-17,-17)4(.96885,.9685)4(.96885,.9685)4oscillatingoscillatingoscillating(-6,-6)(-.96885,-.9685)3(-.96885,-.9685)4 (0,0) 6(-.96885,-.9685)2(-.96885,-.9685)4 (0,0) 5oscillatingvarying varying
    • It can be perceived from Powell’s method that Ackley fn has local minima atseveral pointsBut as the algorithm reaches that point algorithm does not stop, it just oscillatesaround it and one can guess that it happens possibly due to large range of alpha(the function has several local minima within a very short distance) and thisperception was validated when the algorithm reached to local or globalminimum points when we ran the algorithm for small range of alpha (-2,3)starting from several starting points!For alpha (-5,20) the value sometimes almost reaches the minima butthen bounce back to another point not close to the previous pointAckley function
    • In Powell method for Ackley function for range of alpha (-2,3) it happily reachedA minima at (0,0) within 2 iterations though powell’s method did not work wellfor large range of alphaxyAckley function
    • Goldstein-Price function2 2 2 2 2 2( , ) (1 ( 1) (19 14 3 14 6 3 ))(30 (2 3 ) (18 32 12 48 36 27 ))f x y x y x x y xy y x y x x y xy y3d plot of Goldstein-Price function
    • Goldstein-Price functionContour plot of Goldstein-Price function
    • Goldstein-Price functionInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)Infinite (0,-1) 11 (0,-1) 25 Infinite (0,-1) 8 (0,-1) 9(1,1)Infinite (0,-1) 15 (0,-1) 19 Infinite (0,-1) 10 (0,-1) 26(3,3)Infinite Infinite (0,-1) 18 Infinite Infinite (0,-1) 34(5,5)Infinite Infinite (0,-1) 39 Infinite Infinite (0,-1) 22(-2,-2)Infinite Infinite (0,-1) 33 Infinite Infinite (0,-1) 10(-4,-4)Infinite Infinite (0,-1) 49 Infinite Infinite (0,-1) 11(-6,-6)Infinite InfiniteDoes notconvergeInfinite InfiniteDoes notconverge α=(-50,50) works better compared to other α ranges.Changing α to(-100,100) results in convergence for (-6,-6) initial point also.
    • Goldstein-Price functionDFP and BFGS gives better result (less # of iterations) compared to F-R and P-R .InitialPointDavidon-Fletcher-Powell (DFP) methodBroyden-Fletcher-Goldfarb-Shanno (BFGS)methodPowells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0) (0,-1) 12 (0,-1) 6 (0,-1) 5 (0,-1) 12 (0,-1) 6 (0,-1) 5oscillating(0,-1) 1(0,-.999)1(1,1)(1.8,0.2)12 (1.8,2) 6 (0,-1) 7(1.8,0.2)12 (1.8,2) 6 (0,-1) 7 (2.986,1) 2 (0,-1) 4(0.003,-.99)1(3,3) NaN 100 NaN 100 (0,-1) 11 NaN 100 (0,-1) 17 (0,-1) 11oscillating(0,-1) 2 (0,-1) 5(5,5) NaN 100 NaN 100 (0,-1) 11 NaN 100 (0,-1) 17 (0,-1) 12 (0,-1) 3 (0,-1) 3(-2,-2)(0,-1) 16 (0,-1) 32(-.6,-.4)7 (0,-1) 16 (0,-1) 12(-.6,-.4)7(2.497,.675)4(-.6,-.4)3 (0,-1) 4(-4,-4)NaN 100(-.6,-.4)22 (0,-1) 16 NaN 100 (0,-1) 14 (0,-1) 11(2.99330.9896)3 (0,-1) 3oscillating(-6,-6)NaN 100 NaN 100 (0,-1) 10 NaN 100 (0,-1) 17 (0,-1) 10(1.84340.2297)3 (0,-1) 3*3 (0,-1) 3
    • Cross-in-tray function2 20.1100( , ) 0.0001 sin( )sin( ) 1x yf x y x y e3d plot of Cross-in -Tray function
    • Cross-in-tray functionContour plot of Cross-in -Tray function
    • Cross-in-tray functionInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0) Infinite Infinite Infinite Infinite Infinite Infinite(1,1)(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(3,3)(1.3494,1.3494)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(1.3494,1.3494)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(5,5)(4.4910,4.4910)2(4.4910,4.4910)2(4.4910,4.4910)2(4.4910,4.4910)2(4.4910,4.4910)2(4.4910,4.4910)2(-2,-2)(-1.3494,-1.3494)2(-1.3494,-1.3494)2(1.3494,1.3494)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(1.3494,1.3494)2(-4,-4)(-4.4910,-4.4910)2(-4.4910,-4.4910)2(-4.4910,-4.4910)2(-4.4910,-4.4910)2(-4.4910,-4.4910)2(-4.4910,-4.4910)2(-6,-6)(-4.4910,-4.4910)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(-4.4910,-4.4910)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2At(0,0) gradient of f(x,y) in not defined and hence the result shows InfiniteFor α=(0,10), positive initial point gives positive local minima only whichis not the case for other α.
    • Cross-in-tray functionAll methods give local minima closest to initial point.Davidon-Fletcher-Powell (DFP) method Broyden-Fletcher-Goldfarb-Shanno (BFGS) method Powells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsNaN NaN NaN NaNNaN NaNoscillating(0,-1) 1 oscillating(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2(1.3494,1.3494)2 oscillating(0,-1) 4 oscillating(1.3494,1.3494)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(1.3494,1.3494)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2oscillating(0,-1) 2oscillating(4.491,4.491)2(4.491,4.491)2(4.491,4.491)2(4.491,4.491)2(4.491,4.491)2(4.491,4.491)2 oscillating(0,-1) 3 oscillating(-1.3494,-1.3494)2(-1.3494,-1.3494)2(1.3494,1.3494)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(1.3494,1.3494)2oscillating(-.6,-.4) 3oscillating(-4.491,-4.491)2(-4.491,-4.491)2(-4.491,-4.491)2(-4.491,-4.491)2(-4.491,-4.491)2(-4.491,-4.491)2 oscillating(0,-1) 3 oscillating(-4.491,-4.491)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2(-4.491,-4.491)2(-1.3494,-1.3494)2(-1.3494,-1.3494)2oscillating(0,-1) 3*3oscillating
    • Bukin function2( , ) 100 | 0.01 | 0.01| 10|f x y y x x3d plot of Bukin function
    • Bukin function3d plot of Bukin function
    • Bukin functionInitialPointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0)Infinite Infinite Infinite Infinite Infinite Infinite(1,1)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(3,3)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(5,5)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(-2,-2)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(-4,-4)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(-6,-6)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge
    • Bukin functionx1x2Bukin’s function for (-2,-2) initial point and α=(-50,50) for F-R method
    • Bukin functionInitialPointDavidon-Fletcher-Powell (DFP) method Broyden-Fletcher-Goldfarb-Shanno (BFGS) method Powells conjugate direction methodα α α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinal Point# ofiterationsFinal Point# ofiterationsFinal Point# ofiterations(0,0)(-0.01620.0000) 1(-0.01620.0000) 2(-0.01620.0000) 1(1,1)(10,1) notconvergingproperly 1(10,1) notconvergingproperly 1varying oroscillating(3,3)NOTWORKINGvarying oroscillatingvaryingvarying oroscillating(5,5)varying oroscillatingoscillatingvarying oroscillating(-2,-2)varying oroscillating(0,0)1varying oroscillating(-4,-4)varying oroscillatingconvergesto(14.32482.0520)but rherewere somepointshaving lessfunctionvalue 6varying oroscillating(-6,-6)varying oroscillating(-0.01620.0000)1varying oroscillating
    • For Powell method alpha (-50,50) starting from (3,3) the ultimate point just oscillates fro(-17.02,3) to (17.02,3)xy
    • Rosenbrock function12 2 211( ) {100( ) ( 1) }ni i iif X x x x
    • Initial PointFletcher-Reeves Polak-Ribiereα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)FinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterationsFinalPoint# ofiterations(0,0,0,0,0,0,0,0)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(1,1,1,1,1,1,1,1)(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(3,3,3,3,3,3,3,3)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(5,5,5,5,5,5,5,5)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(-2,-2,-2,-2,-2,-2,-2,-2)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(-4,-4,-4,-4,-4,-4,-4,-4)Does not converge Does not converge Does not converge Does not converge Does not converge Does not converge(-6,-6,-6,-6,-6,-6,-6,-6)Does not converge Does not converge Does not converge Does not converge Does not converge Does not convergeThe point (1,1,1,1,1,1,1,1) is itself a minimum point and hence itconverges for that initial point onlyRosenbrock (8 variable) function
    • Rosenbrock (8 variable) functionF-R very slowly converges 10 the minimum point.F-R method for α=(0,10) and (-2, -2, -2, -2, -2, -2, -2, -2) initial point
    • InitialPointDavidon-Fletcher-Powell (DFP) method Broyden-Fletcher-Goldfarb-Shanno (BFGS) methodα α(0,10) (-5,20) (-50,50) (0,10) (-5,20) (-50,50)Final Point# ofiterationsand # offunctionevaluations Final Point# ofiterationsand # offunctionevaluations Final Point# ofiterationsand # offunctionevaluations Final Point# ofiterationsand # offunctionevaluations Final Point# ofiterationsand # offunctionevaluations Final Point# ofiterationsand # offunctionevaluations(0,0,0,0,0,0,0,0)(1,1,1,1,1,1,1,1)59(1,1,1,1,1,1,1,1)45(1,1,1,1,1,1,1,1)45(1,1,1,1,1,1,1,1)46(1,1,1,1,1,1,1,1)45(1,1,1,1,1,1,1,1)45(1,1,1,1,1,1,1,1)(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(1,1,1,1,1,1,1,1)1(3,3,3,3,3,3,3,3)slowlyconverging(1,1,1,1,1,1,1,1)44(1,1,1,1,1,1,1,1)39(1,1,1,1,1,1,1,1)31(1,1,1,1,1,1,1,1)34(1,1,1,1,1,1,1,1)37(5,5,5,5,5,5,5,5)notconvergingor veryslowlyconverging(1,1,1,1,1,1,1,1)44(1,1,1,1,1,1,1,1)271(1,1,1,1,1,1,1,1)75(1,1,1,1,1,1,1,1)48(1,1,1,1,1,1,1,1)72(-2,-2,-2,-2,-2,-2,-2,-2)slowlyconverging/oscillating(1,1,1,1,1,1,1,1)69(1,1,1,1,1,1,1,1)65(1,1,1,1,1,1,1,1)52(1,1,1,1,1,1,1,1)54(1,1,1,1,1,1,1,1)51(-4,-4,-4,-4,-4,-4,-4,-4)slowlyconverging/oscillating(1,1,1,1,1,1,1,1)slowlyconverging/oscillating(1,1,1,1,1,1,1,1)64(1,1,1,1,1,1,1,1)56(1,1,1,1,1,1,1,1)41(1,1,1,1,1,1,1,1)56(-6,-6,-6,-6,-6,-6,-6,-6)(1,1,1,1,1,1,1,1)30(1,1,1,1,1,1,1,1)126(1,1,1,1,1,1,1,1)182(1,1,1,1,1,1,1,1)23(1,1,1,1,1,1,1,1)37(1,1,1,1,1,1,1,1)62Rosenbrock(8 variable) function
    • Conclusions For quadratic problems, all methods gives satisfactory results.Powell’s method is working satisfactorily for small (i.e.lower-dimensional)problems [particularly for α=(-5,20)]DFP-BFGS works very good for bad functions (e.g Goldstein Pricefunction) where F-R and P-R does not work well.Suitability of DFP, BFGS for higher dimensional problems need to bestudied more.Range of step size if chosen small and searches on both side of the givenpoint [α=(-5,20)], it works well for all problems.