Quasi-Newton Differential Dynamic Programmingfor Robust Low-Thrust OptimizationEtienne Pellegrini and Ryan P. RussellAIAA/...
Summary• Introduction• The Hybrid Differential Dynamic Programming (HDDP)Algorithm [Lantoine & Russell]– State-Transition ...
State of the ArtLow thrust trajectories Highly nonlinear, constrained problems Need for specific and efficient NLP solve...
Classic NLP Solvers DDP MethodsIntroduction4 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 ...
Classic NLP Solvers HDDP MethodIntroduction5 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 ...
The HDDP algorithm6 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
The HDDP algorithm: STM approachSensitivities are obtained usingthe STMs• Initialize 𝐽 𝑥,𝑁∗(𝑥) and 𝐽 𝑥𝑥,𝑁∗(𝑥)• 𝐽 𝑥,𝑘 𝑥, 𝑢 ...
• Decouples the optimization step from the propagation step– Allows for parallelization of the computation– Allows for app...
• Introduced in 1959 [Davidon]• Used in many optimization applications• Aim: approximating the curvature of the problem E...
Application to HDDP: estimating 𝚽 𝟐,𝒌• Different from traditional quasi-Newton:– Not as suitable to estimate the Hessian o...
SR1 Update• Variety of quasi-Newton updates have been developed– BFGS, DFP, Powell’s Damped BFGS, SR1, etc…• Most of them:...
Results: Framework• Tested on a set of 6 fixed final time problems• Implemented using Matlab. Similar results are expected...
Results: 1D Landing13 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MNRun ti...
Results: 2D Spacecraft Problem• Transfer between two coplanar circular orbits; minimize fuel14 Etienne Pellegrini – AIAA/A...
Metric value for 4 different strategies Run time for different strategiesOtherResults: 2D Spacecraft Problem• Different sc...
• Similar problem, longer time of flight (35 TU), lower maximumthrust (0.05 MU.LU/TU2)• Bang-bang structure as expectedRes...
Results: Complete Set• Comparison of all test cases• Metric: 2nd order STM well approximated for most cases• Run time: sho...
Conclusions• Possibility of restarting the estimate with the real STM inorder to improve confidenceEtienne Pellegrini – AI...
Future Work• Testing on representative space trajectories• Use of multi-step quasi-Newton methods• Other updates• Integrat...
Thank you for your attention20 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis,...
Backup Slides21 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
Set of test problems22 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
• Small perturbation to the state:(1)• Taylor series:(2)• Replace 𝛿𝑋 in (1):(3)• Equate (2) and (3):23Derivation of the ST...
• Taylor series:• Quasi-Newton equation:• Rank-1 update:• Because 𝑎𝑢 𝑇Δ𝑌𝑝 is a scalar:• Finally:24Derivation of the SR1 up...
• 𝐽 𝑋,𝑘𝑖and 𝐽 𝑋𝑋,𝑘𝑖are function of the downstream control law(𝑢 𝑞, 𝑘 + 1 ≤ 𝑞 ≤ 𝑁)• They are only accurate for a trajectory...
“An optimal policy has the property thatwhatever the initial state and initial decisionare, the remaining decisions must c...
Upcoming SlideShare
Loading in...5
×

Quasi-Newton Differential Dynamic Programming for Robust Low-Thrust Optimization

260

Published on

Presentation given at the AIAA/AAS Astrodynamics Specialists Conference in Minneapolis, MN, on 8/13/12.
The conference proceedings are available here: http://arc.aiaa.org/doi/abs/10.2514/6.2012-4425

Low-thrust trajectories result in challenging nonlinear optimization problems. To solve these problems, efficient and robust optimization techniques are required. The increasing complexity of such problems induces a rise in the computational intensity of the nonlinear solvers. The current work is based on a Hybrid Differential Dynamic Programming (HDDP) algorithm and aims at improving its computational efficiency. A quasi-Newton based method is presented to approximate the expensive second-order sensitivities in efforts to reduce computation times, while maintaining attractive convergence properties. A variety of quasi-Newton rank-one and rank-two updates are tested using several benchmark optimal control problems, including a simple spacecraft finite thrust trajectory problem. The quasi-Newton methods are used in this study to approximate the Hessian of the state transition functions, as opposed to the Hessian of the performance index like in classic optimization applications. Accordingly, the symmetric rank-one update is found to be most suitable for this HDDP application. The approximations are demonstrated to be sufficiently close to the true Hessian for the problems considered. In comparing the full second order version of HDDP with the quasi-Newton modification, the converged iteration counts are comparable while the run times for the quasi-Newton cases show up to an order of magnitude improvement. The speedups are demonstrated to increase with the problem dimension and the degree of coupling within the dynamic equations.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
260
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Quasi-Newton Differential Dynamic Programming for Robust Low-Thrust Optimization

  1. 1. Quasi-Newton Differential Dynamic Programmingfor Robust Low-Thrust OptimizationEtienne Pellegrini and Ryan P. RussellAIAA/AAS Astrodynamics Specialists ConferenceMinneapolis, MN, 8/13/12
  2. 2. Summary• Introduction• The Hybrid Differential Dynamic Programming (HDDP)Algorithm [Lantoine & Russell]– State-Transition Matrices• Quasi-Newton methods– Application to HDDP– The SR1 update• Results– 1D Landing– 2D Spacecraft Problem [Bryson & Ho]– Complete set of test problems• Conclusions & Future work2 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  3. 3. State of the ArtLow thrust trajectories Highly nonlinear, constrained problems Need for specific and efficient NLP solvers• DDP methods were introduced in late 60s [Mayne, Jacobson]• Static/Dynamic Algorithm: uses Hessian shifting [Whiffen]• HDDP: uses State-Transition Matrices approach Motivation for this paper:High computational intensity for all those methods.3 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  4. 4. Classic NLP Solvers DDP MethodsIntroduction4 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  5. 5. Classic NLP Solvers HDDP MethodIntroduction5 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  6. 6. The HDDP algorithm6 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  7. 7. The HDDP algorithm: STM approachSensitivities are obtained usingthe STMs• Initialize 𝐽 𝑥,𝑁∗(𝑥) and 𝐽 𝑥𝑥,𝑁∗(𝑥)• 𝐽 𝑥,𝑘 𝑥, 𝑢 and 𝐽 𝑥𝑥,𝑘(𝑥, 𝑢) areobtained from backwardmapping of 𝐽 𝑥,𝑘+1∗(𝑥) and𝐽 𝑥𝑥,𝑘+1∗(𝑥)• The control law allows todeduce state only sensitivities𝐽 𝑥,𝑘∗(𝑥) and 𝐽 𝑥𝑥,𝑘∗(𝑥)7 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  8. 8. • Decouples the optimization step from the propagation step– Allows for parallelization of the computation– Allows for approximations to the partial derivatives• Forward sweep:– n equation for the state– n2 equations for the 1st order STM– n3 equations for the 2nd order STM• Propagation of the STMs takes more than 80% of thecompute time• Necessitates the user to provide the second-order partialderivatives of the state dynamics8 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MNThe HDDP algorithm: STM approach
  9. 9. • Introduced in 1959 [Davidon]• Used in many optimization applications• Aim: approximating the curvature of the problem Estimating the Hessian of the objective function• Classical approach– Gradient and estimate of the Hessian used to define a searchdirection– Step chosen with a line search or trust region method– Estimate of the Hessian is updated• Estimate of the Hessian has to be positive definite9Quasi-Newton MethodsEtienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  10. 10. Application to HDDP: estimating 𝚽 𝟐,𝒌• Different from traditional quasi-Newton:– Not as suitable to estimate the Hessian of the cost function– Estimates the 2nd order STM Results in changes to the traditional methods– No enforcement of the positive definiteness– Requires a quasi-Newton update that approximates theHessian accurately– Step decided by the propagation of the new control law– The 2nd order STM is a tensor composed of n Hessians n quasi-Newton updates to apply• Computation of the STM is decoupled: the optimizationsteps are untouched• The user does not need to provide 2nd order derivatives10 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  11. 11. SR1 Update• Variety of quasi-Newton updates have been developed– BFGS, DFP, Powell’s Damped BFGS, SR1, etc…• Most of them: enforce positive definiteness of the estimate– In classical quasi-Newton framework, a descent direction isneeded– In our application: we don’t need the estimate to be pos. def.• Symmetric Rank 1 update– Does not enforce convexity– Results in estimates closer to the true Hessian [Conn et al.]11 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  12. 12. Results: Framework• Tested on a set of 6 fixed final time problems• Implemented using Matlab. Similar results are expectedusing another programming language• Metric to evaluate how accurate the Hessian estimates are:[Khalfan et al.]• Average taken on every stage and every state.12 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  13. 13. Results: 1D Landing13 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MNRun time IterationsHDDP 22.95 11QHDDP 7.19 11Controls obtained with HDDP and QHDDPStates and controls found by QHDDP• 3 states: vertical position and velocity, and fuel• 1 control: thrust
  14. 14. Results: 2D Spacecraft Problem• Transfer between two coplanar circular orbits; minimize fuel14 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MNTrajectory obtained with QHDDP Controls obtained with HDDP and QHDDPRun time IterationsHDDP 551.27 89QHDDP 32.35 82
  15. 15. Metric value for 4 different strategies Run time for different strategiesOtherResults: 2D Spacecraft Problem• Different scenarios: Test of a restart strategy Trade-off between confidence in the estimate andcomputation time• NB: User has to provide 2nd order derivatives again15 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  16. 16. • Similar problem, longer time of flight (35 TU), lower maximumthrust (0.05 MU.LU/TU2)• Bang-bang structure as expectedResults: Multi-Rev Spacecraft Problem16 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MNThrust and eccentricity (QHDDP)0 10 20 300.060.040.20Thrust(MULU/TU2)0.30.20.10EccentricityTrajectory found by QHDDP
  17. 17. Results: Complete Set• Comparison of all test cases• Metric: 2nd order STM well approximated for most cases• Run time: show that the baseline case is mostly faster17 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MNTimings for all test cases Metric for all text cases
  18. 18. Conclusions• Possibility of restarting the estimate with the real STM inorder to improve confidenceEtienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN18• Propagation becomes5.4 to 30 times faster• Total computationtime becomes 2.8 to17 times faster
  19. 19. Future Work• Testing on representative space trajectories• Use of multi-step quasi-Newton methods• Other updates• Integration of numerical differencing or complex stepdifferentiation• Parallelization of the propagation19 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  20. 20. Thank you for your attention20 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  21. 21. Backup Slides21 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  22. 22. Set of test problems22 Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  23. 23. • Small perturbation to the state:(1)• Taylor series:(2)• Replace 𝛿𝑋 in (1):(3)• Equate (2) and (3):23Derivation of the STMsEtienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  24. 24. • Taylor series:• Quasi-Newton equation:• Rank-1 update:• Because 𝑎𝑢 𝑇Δ𝑌𝑝 is a scalar:• Finally:24Derivation of the SR1 updateEtienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  25. 25. • 𝐽 𝑋,𝑘𝑖and 𝐽 𝑋𝑋,𝑘𝑖are function of the downstream control law(𝑢 𝑞, 𝑘 + 1 ≤ 𝑞 ≤ 𝑁)• They are only accurate for a trajectory that follows exactlythis control law• In HDDP, the next iteration changes the downstreamcontrol law  𝐽 𝑋,𝑘𝑖and 𝐽 𝑋𝑋,𝑘𝑖do not hold information aboutthe new performance index 𝐽𝑖• The quasi-Newton equation does not hold, even with exactsecond-order derivatives• Applying a quasi-Newton method, which enforces thisquasi-Newton equation, can not predict the right 𝐽 𝑋𝑋,𝑘𝑖+125Why not apply quasi-Newton to 𝑱 𝑿𝑿computation?Etienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  26. 26. “An optimal policy has the property thatwhatever the initial state and initial decisionare, the remaining decisions must constitutean optimal policy with regard to the stateresulting from the first decision.”Bellman, R., Dynamic Programming, Princeton University Press,Princeton, New Jersey, 1957.26Bellman’s Principle of OptimalityEtienne Pellegrini – AIAA/AAS Astrodynamics Specialists Conference – 8/13/12 – Minneapolis, MN
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×