Talk charts from 2005 AAS/AIAA Astrodynamics Specialists Conference, held Aug 7-11, 2005, in Lake Tahoe, CA.

1. 1
Discrete Nonlinear Optimal Control
of S/C Formations Near the L1 and L2 Points
of the Sun-Earth/Moon System
B. G. Marchand and K. C. Howell
School of Aeronautics and Astronautics
Purdue University
J. T. Betts
Technical Fellow
Mathematics and Engineering Analysis
The Boeing Company

2. 2
Formations Near the Libration Points
Moon
1L
2L
ˆx
( )ˆ inertialX
B
θ
ˆy
Chief S/C Path
(Lissajous Orbit)
EPHEM = Sun + Earth + Moon Motion From Ephemeris w/ SRP
CR3BP = Sun + Earth/Moon barycenter Motion Assumed Circular w/o SRP
Deputy S/C
(Orbit Chief Vehicle)

3. 3
Formation Keeping via
Nonlinear Optimal Control
• Incorporate nonlinearities into control design process
• Allows for the addition of control and path constraints
– Upper and lower bounds on thrust output
– Specifications on relative path error
– Allow for thruster on-off times while min. the impact on the path
• Min. # of assumptions  better assessment of feasibility

4. 4
Optimal Control Solution
• Method #1: Partial Discretization
– Divide Trajectory into Segments and Nodes
– Numerically integrate node states
– Impulsive Control at Nodes (or Constant Thrust Between Nodes)
– Numerically integrated gradients
– Solve Using Subspace Trust Region Method
• Method #2: Transcription and Nonlinear Programming
– Divide Trajectory Into Segments and Nodes
– Solve using Sparse Optimal Control Software (SOCS)
• Use Hermite-Simpson discretization (others available)
• Jacobian and Hessian computed via Sparse Finite Differencing.
• Estimate cost index to second order
• Use SQP algorithm

5. 5
Identification of Startup Solution
• Possible Startup Solution Options
– Non-Natural Arcs  IFL/OFL Nonlinear Control
• Specify some nominal motion
• Apply IFL/OFL control to achieve desired nominal
• Use results as initial guess to optimal control process with
– Natural Arcs  Floquet Analysis of Chief S/C Linearized Equations
• Deputy dynamics modeled as a perturbation relative to chief path
• Floquet controller applied to establish natural relative formation
• Transition into NL system via 2-level corrector
• Use results as initial guess to optimal control process with
( )u t 0≠
( )u t 0=

6. 6
Method #1: Optimal Control
by Partial Discretization
( ) ( ) ( ) ( )
11 1
0 0
min , , , ,φ φ
+− −
= =
=+ =+∑ ∑ ∫ 
j
j
tN N
N j j j N
j j t
J x L t x u x L t x u dt
Subject to:
( ) ( )1 0, , ; Subject to 0+= = =j j j j ix F t x u x x x
( ) ( ) ( )
( )
( )
( ) ( )
( )
1
1
1
1 1 1
0
0
min
, ,
, , ;
, ,
Subject to
0
φ φ+
+
+
+ + +
= + =
  
 = = = 
 +    
 
=  
 
 
 

N n N N
j j jj
j j j j
n j n j j j j
J x x t x
F t x ux
x F t x u
x t x t L t x u
x
x
Equivalent Representation as Augmented Nonlinear System:

7. 7
Euler-Lagrange Optimality Conditions
(Based on Calculus of Variations)
( )
1
1
Condition #1: 1
Condition #2: 0 ; 0, , 1
φ
λ λ λ
λ
+
+
∂ ∂ 
= = → =  
∂ ∂ 
∂
= = =
∂
∂∂
∂
−
∂  

 



j NT T T
j j N
j
j
j
j N
j T
j
j j
H x
x x
j
F
u
H
N
u
F
x
( )1 , ,λ +=  T
j j j j jH F t x u
Identify and from augmented linear system.
∂ ∂
∂ ∂
 

j j
j j
F F
x u

8. 8
Identification of Gradients
From the Augmented Linearized Model
( ) ( ) ( ) ( ) ( )x t A t x t B t u tδ δ δ= +   
( )
( ) 0
0
 
 = ∂ 
 ∂ 
 
dA t
A t L
x
( )
3
3
0
0T
B t I
 
 =  
  

( )
( )
( )
( )
0
11
0, ,
;
0 0, , nn
x xf t x ux
xL t x ux ++
      
= =      
     


Nonlinear System:
Linear System:

9. 9
Solution to Linearized Equations
( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( )
0
0 0
0 0 0 0 7
, ,
, , ; ,
t
t
x t t t x t B u d
t t A t t t t t I
δ δ τ τ δ τ τ=Φ + Φ
Φ = Φ Φ =
∫  
   
∂
∂

j
F
x
Relation to Gradients in E-L Optimality Conditions:
( ) ( ) ( ) ( )
1
1 1 1, ,
j
j
t
j j j j j
t
x t t x t B u dδ δ τ τ δ τ τ
+
+ + +=Φ + Φ∫  

10. 10
Control Gradient for Impulsive Control
( ) ( ) ( ) ( )
1
1 1 1, ,
j
j
t
j j j j j
t
x t t x t B u dδ δ τ τ δ τ τ
+
− +
+ + +=Φ + Φ∫  
( )( )
( ) ( )
1
1 1
,
, ,
j j j j
j j j j j j
t t x B V
t t x t t B V
δ
δ
−
+
−
+ +
=Φ + ∆
= Φ + Φ ∆
 
 
∂
∂

j
F
u
( )1,+
∂
= Φ
∂


j j
j
F
t t B
u

11. 11
Control Gradient for Constant Thrust Arcs
( ) ( ) ( )
1
1 1 1, ,
j
j
t
j j j j j j
t
x t t x t B d uδ δ τ τ τ δ
+
+ + +
 
 =Φ + Φ
  
∫  
∂
∂

j
F
u
( ) ( ) ( )
1
1
1, ,
j
j
t
j j j
j t
F
t t t B d
u
τ τ τ
+
−
+
 ∂
 =Φ Φ
∂   
∫


( )
( )
( )
( )
( ) ( )
( ) ( )
1
1*
, ,
, ,
,,
, ,
n
jj
j j
f t x ux
L t x ux
A t t tt t
t t t t B τ
+
−
  
  
  
=    ΦΦ   
  Φ Φ    


  
 
( )
( )
Only , available from numerical integration
Use STM properties to rewrite in terms of , .
τ
τ
Φ
∂
Φ
∂

j
j
t
F
t
u

12. 12
Numerical Solution Process
( )
0(1) ; ( 0,1,..., 1)
(2)1-Scalar Equation to Optimize in 3 1 Control Variables
Use optimizer to identify optimal
Input , , and ini
given .
During each it
tial guess for
e ati nr o
= −
−
∂
∂
 N i
i
i
i
i N
N
H
u
u
x t u
( )
( ) Sequence (by numerical integration) forward and stor
of the optimizer, the following steps are followed:
( ) Evaluate cost functional,
( ) Evaluat
e; 1,
e
, 1
φ
φ
λ
= … −
=
∂
=
 
 
N
N
i
T
a x i
b J x
c
N
x( )
( )
( ) Sequence backward and compute the search direction ; 1, ,1
and used in next update of control input. (Subspace Trust Region Method)
1
φ
λ
 ∂
=  
∂ ∂
= −
∂
∂
∂

∂

 i
i
i
i
i
N N
N N
H
d i N
u
H
e J
u
x x

13. 13
State Corrector vs. Nonlinear Optimal Control:
Magnitude of Radial Error
( ) ˆ5000 km Zρ =
( ) ( )
11
0
1
min
2
+−
=
= − −∑ ∫
 
j
j
t
N
T
i t
J x x Q x x d t

14. 14
State Corrector vs. Nonlinear Optimal Control:
Cost Function

15. 15
State Corrector vs. Nonlinear Optimal Control:
Impulsive Maneuver Differences

16. 16
Method #2: Nonlinear Programming
• General Nonlinear Programming (NLP) Problem
– Sequential Programming Solution  Algebraic System
• Approximate Lagrangian to 2nd Order
• Approximate constraints as linear
• Iterative solution via globalized Newton methods
( )( ) ( )= ≤ ≤ ≤ ≤min ; andL U L UJ F x c c x c x x x
( ) ( ) ( )
( ) ( ) ( ) ( )
λ
∂
∂
 ∂
= −
≈ − + − −
∂ 
  ∂ ∂  
*
* *
*
*1
2
T
T
L x F x c x
L x x x x x
L
x x
x
x
x
L
Jacobian
Matrix
Hessian
MatrixSearch
Direction

17. 17
Dynamic Optimization
via Nonlinear Programming
• Divide trajectory into phases (segments)
• Define objective function
• For each phase, define
– Dynamic variables
– State equation
– Nonlinear constraints
– State Vector Limits
– Control Vector Limits
– Phase boundary conditions
• Approximate State Equations by Direct Transcription
• Use SOCS SQP algorithm to solve
( ) ( ) ( )
= [ , ]
k k k
z y u
( ) ( ) ( )
( ) ( )
( ) ( )
= [ , , , ]
k k k k k
y f y t u t p t
( ) ( ) ( )
( ) ( )
( ) ( ) ( )
≤ ≤[ , , , ]
k k k k k k
l ug g y t u t p t g
( ) ( )
( ) ( )
≤ ≤
k k k
l uy y t y
( ) ( )
( ) ( )
≤ ≤
k k k
l uu u t u
Ψ ≤ Ψ ≤ Ψl u

18. 18
Direct Transcription
Example: Hermite-Simpson Discretization
( ) ( )
( )
( ) ( )
ς
ς
τ
+ + +
+ + +
+ + +
′= − − + +
 
′ ′ ′= + 
 
′= + + −
= ∆
1 1 1
1 1 1
1 1 1
Given an initial guess for , at each node, the defect ( ) at node:
4
6
where
, ,
2
1
2 8
k k th
k
k
k k k k k k
k
k k k k
k
k k k k k
k k
y u k
h
y y f f f
h
f f y u t
h
y y y f f
h t
• Treat the defects as a constraint, ςk = 0, imposed on the cost function!
• The partials of the defect equations lead to large sparse matrices.
• Use SOCS (Sparse Optimal Control Software) to ID solution.
+
1y
−
2y
+
2y
−
3y
1t 2t 3t
( )y t
ςk
+
3y

19. 19
Sample Startup Solution:
Slowly Drifting Vertical Orbit
Origin = Chief S/C
( )0r ( )fr t
100 Revolutions = 18,000 days
y
z

20. 20
Example 2: Continuous Optimal Control
Goal  Periodicity
( ) ( )( )
( )
1
k
f
k
i
N t
T
t
k
J u t u t dt
=
= ∑∫
( )
fixed, for 1, ,4= = …k
it k
( )
fixed, for 1, ,4= = …
k
ft k
( )
( )1
0= −i ir t r
( )
( )1
0 = iu t
( )
( ) ( )
( )1
0 , 1, ,3
+
= − =k k
f iz t z t k
( )
( ) ( )
( )1 4
0 i fy t y t= −
( )
10,000 km 10,000 km
10,000 km 10,000 km
10,000 km 10,000 km
4 m/sec 4 m/sec
4 m/sec 4 m/sec
4 m/sec 4 m/sec
−   
   −   
   −
≤ ≤   
−   
   −
   
−      
y t
( )
2 2
2 2
2 2
4 m/s 4 m/s
4 m/s 4 m/s
4 m/s 4 m/s
   −
   
− ≤ ≤   
   −   
u t
Constraints: Search Space:

21. 21
Numerical Issues
• Relative Scaling Problems
– Non-convergence
– All constraints met except for control acceleration continuity
• Source
– Small control accelerations trick the software into convergence
• Solution
– Chief S/C path pre-determined and stored using B-splines
– Internal rescaling of variables
– Use dimensional form of relative equations of motion

22. 22
EX2: Periodicity
Via Continuous Control
Startup Solution
Converged Solution

23. 23
Conclusions
• Direct Transcription Method
– With proper variable scaling, responds well to
dynamical sensitivity of n-body problem.
– Accuracy issues overcome through mesh refinement.
– Availability of higher order representations may be
useful in reducing mesh refinement iterations. These
methods not currently present in SOCS.
• Partial Discretization Method
– Similar optimization scheme in some respects
– No constraints presently included in the formulation
– Solution speed hindered by sequencing
– Accuracy controlled by integrator selection

24. 24
Backups

25. 25
Distributed S/C Systems
• Generic formulation  Application Independent
– Formation Flight
• Vehicles share and exchange information to accomplish mission
– Central vehicle  Chief S/C
– Other vehicles  Deputies
– Examples
» Interferometry
» Surface Imaging
» Radar
» Geolocation
– Vehicle Rendezvous & Docking
• Resources and information may be transferred (application dependent)
– Central “chief” vehicle  not necessarily aware of the presence or activities of other
spacecraft or “deputies”.
– Deputy vehicles  perform operations on or in the vicinity of the chief
– Examples:
» Resource transfer (fuel, equipment, etc.) between vehicles
» Unmanned on-orbit servicing of satellites
» Space based threat assessment and handling

26. 26
Relative Dynamics
(Frame Independent Formulation)
( )
( ) ( )
=
= +


c c
d d
y f y
y f y Bu t
( ) ( ) ( )
( ) ( ) ( )
( ), ,
= − +
= + − +
=



d c
c c
c
x f y f y u t
x f y x f y u t
x F x u y
( ) ( ) ( )
( ) ( ) ( ) ( )
δ δ
δ δ δ
=
= +


c c c
d d d
y t A t y t
y t A t y t B u t
( ) ( ) ( )
( ) ( ) ( )
δ δ δ
δ δ
∂ ∂
= +
∂ ∂
= +

d
F F
x t x t u t
x u
A t x t B u t
Absolute Dynamical Model Relative Dynamical Model
Linear System
Define the mathematical model that preserves generality for all apps/systems.
Linear System
Nonlinear System Nonlinear System

27. 27
Impulsive Radial Optimal Control
( )
11
2
0
1
min
2
+−
=
= −∑ ∫

j
j
t
N
j t
J q r r dt

28. 28
Radial Optimal Control:
Cost Functional

29. 29
Radial Optimal Control:
Maneuver History

30. 30
( ) ( ) ( )
( )
> > >
′ =
 * *
1 2 1 2
Direct
Find a sequence of points , , , such that .
This only requires a comparison of the objective function at each point.
Indirect
Identify the root of the necessary condition 0. T
z z z F z F z F z
F z
→
his requires that the user
compute the derivative of the cost index and determine if it meets the specified
tolerance.
From an optimal control perspective:
Indirect Optimization Identify the roots of th
→
e KKT Conditions (Euler-Lagrange)
Direct Optimization Does NOT require the explicit derivation and construction
of the necessary conditions (i.e. the adjoint equations, the control equations,
or the transversality conditions) that are required by the Euler-Lagrange equations.
Dynamic Optimization Approach

31. 31
Direct vs. Indirect
• If the optimality conditions are already determined by the EL-
equations, why not use an indirect method? For the formation
keeping problem, this approach has been investigated but is not
recommended.
– The partial derivative matrices, in this case, involve a matrix quadrature
of a function of the STM. This is computationally intensive of course.
– Requires a good estimate of the constrained trajectory arc to start the
optimization process.
– In general, the numerical process is extremely sensitive (ill-conditioned)
to the initial guess for the Lagrange multipliers. This problem is even
more difficult to deal with in the n-body problem.
• Since, in the n-body problem, an exact solution is not available for
the KKT equations, a direct method is better suited for nonlinear
optimization in this case.

32. 32
Direct Methods: Nonlinear Programming
• Parameter Optimization
– Finite dimensional
– Solution  Globalized Newton Methods
• Functional Optimization  Optimal Control
– Infinite dimensional
– Solution  Transcribe into finite dimensional problem
• Represent dynamical system in terms of finite set of variables
• Solve the finite dimensional problem using NLP
– Reduces problem to solving an algebraic system of eqns.
• Assess accuracy of finite dimensional approximation
• If needed, refine grid and repeat first two steps

33. 33
Optimal Control Preliminaries
• Observations
– The cost index depends on point functions and quadrature functions
– Nonlinear point functions can include variables from all phases
– Quadrature functions are evaluated along the length of the phase by
augmenting the state vector:
– The boundary conditions also depend on variables from all phases
– Each phase is divided into N mesh points for the discretization
– Each interior grid point is assigned a control variable,
( )
( ) ( )
( ) ( )
( )
( ) ( )
( ) ( )
( )
( ) ( )
( ) ( )
  
    =
    
   
, , ,
, , ,
, , ,
k k k
k k k
k k k
f y t u t p t
F y t u t p t
w y t u t p t
( )
( ) ( )
=
k k
j ju t u

34. 34
Define Optimal Control Problem
( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )
( ) ( )
( ) ( ) ( )
( )
=
 = Φ
 
 +
 ∑ ∫

( )
1 1 1 1 1
1
min , , , , , , , , , ,
, ,
j
F
j
I
N N N N N
i i f f i i f f
tN
j j j j
j t
J z t z t p z t z t p
w z t t p dt
( )
( ) ( )
( ) ( )
( ) ( ) ( )
( ) ( ) ( )
( ) ( )
( ) ( )
( ) ( )
( ) ( )
( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )
≤ ≤
≤ ≤
≤ ≤
 Ψ ≤ Ψ ≤ Ψ
 


1 1 1 1 1
Each phase is subject to:
, , ;
, ,
The phases are linked by boundary conditions of the form:
, , , , , , , , , ,
k k k k k
i f
k k k k k
l u
k k k
l u
N N N N N
l i i f f i i f f u
y t f z t p t t t t
g g z t p t g
z z t z
z t z t p z t z t p
( )
( ) ( )
( ) ( )
( )
( )
( ) ( )
( )
 =
 
For each phase, , define a vector of dynamic variables, ,
that includes both the state vector, , and the control input vector, .
k k k
k k
k z t y t u t
y t u t

35. 35
Optimizer Test
• Identification of a good startup solution for the optimizer
is necessary to ensure convergence.
– Determine non-periodic but bounded relative orbits in the
linearized system using the Floquet Controller.
– Employ a 2-level differential corrections process to converge the
solution in the nonlinear system.
– Transfer this solution as an initial guess to the nonlinear optimal
control process.
– Choose mathematical model that is consistent with ephemeris
formulation for later transition into the Generator FORMATION
tool.
– Impose closed-path constraint as a test case.

36. 36
Impulsive Optimal Control
Minimize State Error with End-State Weighting
( ) ( ) ( ) ( )
11
0
1 1
min
2 2
+−
=
= − − + − −∑ ∫
   
j
j
t
NT T
N N N N
i t
J x x W x x x x Q x x d t
( ) ˆ500 mρ = Y

37. 37
EX1: Impulsive Optimal Control:
Closed Relative Path (Small)
Chief S/C Halo Orbit
Az = 300,000 km
Deputy S/C Relative Orbit
rmax = 82 km

38. 38
EX3: Sensitivity of Solution to Initial Guess
0
0
8254 km
Goal: Determine min( V) for f
r
r r
=
∆ =
Given a bad initial guess for the optimizer …

39. 39
Chief S/C Halo Orbit
Az = 300,000 km
Deputy S/C Relative Orbit
rmax = 8254 km
Converged Periodic Solution
(Max. Amplitude 8254 km)
… the numerical process is still able to identify the desired solution

40. 40
Example #1: Impulsive Optimal Control
to Achieve Closed Path
0 0Cost Index: min =∆ ∆T
J V V
( ) ( )
( )
( ) ( )
( )
0
0
0
0 0 0
1
Dynamical Constraint:
, ; 0
Terminal Path Constraint:
0
Initial Velocity Constraint:
0
Continuity Constraints:
0
−
− +
+
   
= = =   
   
− =
+ ∆ − ∆ =
− =


c
f
k k
f i
r r
y f y y x
VV
r r
V V V
y t y t

41. 41
Chief S/C Halo Orbit
Az = 300,000 km
Deputy S/C Relative Orbit
rmax = 81057.8 km
EX1: Impulsive Optimal Control:
Closed Relative Path (Large)
2 m/sec
ΔV

42. 42
Deputy S/C Path
Chief S/C @ Origin

43. 43
80, 81
160, 161
240, 241
320
1
Discontinuities in Control Acceleration

44. 44
EX2: Control Acceleration Profile