SlideShare a Scribd company logo
1 of 280
Download to read offline
Morphing UAV Intelligent Shape and Flight Control - I
Kumar Digvijay Mishra
Control of Morphing Aerostructures
kumardigvijaymishra@yahoo.co.uk
January 11, 2021
Kumar Digvijay Mishra January 11, 2021 1 / 69
Overview
1 Introduction
2 A-RLC Architecture Functionality
3 Learning Air Vehicle Shape Changes
Overview of RL
Implementation of Shape Change Learning Agent
4 Mathematical Modeling of Morphing Air Vehicle
Aerodynamic Modeling
Constitutive Equations
Model Grid
Dynamical Modeling
Reference Trajectory
Shape Memory Alloy Actuator Dynamics
Control Effectors on Morphing Wing
5 Morphing Control Law
6 Numerical Examples
7 Conclusion
Kumar Digvijay Mishra January 11, 2021 2 / 69
Introduction
Definition
Morphing refers to both small-scale and large-scale shape changes or
transconfigurations
Two classes of morphing in context of flight vehicles:
1 Morphing for Control
A small-scale or component level, in-flight physical or virtual
shape change, used to achieve multiple control objectives such as
noise suppression, flutter suppression, load alleviation and active flow
separation control
Kumar Digvijay Mishra 1. Introduction January 11, 2021 3 / 69
Introduction
Definition
Morphing refers to both small-scale and large-scale shape changes or
transconfigurations
Two classes of morphing in context of flight vehicles:
1 Morphing for Control
A small-scale or component level, in-flight physical or virtual
shape change, used to achieve multiple control objectives such as
noise suppression, flutter suppression, load alleviation and active flow
separation control
2 Morphing for Mission Adaptation (MMA)
A large-scale, relatively slow, in-flight shape change to enable a
single vehicle to perform multiple diverse mission profiles
Kumar Digvijay Mishra 1. Introduction January 11, 2021 3 / 69
Introduction
Essential functionalities of practical MMA capability are:
1 When to reconfigure: Driven by mission priorities/tasks, and leads to
optimal shape being a system parameter
Kumar Digvijay Mishra 1. Introduction January 11, 2021 4 / 69
Introduction
Essential functionalities of practical MMA capability are:
1 When to reconfigure: Driven by mission priorities/tasks, and leads to
optimal shape being a system parameter
2 How to reconfigure: Problem of sensing, actuation, and control
Kumar Digvijay Mishra 1. Introduction January 11, 2021 4 / 69
Introduction
Essential functionalities of practical MMA capability are:
1 When to reconfigure: Driven by mission priorities/tasks, and leads to
optimal shape being a system parameter
2 How to reconfigure: Problem of sensing, actuation, and control
3 Learning to reconfigure: Actuation scheme(s) to produce most
optimal shape
Kumar Digvijay Mishra 1. Introduction January 11, 2021 4 / 69
Introduction
Previous control design research1:
◦ Vortex lattice method used to calculate aerodynamics for the
case of variable wing dihedral2
◦ Desired dynamics chosen for each mission phase
◦ Developed H∞ model-following controllers
1
Boothe et al. (2005) and Abdulrahim and Lind(2005)
2
inclination of aircraft’s wing from horizontal, esp upwards, away from fuselage
Kumar Digvijay Mishra 1. Introduction January 11, 2021 5 / 69
Introduction
Previous control design research1:
◦ Vortex lattice method used to calculate aerodynamics for the
case of variable wing dihedral2
◦ Desired dynamics chosen for each mission phase
◦ Developed H∞ model-following controllers
Current control design approach:
◦ High-level description of functionality of A-RLC architecture
◦ Development of learning agent, air vehicle model and simulation,
smart actuators, and adaptive controller
1
Boothe et al. (2005) and Abdulrahim and Lind(2005)
2
inclination of aircraft’s wing from horizontal, esp upwards, away from fuselage
Kumar Digvijay Mishra 1. Introduction January 11, 2021 5 / 69
A-RLC Architecture Functionality
Adaptive-Reinforcement Learning Control composed of:
1 Reinforcement Learning (RL)
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 6 / 69
A-RLC Architecture Functionality
Adaptive-Reinforcement Learning Control composed of:
1 Reinforcement Learning (RL)
2 Structured Adaptive Model Inversion (SAMI)
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 6 / 69
A-RLC Architecture Functionality
Adaptive-Reinforcement Learning Control composed of:
1 Reinforcement Learning (RL)
2 Structured Adaptive Model Inversion (SAMI)
Both sub-systems interact significantly during:
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 6 / 69
A-RLC Architecture Functionality
Adaptive-Reinforcement Learning Control composed of:
1 Reinforcement Learning (RL)
2 Structured Adaptive Model Inversion (SAMI)
Both sub-systems interact significantly during:
1 Episodic learning stage: when optimal shape change policy is learned
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 6 / 69
A-RLC Architecture Functionality
Adaptive-Reinforcement Learning Control composed of:
1 Reinforcement Learning (RL)
2 Structured Adaptive Model Inversion (SAMI)
Both sub-systems interact significantly during:
1 Episodic learning stage: when optimal shape change policy is learned
2 Operational stage: when the plant morphs, and tracks a trajectory
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 6 / 69
A-RLC Architecture Functionality
Adaptive-Reinforcement Learning Control composed of:
1 Reinforcement Learning (RL)
2 Structured Adaptive Model Inversion (SAMI)
Both sub-systems interact significantly during:
1 Episodic learning stage: when optimal shape change policy is learned
2 Operational stage: when the plant morphs, and tracks a trajectory
A-RLC addresses optimal shape changing of air vehicle, rather than
isolated components or individual actuation.
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 6 / 69
A-RLC Architecture Functionality
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 7 / 69
RL Module of A-RLC Architecture
1 Initially commands an arbitrary action from the set of admissible
actions
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 8 / 69
RL Module of A-RLC Architecture
1 Initially commands an arbitrary action from the set of admissible
actions
2 This action is sent to the plant, which produces a shape change
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 8 / 69
RL Module of A-RLC Architecture
1 Initially commands an arbitrary action from the set of admissible
actions
2 This action is sent to the plant, which produces a shape change
3 Cost associated with resultant shape change in terms of system
states, parameters, and user-defined performance measures is
evaluated with the cost function and then passed to the agent
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 8 / 69
RL Module of A-RLC Architecture
1 Initially commands an arbitrary action from the set of admissible
actions
2 This action is sent to the plant, which produces a shape change
3 Cost associated with resultant shape change in terms of system
states, parameters, and user-defined performance measures is
evaluated with the cost function and then passed to the agent
4 Agent modifies its action-value function based on Q-learning
algorithm
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 8 / 69
RL Module of A-RLC Architecture
1 Initially commands an arbitrary action from the set of admissible
actions
2 This action is sent to the plant, which produces a shape change
3 Cost associated with resultant shape change in terms of system
states, parameters, and user-defined performance measures is
evaluated with the cost function and then passed to the agent
4 Agent modifies its action-value function based on Q-learning
algorithm
5 For the next episode, the agent chooses a new action based on the
current policy and its updated action-value function, and the
sequence repeats itself
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 8 / 69
RL Module of A-RLC Architecture
1 Initially commands an arbitrary action from the set of admissible
actions
2 This action is sent to the plant, which produces a shape change
3 Cost associated with resultant shape change in terms of system
states, parameters, and user-defined performance measures is
evaluated with the cost function and then passed to the agent
4 Agent modifies its action-value function based on Q-learning
algorithm
5 For the next episode, the agent chooses a new action based on the
current policy and its updated action-value function, and the
sequence repeats itself
6 RL sub-system improved by applying Sequential Function
Approximation to generalize learning from previously experienced
quantized states and actions to the continuous state-action space.
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 8 / 69
SAMI Module of A-RLC Architecture
1 Shape changes in the plant due to actions generated by RL agent
cause plant dynamics to change
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 9 / 69
SAMI Module of A-RLC Architecture
1 Shape changes in the plant due to actions generated by RL agent
cause plant dynamics to change
2 By calculating commanded moments, SAMI controller tracks a
reference trajectory irrespective of the changing dynamics of the
vehicle caused by these shape changes
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 9 / 69
SAMI Module of A-RLC Architecture
1 Shape changes in the plant due to actions generated by RL agent
cause plant dynamics to change
2 By calculating commanded moments, SAMI controller tracks a
reference trajectory irrespective of the changing dynamics of the
vehicle caused by these shape changes
3 SAMI is a non-linear technique based on concepts of Feedback
Linearization, Dynamic Inversion, and Structured Model Reference
Adaptive Control
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 9 / 69
SAMI Module of A-RLC Architecture
1 Shape changes in the plant due to actions generated by RL agent
cause plant dynamics to change
2 By calculating commanded moments, SAMI controller tracks a
reference trajectory irrespective of the changing dynamics of the
vehicle caused by these shape changes
3 SAMI is a non-linear technique based on concepts of Feedback
Linearization, Dynamic Inversion, and Structured Model Reference
Adaptive Control
4 Dynamic Inversion is approx since system parameters are inaccurately
modeled
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 9 / 69
SAMI Module of A-RLC Architecture
1 Shape changes in the plant due to actions generated by RL agent
cause plant dynamics to change
2 By calculating commanded moments, SAMI controller tracks a
reference trajectory irrespective of the changing dynamics of the
vehicle caused by these shape changes
3 SAMI is a non-linear technique based on concepts of Feedback
Linearization, Dynamic Inversion, and Structured Model Reference
Adaptive Control
4 Dynamic Inversion is approx since system parameters are inaccurately
modeled
5 Adaptive control structure is wrapped around the dynamic inverter to
account for uncertainties in system parameters
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 9 / 69
SAMI Module of A-RLC Architecture
1 Shape changes in the plant due to actions generated by RL agent
cause plant dynamics to change
2 By calculating commanded moments, SAMI controller tracks a
reference trajectory irrespective of the changing dynamics of the
vehicle caused by these shape changes
3 SAMI is a non-linear technique based on concepts of Feedback
Linearization, Dynamic Inversion, and Structured Model Reference
Adaptive Control
4 Dynamic Inversion is approx since system parameters are inaccurately
modeled
5 Adaptive control structure is wrapped around the dynamic inverter to
account for uncertainties in system parameters
6 Applications: tracking reference trajectories of spacecraft, planetary
entry vehicles, and aircraft; handling of actuator failures; facilitation
of correct adaptation in presence of actuator saturation
Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 9 / 69
3.1 Overview of Reinforcement Learning
Definition
RL is a method of learning from interaction between an agent and its
environment to achieve a specified goal.
Kumar Digvijay Mishra 3. Learning Air Vehicle Shape Changes January 11, 2021 10 / 69
3.1 Overview of Reinforcement Learning
Definition
RL is a method of learning from interaction between an agent and its
environment to achieve a specified goal.
◦ Agent is the learner and decision-maker with the goal of maximizing
total reward it receives over the long run.
Kumar Digvijay Mishra 3. Learning Air Vehicle Shape Changes January 11, 2021 10 / 69
3.1 Overview of Reinforcement Learning
Definition
RL is a method of learning from interaction between an agent and its
environment to achieve a specified goal.
◦ Agent is the learner and decision-maker with the goal of maximizing
total reward it receives over the long run.
◦ Environment is everything outside the agent, that it interacts with.
Kumar Digvijay Mishra 3. Learning Air Vehicle Shape Changes January 11, 2021 10 / 69
3.1 Overview of Reinforcement Learning
Definition
RL is a method of learning from interaction between an agent and its
environment to achieve a specified goal.
◦ Agent is the learner and decision-maker with the goal of maximizing
total reward it receives over the long run.
◦ Environment is everything outside the agent, that it interacts with.
◦ Agent’s Policy is the mapping from states to probabilities of
selecting a specified possible action at a time step
Kumar Digvijay Mishra 3. Learning Air Vehicle Shape Changes January 11, 2021 10 / 69
Overview of Reinforcement Learning
◦ Agent interacts with its environment at discrete time steps, t = 0,
1, 2, 3....
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 11 / 69
Overview of Reinforcement Learning
◦ Agent interacts with its environment at discrete time steps, t = 0,
1, 2, 3....
◦ At each time step t, agent receives some representation of
environment’s state, st ∈ S, where S is a set of possible states, and
on that basis it selects an action, at ∈ A(st), where A(st) is a set of
actions available in state st.
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 11 / 69
Overview of Reinforcement Learning
◦ Agent interacts with its environment at discrete time steps, t = 0,
1, 2, 3....
◦ At each time step t, agent receives some representation of
environment’s state, st ∈ S, where S is a set of possible states, and
on that basis it selects an action, at ∈ A(st), where A(st) is a set of
actions available in state st.
◦ One time step later, as a result of agent’s action, it receives a
numerical reward, rt+1 = R, and finds itself in a new state, st+1.
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 11 / 69
Overview of Reinforcement Learning
◦ Agent interacts with its environment at discrete time steps, t = 0,
1, 2, 3....
◦ At each time step t, agent receives some representation of
environment’s state, st ∈ S, where S is a set of possible states, and
on that basis it selects an action, at ∈ A(st), where A(st) is a set of
actions available in state st.
◦ One time step later, as a result of agent’s action, it receives a
numerical reward, rt+1 = R, and finds itself in a new state, st+1.
◦ Here agent’s policy πt(s, a) indicates the probability that agent
chose action at = a for the environment in given state of st = s at
time t.
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 11 / 69
Overview of Reinforcement Learning
RL algos are based on estimating value functions for a policy π:
State-value function V π(s) estimates how good it is, under policy π,
for the agent to be in state s.
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 12 / 69
Overview of Reinforcement Learning
RL algos are based on estimating value functions for a policy π:
State-value function V π(s) estimates how good it is, under policy π,
for the agent to be in state s.
◦ Defined as the expected return starting from s and thereafter
following π
V π
(st) ≡ E[
∞
X
k=0
γk
rt+k] (1)
where γ is discount factor, rt+k is sequence of rewards
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 12 / 69
Overview of Reinforcement Learning
RL algos are based on estimating value functions for a policy π:
State-value function V π(s) estimates how good it is, under policy π,
for the agent to be in state s.
◦ Defined as the expected return starting from s and thereafter
following π
V π
(st) ≡ E[
∞
X
k=0
γk
rt+k] (1)
where γ is discount factor, rt+k is sequence of rewards
Action-value function Qπ(s, a) estimates how good it is, under policy
π, for the agent to perform action a in state s.
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 12 / 69
Overview of Reinforcement Learning
RL algos are based on estimating value functions for a policy π:
State-value function V π(s) estimates how good it is, under policy π,
for the agent to be in state s.
◦ Defined as the expected return starting from s and thereafter
following π
V π
(st) ≡ E[
∞
X
k=0
γk
rt+k] (1)
where γ is discount factor, rt+k is sequence of rewards
Action-value function Qπ(s, a) estimates how good it is, under policy
π, for the agent to perform action a in state s.
◦ Defined as the expected return starting from s, taking action a,
and thereafter following policy π
Qπ
(s, a) ≡ r(s, a) + γV ∗
(δ(s, a)) (2)
where δ(s, a) denotes the state resulting from applying action a
to state s & V ∗(δ(s, a)) is state-value function for optimal policy π∗
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 12 / 69
Overview of Reinforcement Learning
◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a)
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
Overview of Reinforcement Learning
◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a)
◦ Policy Improvement: For a state s, an action can be chosen from all
possible actions to improve π to π‘ s.t. V π(s) or Qπ(s, a) are
maximized
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
Overview of Reinforcement Learning
◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a)
◦ Policy Improvement: For a state s, an action can be chosen from all
possible actions to improve π to π‘ s.t. V π(s) or Qπ(s, a) are
maximized
Evaluate new V π‘
(s) or Qπ‘
(s, a) for improved policy π‘
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
Overview of Reinforcement Learning
◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a)
◦ Policy Improvement: For a state s, an action can be chosen from all
possible actions to improve π to π‘ s.t. V π(s) or Qπ(s, a) are
maximized
Evaluate new V π‘
(s) or Qπ‘
(s, a) for improved policy π‘
Policy π‘ can be further improved to π“
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
Overview of Reinforcement Learning
◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a)
◦ Policy Improvement: For a state s, an action can be chosen from all
possible actions to improve π to π‘ s.t. V π(s) or Qπ(s, a) are
maximized
Evaluate new V π‘
(s) or Qπ‘
(s, a) for improved policy π‘
Policy π‘ can be further improved to π“
◦ Optimal Policy: A policy π∗ that has :
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
Overview of Reinforcement Learning
◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a)
◦ Policy Improvement: For a state s, an action can be chosen from all
possible actions to improve π to π‘ s.t. V π(s) or Qπ(s, a) are
maximized
Evaluate new V π‘
(s) or Qπ‘
(s, a) for improved policy π‘
Policy π‘ can be further improved to π“
◦ Optimal Policy: A policy π∗ that has :
Either an optimal state-value function V ∗
(s) = maxπV π
(s)
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
Overview of Reinforcement Learning
◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a)
◦ Policy Improvement: For a state s, an action can be chosen from all
possible actions to improve π to π‘ s.t. V π(s) or Qπ(s, a) are
maximized
Evaluate new V π‘
(s) or Qπ‘
(s, a) for improved policy π‘
Policy π‘ can be further improved to π“
◦ Optimal Policy: A policy π∗ that has :
Either an optimal state-value function V ∗
(s) = maxπV π
(s)
Or an optimal action-value function Q∗
(s, a) = maxπQπ
(s, a)
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
Overview of Reinforcement Learning
Goal of RL is to find optimal policy π∗ by policy iteration
◦ Optimal action-value function Q∗ in terms of optimal state-value
function V ∗ :
Q∗
(s, a) ≡ E[r(s, a) + γV ∗
(δ(s, a))]
≡ E[r(s, a)] + E[γV ∗
(δ(s, a))]
≡ E[r(s, a)] + γ
X
s0
P(s
0
|s, a)V ∗
(s
0
)
(3)
where P(s‘|s, a) is probability of taking action a in state s to produce
next state s‘
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 14 / 69
Overview of Reinforcement Learning
Q∗
(s, a) ≡ E[r(s, a)] + γ
X
s0
P(s
0
|s, a)V ∗
(s
0
)
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 15 / 69
Overview of Reinforcement Learning
Q∗
(s, a) ≡ E[r(s, a)] + γ
X
s0
P(s
0
|s, a)V ∗
(s
0
)
Optimal state-value function V ∗(s‘) is action-value function Q(s‘, a‘)
maximized for action a‘
Q∗
(s, a) ≡ E[r(s, a)] + γ
X
s0
P(s
0
|s, a) max
a0
Q(s
0
, a
0
) (4)
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 15 / 69
Overview of Reinforcement Learning
Q∗
(s, a) ≡ E[r(s, a)] + γ
X
s0
P(s
0
|s, a)V ∗
(s
0
)
Optimal state-value function V ∗(s‘) is action-value function Q(s‘, a‘)
maximized for action a‘
Q∗
(s, a) ≡ E[r(s, a)] + γ
X
s0
P(s
0
|s, a) max
a0
Q(s
0
, a
0
) (4)
Iterative update of each action-value function Q(s,a) to converge to
optimal Q∗(s, a) gives following training rule
Qn(s, a) ← (1 − α)Qn−1(s, a) + α[r + γ max
a0
Qn−1(s
0
, a
0
)] (5)
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 15 / 69
Overview of Reinforcement Learning
Q∗
(s, a) ≡ E[r(s, a)] + γ
X
s0
P(s
0
|s, a)V ∗
(s
0
)
Optimal state-value function V ∗(s‘) is action-value function Q(s‘, a‘)
maximized for action a‘
Q∗
(s, a) ≡ E[r(s, a)] + γ
X
s0
P(s
0
|s, a) max
a0
Q(s
0
, a
0
) (4)
Iterative update of each action-value function Q(s,a) to converge to
optimal Q∗(s, a) gives following training rule
Qn(s, a) ← (1 − α)Qn−1(s, a) + α[r + γ max
a0
Qn−1(s
0
, a
0
)] (5)
For Q-learning algorithm,
Q(s, a) ← Q(s, a) + α[r + γ max
a0
Q(s
0
, a
0
) − Q(s, a)] (6)
Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 15 / 69
Policy Iteration
Major methods for PI:
1 Dynamic Programming
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 16 / 69
Policy Iteration
Major methods for PI:
1 Dynamic Programming
2 Monte Carlo Method
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 16 / 69
Policy Iteration
Major methods for PI:
1 Dynamic Programming
2 Monte Carlo Method
3 Temporal-Difference Learning
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 16 / 69
Policy Iteration
Major methods for PI:
1 Dynamic Programming
2 Monte Carlo Method
3 Temporal-Difference Learning
4 Q-Learning Technique
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 16 / 69
Major Methods of Policy Iteration
1. Dynamic Programming
Refers to a collection of algos to compute optimal policies given a
perfect model of the environment as a Markov decision process
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 17 / 69
Major Methods of Policy Iteration
1. Dynamic Programming
Refers to a collection of algos to compute optimal policies given a
perfect model of the environment as a Markov decision process
Use value functions to organize and structure the search for good
policies
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 17 / 69
Major Methods of Policy Iteration
1. Dynamic Programming
Refers to a collection of algos to compute optimal policies given a
perfect model of the environment as a Markov decision process
Use value functions to organize and structure the search for good
policies
Limited utility in RL because of
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 17 / 69
Major Methods of Policy Iteration
1. Dynamic Programming
Refers to a collection of algos to compute optimal policies given a
perfect model of the environment as a Markov decision process
Use value functions to organize and structure the search for good
policies
Limited utility in RL because of
1 Requires model of environment’s dynamics to learn optimal behavior
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 17 / 69
Major Methods of Policy Iteration
1. Dynamic Programming
Refers to a collection of algos to compute optimal policies given a
perfect model of the environment as a Markov decision process
Use value functions to organize and structure the search for good
policies
Limited utility in RL because of
1 Requires model of environment’s dynamics to learn optimal behavior
2 their assumption of a perfect model, and
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 17 / 69
Major Methods of Policy Iteration
1. Dynamic Programming
Refers to a collection of algos to compute optimal policies given a
perfect model of the environment as a Markov decision process
Use value functions to organize and structure the search for good
policies
Limited utility in RL because of
1 Requires model of environment’s dynamics to learn optimal behavior
2 their assumption of a perfect model, and
3 computational expense
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 17 / 69
Major Methods of Policy Iteration
2. Monte Carlo Methods
Estimates functions using an iterative, incremental procedure
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
Major Methods of Policy Iteration
2. Monte Carlo Methods
Estimates functions using an iterative, incremental procedure
Solution based on averaging sample returns
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
Major Methods of Policy Iteration
2. Monte Carlo Methods
Estimates functions using an iterative, incremental procedure
Solution based on averaging sample returns
To ensure availability of well-defined returns, returns are defined only
for episodic tasks
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
Major Methods of Policy Iteration
2. Monte Carlo Methods
Estimates functions using an iterative, incremental procedure
Solution based on averaging sample returns
To ensure availability of well-defined returns, returns are defined only
for episodic tasks
Upon completion of an episode, value estimates and policies are
changed
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
Major Methods of Policy Iteration
2. Monte Carlo Methods
Estimates functions using an iterative, incremental procedure
Solution based on averaging sample returns
To ensure availability of well-defined returns, returns are defined only
for episodic tasks
Upon completion of an episode, value estimates and policies are
changed
Advantages:
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
Major Methods of Policy Iteration
2. Monte Carlo Methods
Estimates functions using an iterative, incremental procedure
Solution based on averaging sample returns
To ensure availability of well-defined returns, returns are defined only
for episodic tasks
Upon completion of an episode, value estimates and policies are
changed
Advantages:
1 Contrary to DP, MC methods can be used to learn optimal behavior
directly from interaction with the environment with no model of
environment’s dynamics
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
Major Methods of Policy Iteration
2. Monte Carlo Methods
Estimates functions using an iterative, incremental procedure
Solution based on averaging sample returns
To ensure availability of well-defined returns, returns are defined only
for episodic tasks
Upon completion of an episode, value estimates and policies are
changed
Advantages:
1 Contrary to DP, MC methods can be used to learn optimal behavior
directly from interaction with the environment with no model of
environment’s dynamics
2 Can be used with simulation
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
Major Methods of Policy Iteration
2. Monte Carlo Methods
Estimates functions using an iterative, incremental procedure
Solution based on averaging sample returns
To ensure availability of well-defined returns, returns are defined only
for episodic tasks
Upon completion of an episode, value estimates and policies are
changed
Advantages:
1 Contrary to DP, MC methods can be used to learn optimal behavior
directly from interaction with the environment with no model of
environment’s dynamics
2 Can be used with simulation
3 Easy and efficient to focus MC methods on a small subset of states
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
Major Methods of Policy Iteration
2. Monte Carlo Methods
Estimates functions using an iterative, incremental procedure
Solution based on averaging sample returns
To ensure availability of well-defined returns, returns are defined only
for episodic tasks
Upon completion of an episode, value estimates and policies are
changed
Advantages:
1 Contrary to DP, MC methods can be used to learn optimal behavior
directly from interaction with the environment with no model of
environment’s dynamics
2 Can be used with simulation
3 Easy and efficient to focus MC methods on a small subset of states
Limitation:
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
Major Methods of Policy Iteration
2. Monte Carlo Methods
Estimates functions using an iterative, incremental procedure
Solution based on averaging sample returns
To ensure availability of well-defined returns, returns are defined only
for episodic tasks
Upon completion of an episode, value estimates and policies are
changed
Advantages:
1 Contrary to DP, MC methods can be used to learn optimal behavior
directly from interaction with the environment with no model of
environment’s dynamics
2 Can be used with simulation
3 Easy and efficient to focus MC methods on a small subset of states
Limitation:
Convergence properties not well-understood for RL
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
Major Methods of Policy Iteration
3. Temporal-Difference Methods
Sutton’s3method of TD is a form of policy evaluation method in DP
which attempts to choose a control policy π0
3
Sutton (1988)
4
Williams and Baird 1993
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
Major Methods of Policy Iteration
3. Temporal-Difference Methods
Sutton’s3method of TD is a form of policy evaluation method in DP
which attempts to choose a control policy π0
Expected discounted rewards, V π(i), for each state i in S are
predicted using policy π0 to improve to new policy π1
3
Sutton (1988)
4
Williams and Baird 1993
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
Major Methods of Policy Iteration
3. Temporal-Difference Methods
Sutton’s3method of TD is a form of policy evaluation method in DP
which attempts to choose a control policy π0
Expected discounted rewards, V π(i), for each state i in S are
predicted using policy π0 to improve to new policy π1
Advantages:
3
Sutton (1988)
4
Williams and Baird 1993
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
Major Methods of Policy Iteration
3. Temporal-Difference Methods
Sutton’s3method of TD is a form of policy evaluation method in DP
which attempts to choose a control policy π0
Expected discounted rewards, V π(i), for each state i in S are
predicted using policy π0 to improve to new policy π1
Advantages:
1 Less computation
3
Sutton (1988)
4
Williams and Baird 1993
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
Major Methods of Policy Iteration
3. Temporal-Difference Methods
Sutton’s3method of TD is a form of policy evaluation method in DP
which attempts to choose a control policy π0
Expected discounted rewards, V π(i), for each state i in S are
predicted using policy π0 to improve to new policy π1
Advantages:
1 Less computation
2 No assumption of perfect model of environment
3
Sutton (1988)
4
Williams and Baird 1993
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
Major Methods of Policy Iteration
3. Temporal-Difference Methods
Sutton’s3method of TD is a form of policy evaluation method in DP
which attempts to choose a control policy π0
Expected discounted rewards, V π(i), for each state i in S are
predicted using policy π0 to improve to new policy π1
Advantages:
1 Less computation
2 No assumption of perfect model of environment
3 Howard’s4
algo converges to some policy
3
Sutton (1988)
4
Williams and Baird 1993
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
Major Methods of Policy Iteration
3. Temporal-Difference Methods
Sutton’s3method of TD is a form of policy evaluation method in DP
which attempts to choose a control policy π0
Expected discounted rewards, V π(i), for each state i in S are
predicted using policy π0 to improve to new policy π1
Advantages:
1 Less computation
2 No assumption of perfect model of environment
3 Howard’s4
algo converges to some policy
Limitations:
3
Sutton (1988)
4
Williams and Baird 1993
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
Major Methods of Policy Iteration
3. Temporal-Difference Methods
Sutton’s3method of TD is a form of policy evaluation method in DP
which attempts to choose a control policy π0
Expected discounted rewards, V π(i), for each state i in S are
predicted using policy π0 to improve to new policy π1
Advantages:
1 Less computation
2 No assumption of perfect model of environment
3 Howard’s4
algo converges to some policy
Limitations:
1 Can’t learn optimal functions directly. Why ?
3
Sutton (1988)
4
Williams and Baird 1993
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
Major Methods of Policy Iteration
3. Temporal-Difference Methods
Sutton’s3method of TD is a form of policy evaluation method in DP
which attempts to choose a control policy π0
Expected discounted rewards, V π(i), for each state i in S are
predicted using policy π0 to improve to new policy π1
Advantages:
1 Less computation
2 No assumption of perfect model of environment
3 Howard’s4
algo converges to some policy
Limitations:
1 Can’t learn optimal functions directly. Why ?
Because it fixes the policy and then determines corresponding value
functions
3
Sutton (1988)
4
Williams and Baird 1993
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
Major Methods of Policy Iteration
4. Q-Learning Technique
A form of successive approximation technique of DP
5
Watkins and Davan 1989
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
Major Methods of Policy Iteration
4. Q-Learning Technique
A form of successive approximation technique of DP
Proposed and developed by Watkins5
5
Watkins and Davan 1989
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
Major Methods of Policy Iteration
4. Q-Learning Technique
A form of successive approximation technique of DP
Proposed and developed by Watkins5
Learns optimal value functions directly, as opposed to fixing a policy
and determining corresponding value functions, like TD
5
Watkins and Davan 1989
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
Major Methods of Policy Iteration
4. Q-Learning Technique
A form of successive approximation technique of DP
Proposed and developed by Watkins5
Learns optimal value functions directly, as opposed to fixing a policy
and determining corresponding value functions, like TD
Advantages:
5
Watkins and Davan 1989
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
Major Methods of Policy Iteration
4. Q-Learning Technique
A form of successive approximation technique of DP
Proposed and developed by Watkins5
Learns optimal value functions directly, as opposed to fixing a policy
and determining corresponding value functions, like TD
Advantages:
1 Learns optimal value functions directly
5
Watkins and Davan 1989
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
Major Methods of Policy Iteration
4. Q-Learning Technique
A form of successive approximation technique of DP
Proposed and developed by Watkins5
Learns optimal value functions directly, as opposed to fixing a policy
and determining corresponding value functions, like TD
Advantages:
1 Learns optimal value functions directly
2 Avoids need to sweep over the state-action space
5
Watkins and Davan 1989
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
Major Methods of Policy Iteration
4. Q-Learning Technique
A form of successive approximation technique of DP
Proposed and developed by Watkins5
Learns optimal value functions directly, as opposed to fixing a policy
and determining corresponding value functions, like TD
Advantages:
1 Learns optimal value functions directly
2 Avoids need to sweep over the state-action space
3 First provably convergent direct adaptive optimal control algo
5
Watkins and Davan 1989
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
Major Methods of Policy Iteration
4. Q-Learning Technique
A form of successive approximation technique of DP
Proposed and developed by Watkins5
Learns optimal value functions directly, as opposed to fixing a policy
and determining corresponding value functions, like TD
Advantages:
1 Learns optimal value functions directly
2 Avoids need to sweep over the state-action space
3 First provably convergent direct adaptive optimal control algo
4 Less computations
5
Watkins and Davan 1989
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
Reinforcement Learning
◦ Has been applied to wide variety of physical control tasks, both real
and simulated
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
Reinforcement Learning
◦ Has been applied to wide variety of physical control tasks, both real
and simulated
◦ When state-space is too large to enumerate value functions,
function approximators are used including,
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
Reinforcement Learning
◦ Has been applied to wide variety of physical control tasks, both real
and simulated
◦ When state-space is too large to enumerate value functions,
function approximators are used including,
1 Neural networks
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
Reinforcement Learning
◦ Has been applied to wide variety of physical control tasks, both real
and simulated
◦ When state-space is too large to enumerate value functions,
function approximators are used including,
1 Neural networks
2 Clustering
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
Reinforcement Learning
◦ Has been applied to wide variety of physical control tasks, both real
and simulated
◦ When state-space is too large to enumerate value functions,
function approximators are used including,
1 Neural networks
2 Clustering
3 Nearest-neighbor methods
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
Reinforcement Learning
◦ Has been applied to wide variety of physical control tasks, both real
and simulated
◦ When state-space is too large to enumerate value functions,
function approximators are used including,
1 Neural networks
2 Clustering
3 Nearest-neighbor methods
4 Tile coding, and
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
Reinforcement Learning
◦ Has been applied to wide variety of physical control tasks, both real
and simulated
◦ When state-space is too large to enumerate value functions,
function approximators are used including,
1 Neural networks
2 Clustering
3 Nearest-neighbor methods
4 Tile coding, and
5 Cerebellar model articulator controller
Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
3.2 Implementation of Shape Change Learning Agent
For morphing wing problem:
RL agent independently maneuvers wing from some initial state to
final state, characterized by aerodynamic properties of the wing
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
3.2 Implementation of Shape Change Learning Agent
For morphing wing problem:
RL agent independently maneuvers wing from some initial state to
final state, characterized by aerodynamic properties of the wing
Agent learns from its interaction with the environment to command a
series of actions that changes the states of morphing wing’s thickness
or camber toward an optimal shape - optimal policy
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
3.2 Implementation of Shape Change Learning Agent
For morphing wing problem:
RL agent independently maneuvers wing from some initial state to
final state, characterized by aerodynamic properties of the wing
Agent learns from its interaction with the environment to command a
series of actions that changes the states of morphing wing’s thickness
or camber toward an optimal shape - optimal policy
Environment is the resulting aerodynamics to which the wing is
subjected to
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
3.2 Implementation of Shape Change Learning Agent
For morphing wing problem:
RL agent independently maneuvers wing from some initial state to
final state, characterized by aerodynamic properties of the wing
Agent learns from its interaction with the environment to command a
series of actions that changes the states of morphing wing’s thickness
or camber toward an optimal shape - optimal policy
Environment is the resulting aerodynamics to which the wing is
subjected to
RL agent has no prior knowledge of relationship between actions and
thickness and camber of morphing wing
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
3.2 Implementation of Shape Change Learning Agent
For morphing wing problem:
RL agent independently maneuvers wing from some initial state to
final state, characterized by aerodynamic properties of the wing
Agent learns from its interaction with the environment to command a
series of actions that changes the states of morphing wing’s thickness
or camber toward an optimal shape - optimal policy
Environment is the resulting aerodynamics to which the wing is
subjected to
RL agent has no prior knowledge of relationship between actions and
thickness and camber of morphing wing
RL agent knows all possible actions that can be applied
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
3.2 Implementation of Shape Change Learning Agent
For morphing wing problem:
RL agent independently maneuvers wing from some initial state to
final state, characterized by aerodynamic properties of the wing
Agent learns from its interaction with the environment to command a
series of actions that changes the states of morphing wing’s thickness
or camber toward an optimal shape - optimal policy
Environment is the resulting aerodynamics to which the wing is
subjected to
RL agent has no prior knowledge of relationship between actions and
thickness and camber of morphing wing
RL agent knows all possible actions that can be applied
RL agent has accurate, real-time information of morphing wing shape,
present aerodynamics, and current reward provided by environment
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
3.2 Implementation of Shape Change Learning Agent
For morphing wing problem:
RL agent independently maneuvers wing from some initial state to
final state, characterized by aerodynamic properties of the wing
Agent learns from its interaction with the environment to command a
series of actions that changes the states of morphing wing’s thickness
or camber toward an optimal shape - optimal policy
Environment is the resulting aerodynamics to which the wing is
subjected to
RL agent has no prior knowledge of relationship between actions and
thickness and camber of morphing wing
RL agent knows all possible actions that can be applied
RL agent has accurate, real-time information of morphing wing shape,
present aerodynamics, and current reward provided by environment
◦ Sensory info from SMA
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
3.2 Implementation of Shape Change Learning Agent
RL agent uses a 1-step Q-learning method - an off-policy TD control algo
Q(s, a) ← Q(s, a) + α[r + γ max
a0
Q(s
0
, a
0
) − Q(s, a)]
Initialize Q(s,a) arbitrarily
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 23 / 69
3.2 Implementation of Shape Change Learning Agent
RL agent uses a 1-step Q-learning method - an off-policy TD control algo
Q(s, a) ← Q(s, a) + α[r + γ max
a0
Q(s
0
, a
0
) − Q(s, a)]
Initialize Q(s,a) arbitrarily
Repeat (for each episode)
- Initialize s
- Repeat (for each step of the episode)
* Choose a from s using policy derived from Q(s,a) (e.g.
-Greedy Policy)
* Take action a, observe reward r, state s’
* Q(s, a) ← Q(s, a) + α[r + γ maxa0 Q(s
0
, a
0
) − Q(s, a)]
* s ← s’
- until s is terminal
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 23 / 69
3.2 Implementation of Shape Change Learning Agent
RL agent uses a 1-step Q-learning method - an off-policy TD control algo
Q(s, a) ← Q(s, a) + α[r + γ max
a0
Q(s
0
, a
0
) − Q(s, a)]
Initialize Q(s,a) arbitrarily
Repeat (for each episode)
- Initialize s
- Repeat (for each step of the episode)
* Choose a from s using policy derived from Q(s,a) (e.g.
-Greedy Policy)
* Take action a, observe reward r, state s’
* Q(s, a) ← Q(s, a) + α[r + γ maxa0 Q(s
0
, a
0
) − Q(s, a)]
* s ← s’
- until s is terminal
return Q(s,a)
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 23 / 69
Implementation of Shape Change Learning Agent
Q(s,a) can be updated using:
1 a random policy, with each action having the same probability of
selection
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 24 / 69
Implementation of Shape Change Learning Agent
Q(s,a) can be updated using:
1 a random policy, with each action having the same probability of
selection
2 an -greedy policy, where  is a small value
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 24 / 69
Implementation of Shape Change Learning Agent
Q(s,a) can be updated using:
1 a random policy, with each action having the same probability of
selection
2 an -greedy policy, where  is a small value
Action a with the maximum Q(s,a) is selected with probability 1 − ,
otherwise a random action is selected
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 24 / 69
Implementation of Shape Change Learning Agent
Q(s,a) can be updated using:
1 a random policy, with each action having the same probability of
selection
2 an -greedy policy, where  is a small value
Action a with the maximum Q(s,a) is selected with probability 1 − ,
otherwise a random action is selected
RL agent learns -greedy policy defined as:
if (probability  1 − )
a = argmaxaQ(s, a)
else
a = rand(ai )
(7)
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 24 / 69
Implementation of Shape Change Learning Agent
As # of learning episodes increase, learned action-value function
Q(s,a) converges asymptotically to optimal action-value function
Q∗(s, a)
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 25 / 69
Implementation of Shape Change Learning Agent
As # of learning episodes increase, learned action-value function
Q(s,a) converges asymptotically to optimal action-value function
Q∗(s, a)
Q-Learning method is an off-policy one because it evaluates target
policy (the greedy policy) while following another policy
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 25 / 69
Implementation of Shape Change Learning Agent
With fewer # of states  actions of RL problem, action-value
function Q(s,a) can be tabulated, where an entity in the table is
action value for each state-action pair
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 26 / 69
Implementation of Shape Change Learning Agent
With fewer # of states  actions of RL problem, action-value
function Q(s,a) can be tabulated, where an entity in the table is
action value for each state-action pair
States in RL problem for morphing vehicle are shape of wings
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 26 / 69
Implementation of Shape Change Learning Agent
With fewer # of states  actions of RL problem, action-value
function Q(s,a) can be tabulated, where an entity in the table is
action value for each state-action pair
States in RL problem for morphing vehicle are shape of wings
States in morphing vehicle are continuous  there are an infinite # of
state-action pairs
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 26 / 69
Implementation of Shape Change Learning Agent
With fewer # of states  actions of RL problem, action-value
function Q(s,a) can be tabulated, where an entity in the table is
action value for each state-action pair
States in RL problem for morphing vehicle are shape of wings
States in morphing vehicle are continuous  there are an infinite # of
state-action pairs
To reduce # of state-action pairs, artificially quantize states into
discrete sets while maintaining integrity of learned action-value
function
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 26 / 69
Implementation of Shape Change Learning Agent
With fewer # of states  actions of RL problem, action-value
function Q(s,a) can be tabulated, where an entity in the table is
action value for each state-action pair
States in RL problem for morphing vehicle are shape of wings
States in morphing vehicle are continuous  there are an infinite # of
state-action pairs
To reduce # of state-action pairs, artificially quantize states into
discrete sets while maintaining integrity of learned action-value
function
Depth of quantization is determined through experiments
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 26 / 69
Implementation of Shape Change Learning Agent
With fewer # of states  actions of RL problem, action-value
function Q(s,a) can be tabulated, where an entity in the table is
action value for each state-action pair
States in RL problem for morphing vehicle are shape of wings
States in morphing vehicle are continuous  there are an infinite # of
state-action pairs
To reduce # of state-action pairs, artificially quantize states into
discrete sets while maintaining integrity of learned action-value
function
Depth of quantization is determined through experiments
# of state-action pairs must be kept manageable while adding more
morphing state variables to existing wing states of thickness  camber
Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 26 / 69
4.1 Aerodynamic Modeling of Morphing Air Vehicle
Constant strength doublet-source panel6model used to calculate
aerodynamic properties of various wing configurations
6
Niksch 2009
Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
4.1 Aerodynamic Modeling of Morphing Air Vehicle
Constant strength doublet-source panel6model used to calculate
aerodynamic properties of various wing configurations
Better than CFD methods
6
Niksch 2009
Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
4.1 Aerodynamic Modeling of Morphing Air Vehicle
Constant strength doublet-source panel6model used to calculate
aerodynamic properties of various wing configurations
Better than CFD methods
Assumption: Flow is incompressible and inviscid
6
Niksch 2009
Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
4.1 Aerodynamic Modeling of Morphing Air Vehicle
Constant strength doublet-source panel6model used to calculate
aerodynamic properties of various wing configurations
Better than CFD methods
Assumption: Flow is incompressible and inviscid
=⇒ Model is valid for linear range of angle-of-attack
6
Niksch 2009
Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
4.1 Aerodynamic Modeling of Morphing Air Vehicle
Constant strength doublet-source panel6model used to calculate
aerodynamic properties of various wing configurations
Better than CFD methods
Assumption: Flow is incompressible and inviscid
=⇒ Model is valid for linear range of angle-of-attack
Limitations of the model:
6
Niksch 2009
Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
4.1 Aerodynamic Modeling of Morphing Air Vehicle
Constant strength doublet-source panel6model used to calculate
aerodynamic properties of various wing configurations
Better than CFD methods
Assumption: Flow is incompressible and inviscid
=⇒ Model is valid for linear range of angle-of-attack
Limitations of the model:
1 Sensitive to the grid size, location of panels, and # of panels
6
Niksch 2009
Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
4.1 Aerodynamic Modeling of Morphing Air Vehicle
Constant strength doublet-source panel6model used to calculate
aerodynamic properties of various wing configurations
Better than CFD methods
Assumption: Flow is incompressible and inviscid
=⇒ Model is valid for linear range of angle-of-attack
Limitations of the model:
1 Sensitive to the grid size, location of panels, and # of panels
because model uses a panel method to determine aerodynamics
6
Niksch 2009
Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
4.1 Aerodynamic Modeling of Morphing Air Vehicle
Constant strength doublet-source panel6model used to calculate
aerodynamic properties of various wing configurations
Better than CFD methods
Assumption: Flow is incompressible and inviscid
=⇒ Model is valid for linear range of angle-of-attack
Limitations of the model:
1 Sensitive to the grid size, location of panels, and # of panels
because model uses a panel method to determine aerodynamics
2 Only NACA 4-Digit Series airfoils considered
◦ Have explicit equations that describe upper and lower
geometries of airfoil sections as a function of camber and thickness
◦ Easier to examine and optimize to achieve best possible wing
shape
◦ Have blunt leading edges for thick/thin airfoil sections
◦ Are ideal for subsonic flight conditions
6
Niksch 2009
Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
Versatile Aerodynamic Modeling of Morphing Air Vehicle
Morphing DOF and flight condition parameters in model are:
1 Wing thickness, t
2 Maximum camber
3 Location of maximum camber
4 Root chord, cr
5 Tip chord, ct
6 Sweep angle, Λ
7 Dihedral angle, Γ
8 Wing span, b
9 Wing angle-of-attack, α
Kumar Digvijay Mishra 4.1 Mathematical Modeling January 11, 2021 28 / 69
4.2 Constitutive Equations
Constant strength doublet-source panel method is used for modeling
wing
7
Katz and Plotkin 2001
Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 29 / 69
4.2 Constitutive Equations
Constant strength doublet-source panel method is used for modeling
wing
Basic potential flow theory7used to obtain equations for aerodynamic
forces on wing
52
φ = 0 (8)
Laplace equation where 52 is the Laplace operator and φ is the
velocity potential
7
Katz and Plotkin 2001
Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 29 / 69
4.2 Constitutive Equations
Constant strength doublet-source panel method is used for modeling
wing
Basic potential flow theory7used to obtain equations for aerodynamic
forces on wing
52
φ = 0 (8)
Laplace equation where 52 is the Laplace operator and φ is the
velocity potential
Assumption: Incompressible and inviscid flow
7
Katz and Plotkin 2001
Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 29 / 69
Constitutive Equations
Using Green’s identity, solution to Laplace equation is formed with a
sum of source σ and doublet µ distributions along the boundary SB
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 30 / 69
Constitutive Equations
Using Green’s identity, solution to Laplace equation is formed with a
sum of source σ and doublet µ distributions along the boundary SB
φ = −
1
4π
Z
SB
[σ(
1
r
) − µn · 5(
1
r
)]dS + φ∞ (9)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 30 / 69
Constitutive Equations
Using Green’s identity, solution to Laplace equation is formed with a
sum of source σ and doublet µ distributions along the boundary SB
φ = −
1
4π
Z
SB
[σ(
1
r
) − µn · 5(
1
r
)]dS + φ∞ (9)
Assuming wake convects at trailing edge of wing as a set of thin
doublets, (9) becomes
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 30 / 69
Constitutive Equations
Using Green’s identity, solution to Laplace equation is formed with a
sum of source σ and doublet µ distributions along the boundary SB
φ = −
1
4π
Z
SB
[σ(
1
r
) − µn · 5(
1
r
)]dS + φ∞ (9)
Assuming wake convects at trailing edge of wing as a set of thin
doublets, (9) becomes
φ =
1
4π
Z
Body+Wake
µn · 5(
1
r
)dS −
1
4π
Z
Body
σ(
1
r
) + φ∞ (10)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 30 / 69
Boundary Conditions
BC is:
No-penetration condition - requires normal velocity of flow at surface
to be zero
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 31 / 69
Boundary Conditions
BC is:
No-penetration condition - requires normal velocity of flow at surface
to be zero
Specified by:
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 31 / 69
Boundary Conditions
BC is:
No-penetration condition - requires normal velocity of flow at surface
to be zero
Specified by:
1 Direct formulation/Neumann condition: forces normal velocity
component of flow at surface to be zero
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 31 / 69
Boundary Conditions
BC is:
No-penetration condition - requires normal velocity of flow at surface
to be zero
Specified by:
1 Direct formulation/Neumann condition: forces normal velocity
component of flow at surface to be zero
2 Indirect formulation/Dirichlet condition: specifies a value for the
potential function, and indirectly satisfies zero normal flow condition on
boundary
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 31 / 69
Constitutive Equations
φ =
1
4π
Z
Body+Wake
µn · 5(
1
r
)dS −
1
4π
Z
Body
σ(
1
r
) + φ∞
Using Dirichlet BC, potential is specified at all points on the boundary
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 32 / 69
Constitutive Equations
φ =
1
4π
Z
Body+Wake
µn · 5(
1
r
)dS −
1
4π
Z
Body
σ(
1
r
) + φ∞
Using Dirichlet BC, potential is specified at all points on the boundary
If a point is placed inside the surface, inner potential, φi , is defined by
the singularity distributions along the surface
φi =
1
4π
Z
Body+Wake
µ
∂
∂n

1
r

dS −
1
4π
Z
Body
σ

1
r

+ φ∞ (11)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 32 / 69
Constitutive Equations
φ =
1
4π
Z
Body+Wake
µn · 5(
1
r
)dS −
1
4π
Z
Body
σ(
1
r
) + φ∞
Using Dirichlet BC, potential is specified at all points on the boundary
If a point is placed inside the surface, inner potential, φi , is defined by
the singularity distributions along the surface
φi =
1
4π
Z
Body+Wake
µ
∂
∂n

1
r

dS −
1
4π
Z
Body
σ

1
r

+ φ∞ (11)
Do you see any problem in the integral ?
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 32 / 69
Constitutive Equations
φ =
1
4π
Z
Body+Wake
µn · 5(
1
r
)dS −
1
4π
Z
Body
σ(
1
r
) + φ∞
Using Dirichlet BC, potential is specified at all points on the boundary
If a point is placed inside the surface, inner potential, φi , is defined by
the singularity distributions along the surface
φi =
1
4π
Z
Body+Wake
µ
∂
∂n

1
r

dS −
1
4π
Z
Body
σ

1
r

+ φ∞ (11)
Do you see any problem in the integral ?
Singularity as r → 0
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 32 / 69
Constitutive Equations
φi =
1
4π
Z
Body+Wake
µ
∂
∂n

1
r

dS −
1
4π
Z
Body
σ

1
r

+ φ∞
To evaluate integrals near singular points, no-penetration BC is
enforced by setting φi to a constant value
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 33 / 69
Constitutive Equations
φi =
1
4π
Z
Body+Wake
µ
∂
∂n

1
r

dS −
1
4π
Z
Body
σ

1
r

+ φ∞
To evaluate integrals near singular points, no-penetration BC is
enforced by setting φi to a constant value
By choosing φi to be φ∞, we get
1
4π
Z
Body+Wake
µ
∂
∂n

1
r

dS −
1
4π
Z
Body
σ

1
r

= 0 (12)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 33 / 69
Constitutive Equations
φi =
1
4π
Z
Body+Wake
µ
∂
∂n

1
r

dS −
1
4π
Z
Body
σ

1
r

+ φ∞
To evaluate integrals near singular points, no-penetration BC is
enforced by setting φi to a constant value
By choosing φi to be φ∞, we get
1
4π
Z
Body+Wake
µ
∂
∂n

1
r

dS −
1
4π
Z
Body
σ

1
r

= 0 (12)
Integral equations are reduced to a set of linear algebraic equations
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 33 / 69
Constitutive Equations
Integral equations are reduced to a set of linear algebraic equations
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 34 / 69
Constitutive Equations
Integral equations are reduced to a set of linear algebraic equations
System divided into N panels for surface  Nw panels for wake
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 34 / 69
Constitutive Equations
Integral equations are reduced to a set of linear algebraic equations
System divided into N panels for surface  Nw panels for wake
BC specified at a collocation point, which for the Dirichlet BC is
inside body and at center of panel
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 34 / 69
Constitutive Equations
1
4π
Z
Body+Wake
µ
∂
∂n

1
r

dS −
1
4π
Z
Body
σ

1
r

= 0
For N collocation points for N panels, above equation becomes,
N
X
k=1
1
4π
Z
BodyPanel
µn · 5

1
r

dS +
Nw
X
l=1
1
4π
Z
WakePanel
µn · 5

1
r

dS
−
N
X
k=1
1
4π
Z
BodyPanel
σ

1
r

dS = 0
(13)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 35 / 69
Constitutive Equations
N
X
k=1
1
4π
Z
BodyPanel
µn · 5

1
r

dS +
Nw
X
l=1
1
4π
Z
WakePanel
µn · 5

1
r

dS
−
N
X
k=1
1
4π
Z
BodyPanel
σ

1
r

dS = 0
For each collocation point inside body,
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 36 / 69
Constitutive Equations
N
X
k=1
1
4π
Z
BodyPanel
µn · 5

1
r

dS +
Nw
X
l=1
1
4π
Z
WakePanel
µn · 5

1
r

dS
−
N
X
k=1
1
4π
Z
BodyPanel
σ

1
r

dS = 0
For each collocation point inside body,
N
X
k=1
Akµk +
Nw
X
l=1
Al µl −
N
X
k=1
Bkσk = 0 (14)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 36 / 69
Constitutive Equations
N
X
k=1
1
4π
Z
BodyPanel
µn · 5

1
r

dS +
Nw
X
l=1
1
4π
Z
WakePanel
µn · 5

1
r

dS
−
N
X
k=1
1
4π
Z
BodyPanel
σ

1
r

dS = 0
For each collocation point inside body,
N
X
k=1
Akµk +
Nw
X
l=1
Al µl −
N
X
k=1
Bkσk = 0 (14)
A is doublet influence coefficient, and
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 36 / 69
Constitutive Equations
N
X
k=1
1
4π
Z
BodyPanel
µn · 5

1
r

dS +
Nw
X
l=1
1
4π
Z
WakePanel
µn · 5

1
r

dS
−
N
X
k=1
1
4π
Z
BodyPanel
σ

1
r

dS = 0
For each collocation point inside body,
N
X
k=1
Akµk +
Nw
X
l=1
Al µl −
N
X
k=1
Bkσk = 0 (14)
A is doublet influence coefficient, and
B is source influence coefficient
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 36 / 69
Constitutive Equations
N
X
k=1
Akµk +
Nw
X
l=1
Al µl −
N
X
k=1
Bkσk = 0
Kutta condition(KC): No circulation at the trailing edge of a wing
section
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 37 / 69
Constitutive Equations
N
X
k=1
Akµk +
Nw
X
l=1
Al µl −
N
X
k=1
Bkσk = 0
Kutta condition(KC): No circulation at the trailing edge of a wing
section
Using KC, doublet strength of wake panel ≡ difference between
trailing edge panels on upper surface and lower surface
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 37 / 69
Constitutive Equations
N
X
k=1
Akµk +
Nw
X
l=1
Al µl −
N
X
k=1
Bkσk = 0
Kutta condition(KC): No circulation at the trailing edge of a wing
section
Using KC, doublet strength of wake panel ≡ difference between
trailing edge panels on upper surface and lower surface
KC eliminates wake contribution in above equation,
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 37 / 69
Constitutive Equations
N
X
k=1
Akµk +
Nw
X
l=1
Al µl −
N
X
k=1
Bkσk = 0
Kutta condition(KC): No circulation at the trailing edge of a wing
section
Using KC, doublet strength of wake panel ≡ difference between
trailing edge panels on upper surface and lower surface
KC eliminates wake contribution in above equation,



a11 . . . a1N
.
.
.
...
.
.
.
aN1 . . . aNN






µ1
.
.
.
µN


 = −



b11 . . . b1N
.
.
.
...
.
.
.
bN1 . . . bNN






σ1
.
.
.
σN


 (15)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 37 / 69
Constitutive Equations



a11 . . . a1N
.
.
.
...
.
.
.
aN1 . . . aNN






µ1
.
.
.
µN


 = −



b11 . . . b1N
.
.
.
...
.
.
.
bN1 . . . bNN






σ1
.
.
.
σN



Set of N equations with N unknown doublet strengths (µ’s)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 38 / 69
Constitutive Equations



a11 . . . a1N
.
.
.
...
.
.
.
aN1 . . . aNN






µ1
.
.
.
µN


 = −



b11 . . . b1N
.
.
.
...
.
.
.
bN1 . . . bNN






σ1
.
.
.
σN



Set of N equations with N unknown doublet strengths (µ’s)
Solvable by matrix inversion



µ1
.
.
.
µN


 = −



a11 . . . a1N
.
.
.
...
.
.
.
aN1 . . . aNN



−1 


b11 . . . b1N
.
.
.
...
.
.
.
bN1 . . . bNN






σ1
.
.
.
σN



Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 38 / 69
Determination of Aerodynamic Forces
After computing doublet strengths, aerodynamic forces acting on
each panel are found
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 39 / 69
Determination of Aerodynamic Forces
After computing doublet strengths, aerodynamic forces acting on
each panel are found
1 Determine tangential (l,m) and normal (n) perturbation velocity
components for each of the panels
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 39 / 69
Determination of Aerodynamic Forces
After computing doublet strengths, aerodynamic forces acting on
each panel are found
1 Determine tangential (l,m) and normal (n) perturbation velocity
components for each of the panels
ql = −
∂µ
∂l
, qm = −
∂µ
∂m
, qn = −σ (16)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 39 / 69
Determination of Aerodynamic Forces
After computing doublet strengths, aerodynamic forces acting on
each panel are found
1 Determine tangential (l,m) and normal (n) perturbation velocity
components for each of the panels
ql = −
∂µ
∂l
, qm = −
∂µ
∂m
, qn = −σ (16)
2 Total velocity of each panel k is computed with
Qk = (Q∞l , Q∞m, Q∞n)k + (ql , qm, qn)k (17)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 39 / 69
Determination of Aerodynamic Forces
Qk = (Q∞l , Q∞m, Q∞n)k + (ql , qm, qn)k
These velocities are expressed in local panel coordinate system
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 40 / 69
Determination of Aerodynamic Forces
Qk = (Q∞l , Q∞m, Q∞n)k + (ql , qm, qn)k
These velocities are expressed in local panel coordinate system
3 Pressure coefficient at each panel is found using modified form of
Bernoulli’s equation
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 40 / 69
Determination of Aerodynamic Forces
Qk = (Q∞l , Q∞m, Q∞n)k + (ql , qm, qn)k
These velocities are expressed in local panel coordinate system
3 Pressure coefficient at each panel is found using modified form of
Bernoulli’s equation
Cpk
= 1 −
Q2
k
Q2
∞
(18)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 40 / 69
Determination of Aerodynamic Forces
Qk = (Q∞l , Q∞m, Q∞n)k + (ql , qm, qn)k
These velocities are expressed in local panel coordinate system
3 Pressure coefficient at each panel is found using modified form of
Bernoulli’s equation
Cpk
= 1 −
Q2
k
Q2
∞
(18)
4 Summation of aerodynamic forces from each panel gives total
aerodynamic force
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 40 / 69
Determination of Aerodynamic Forces
Qk = (Q∞l , Q∞m, Q∞n)k + (ql , qm, qn)k
These velocities are expressed in local panel coordinate system
3 Pressure coefficient at each panel is found using modified form of
Bernoulli’s equation
Cpk
= 1 −
Q2
k
Q2
∞
(18)
4 Summation of aerodynamic forces from each panel gives total
aerodynamic force
X
CFk
= −
Cpk
4S
S
· nk (19)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 40 / 69
Determination of Aerodynamic Forces
5 Parasitic drag coefficient CD0 is calculated based on wing area S and
equivalent parasite area f
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 41 / 69
Determination of Aerodynamic Forces
5 Parasitic drag coefficient CD0 is calculated based on wing area S and
equivalent parasite area f
CD0 =
f
S
(20)
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 41 / 69
Determination of Aerodynamic Forces
5 Parasitic drag coefficient CD0 is calculated based on wing area S and
equivalent parasite area f
CD0 =
f
S
(20)
where equivalent parasite area is related to wetted area by
log10f = a + b log10Swet (21)
a, b - correlation coefficients related to equivalent skin friction of
aircraft
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 41 / 69
Determination of Aerodynamic Forces
5 Parasitic drag coefficient CD0 is calculated based on wing area S and
equivalent parasite area f
CD0 =
f
S
(20)
where equivalent parasite area is related to wetted area by
log10f = a + b log10Swet (21)
a, b - correlation coefficients related to equivalent skin friction of
aircraft
6 Oswald efficiency factor is,
e = 1.78(1 − 0.045 AR0.68
) − 0.64 (22)
valid if sweep angle of leading edge of wing  300
Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 41 / 69
4.3 Model Grid
Cosine spacing used to generate grid for both chordwise and spanwise
paneling8
xvi =
c(y)
2

1 − cos

iπ
N + 1

; i = 1 : N (23)
7
Moran 1984
Kumar Digvijay Mishra 4. Mathematical Modeling of Morphing Air Vehicle January 11, 2021 42 / 69
4.3 Model Grid
Cosine spacing used to generate grid for both chordwise and spanwise
paneling8
xvi =
c(y)
2

1 − cos

iπ
N + 1

; i = 1 : N (23)
Cosine spacing gives more panels near leading and trailing edges as
well as root and tip of the wing where many aerodynamic changes
occur
7
Moran 1984
Kumar Digvijay Mishra 4. Mathematical Modeling of Morphing Air Vehicle January 11, 2021 42 / 69
4.3 Model Grid
Straight tapered wing configuration with cosine spacing
Kumar Digvijay Mishra 4. Mathematical Modeling of Morphing Air Vehicle January 11, 2021 43 / 69
4.3 Model Grid
Swept wing configuration with cosine spacing
More panels placed where many aerodynamic changes occur:
1 Leading and trailing edges as well as root and tip of the wing
2 Four corner points of the wing
Kumar Digvijay Mishra 4. Mathematical Modeling of Morphing Air Vehicle January 11, 2021 44 / 69
4.3 Model Grid
Delta wing configuration with cosine spacing
More panels placed where many aerodynamic changes occur:
1 Leading and trailing edges as well as root and tip of the wing
2 Four corner points of the wing
Kumar Digvijay Mishra 4. Mathematical Modeling of Morphing Air Vehicle January 11, 2021 45 / 69
Model Grid Accuracy
As # of panels decrease, accuracy of model also decreases
Kumar Digvijay Mishra 4.3 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 46 / 69
Model Grid Accuracy
As # of panels decrease, accuracy of model also decreases
Balance between accuracy and computational time is achieved by
defining set # of panels beyond which accuracy does not increase
appreciably
Kumar Digvijay Mishra 4.3 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 46 / 69
Model Grid Accuracy
As # of panels decrease, accuracy of model also decreases
Balance between accuracy and computational time is achieved by
defining set # of panels beyond which accuracy does not increase
appreciably
Ex: Doubling no. of panels is acceptable if accuracy of the
model increased by 10%
Kumar Digvijay Mishra 4.3 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 46 / 69
Model Grid Accuracy
As # of panels decrease, accuracy of model also decreases
Balance between accuracy and computational time is achieved by
defining set # of panels beyond which accuracy does not increase
appreciably
Ex: Doubling no. of panels is acceptable if accuracy of the
model increased by 10%
50% increase in the set of panels is unnecessary, if accuracy is
increased by less than 1%
Kumar Digvijay Mishra 4.3 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 46 / 69
4.4 Dynamical Modeling
Kumar Digvijay Mishra 4. Mathematical Modeling of Morphing Air Vehicle January 11, 2021 47 / 69
Dynamical Modeling
Velocity vector v with body-axis components is
v =


u
v
w

 (24)
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 48 / 69
Dynamical Modeling
Velocity vector v with body-axis components is
v =


u
v
w

 (24)
Angular velocity vector ω with roll(p), pitch(q), and yaw(r)
components in body-axis coordinate system is
ω =


p
q
r

 (25)
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 48 / 69
Dynamical Modeling
Derivative of velocity vector v with respect to time yields
v̇N
= v̇B
+ ω × v =


u̇ + qw − rv
v̇ + ru − pw
ẇ + pv − qu

 (26)
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 49 / 69
Dynamical Modeling
Derivative of velocity vector v with respect to time yields
v̇N
= v̇B
+ ω × v =


u̇ + qw − rv
v̇ + ru − pw
ẇ + pv − qu

 (26)
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 49 / 69
Dynamical Modeling
Derivative of velocity vector v with respect to time yields
v̇N
= v̇B
+ ω × v =


u̇ + qw − rv
v̇ + ru − pw
ẇ + pv − qu

 (26)
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 49 / 69
Dynamical Modeling
Derivative of velocity vector v with respect to time yields
v̇N
= v̇B
+ ω × v =


u̇ + qw − rv
v̇ + ru − pw
ẇ + pv − qu

 (26)
u̇, v̇, and ẇ: acceleration of body along forward, lateral, and
down/up-ward direction
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 49 / 69
Dynamical Modeling
Derivative of velocity vector v with respect to time yields
v̇N
= v̇B
+ ω × v =


u̇ + qw − rv
v̇ + ru − pw
ẇ + pv − qu

 (26)
u̇, v̇, and ẇ: acceleration of body along forward, lateral, and
down/up-ward direction
There is always forward acceleration, but no lateral acceleration for
longitudinal maneuvers
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 49 / 69
Dynamical Modeling
Derivative of velocity vector v with respect to time yields
v̇N
= v̇B
+ ω × v =


u̇ + qw − rv
v̇ + ru − pw
ẇ + pv − qu

 (26)
u̇, v̇, and ẇ: acceleration of body along forward, lateral, and
down/up-ward direction
There is always forward acceleration, but no lateral acceleration for
longitudinal maneuvers
Notice association of {roll(p), pitch(q), yaw(r)} components of
angular velocity with translational velocity components add to
acceleration components (u̇, v̇, ẇ)
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 49 / 69
Dynamical Modeling
Force vector is,
F =


X + Tx + Gx
Y + Ty + Gy
Z + Tz + Gz


=


−mg sin(θ) − D cos(α)cos(β) + L sin(α)cos(β) + T
mg sin(φ)cos(θ) + D cos(α)sin(β) − L sin(α)sin(β)
mg cos(φ)cos(θ) − D sin(α) − L cos(α)


(27)
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 50 / 69
Dynamical Modeling
Force vector is,
F =


X + Tx + Gx
Y + Ty + Gy
Z + Tz + Gz


=


−mg sin(θ) − D cos(α)cos(β) + L sin(α)cos(β) + T
mg sin(φ)cos(θ) + D cos(α)sin(β) − L sin(α)sin(β)
mg cos(φ)cos(θ) − D sin(α) − L cos(α)


(27)
G: Gravitational force; T: Propulsive(thrust) force
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 50 / 69
Dynamical Modeling
Force vector is,
F =


X + Tx + Gx
Y + Ty + Gy
Z + Tz + Gz


=


−mg sin(θ) − D cos(α)cos(β) + L sin(α)cos(β) + T
mg sin(φ)cos(θ) + D cos(α)sin(β) − L sin(α)sin(β)
mg cos(φ)cos(θ) − D sin(α) − L cos(α)


(27)
G: Gravitational force; T: Propulsive(thrust) force
D: Drag force; L: Lift force
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 50 / 69
Dynamical Modeling
Force vector is,
F =


X + Tx + Gx
Y + Ty + Gy
Z + Tz + Gz


=


−mg sin(θ) − D cos(α)cos(β) + L sin(α)cos(β) + T
mg sin(φ)cos(θ) + D cos(α)sin(β) − L sin(α)sin(β)
mg cos(φ)cos(θ) − D sin(α) − L cos(α)


(27)
G: Gravitational force; T: Propulsive(thrust) force
D: Drag force; L: Lift force
(X, Y, Z): Aerodynamic forces calculated using constant strength
source-doublet panel method
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 50 / 69
Dynamical Modeling
Force vector is,
F =


X + Tx + Gx
Y + Ty + Gy
Z + Tz + Gz


=


−mg sin(θ) − D cos(α)cos(β) + L sin(α)cos(β) + T
mg sin(φ)cos(θ) + D cos(α)sin(β) − L sin(α)sin(β)
mg cos(φ)cos(θ) − D sin(α) − L cos(α)


(27)
G: Gravitational force; T: Propulsive(thrust) force
D: Drag force; L: Lift force
(X, Y, Z): Aerodynamic forces calculated using constant strength
source-doublet panel method
ψ, θ, φ: Yaw, pitch  roll angles
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 50 / 69
Dynamical Modeling
Force vector is,
F =


X + Tx + Gx
Y + Ty + Gy
Z + Tz + Gz


=


−mg sin(θ) − D cos(α)cos(β) + L sin(α)cos(β) + T
mg sin(φ)cos(θ) + D cos(α)sin(β) − L sin(α)sin(β)
mg cos(φ)cos(θ) − D sin(α) − L cos(α)


(27)
G: Gravitational force; T: Propulsive(thrust) force
D: Drag force; L: Lift force
(X, Y, Z): Aerodynamic forces calculated using constant strength
source-doublet panel method
ψ, θ, φ: Yaw, pitch  roll angles
(α, β): Aerodynamic angles (AoA, sideslip)
Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 50 / 69
Dynamical Modeling
From Newton’s 2nd law, translational equations of motion are:
m(u̇ +qw −rv) = −mgsin(θ)−Dcos(α)cos(β)+Lsin(α)cos(β)+T (28)
m(v̇ +ru−pw) = −mgsin(φ)cos(θ)−Dcos(α)sin(β)−Lsin(α)sin(β) (29)
m(ẇ + pv − qu) = −mgcos(φ)cos(θ) − Dsin(α) − Lcos(α) (30)
p, q, r: roll, pitch,  yaw components of angular velocity vector, ω
u, v, w: components of body-axis translational velocity vector, v
ψ, θ, φ: yaw(unused), pitch,  roll angles
α, β: aerodynamic angles
T: Thrust; D: Drag ; L: Lift
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 51 / 69
Dynamical Modeling
Euler’s equation L = ḣ
N
gives relationship between vector sum of
moments L and angular momentum vector h
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 52 / 69
Dynamical Modeling
Euler’s equation L = ḣ
N
gives relationship between vector sum of
moments L and angular momentum vector h
Angular momentum vector related to inertia matrix I and body-axis
angular velocity vector ω by
h = Iω (31)
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 52 / 69
Dynamical Modeling
Euler’s equation L = ḣ
N
gives relationship between vector sum of
moments L and angular momentum vector h
Angular momentum vector related to inertia matrix I and body-axis
angular velocity vector ω by
h = Iω (31)
Inertia matrix is,
I =
 Ixx −Ixy −Ixz
−Ixy Iyy −Iyz
−Ixz −Iyz Izz

(32)
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 52 / 69
Dynamical Modeling
Euler’s equation L = ḣ
N
gives relationship between vector sum of
moments L and angular momentum vector h
Angular momentum vector related to inertia matrix I and body-axis
angular velocity vector ω by
h = Iω (31)
Inertia matrix is,
I =
 Ixx −Ixy −Ixz
−Ixy Iyy −Iyz
−Ixz −Iyz Izz

(32)
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 52 / 69
Dynamical Modeling
Euler’s equation L = ḣ
N
gives relationship between vector sum of
moments L and angular momentum vector h
Angular momentum vector related to inertia matrix I and body-axis
angular velocity vector ω by
h = Iω (31)
Inertia matrix is,
I =
 Ixx −Ixy −Ixz
−Ixy Iyy −Iyz
−Ixz −Iyz Izz

(32)
1 These moments-of-inertia are time-varying because aircraft wing
morphs during flight
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 52 / 69
Dynamical Modeling
Euler’s equation L = ḣ
N
gives relationship between vector sum of
moments L and angular momentum vector h
Angular momentum vector related to inertia matrix I and body-axis
angular velocity vector ω by
h = Iω (31)
Inertia matrix is,
I =
 Ixx −Ixy −Ixz
−Ixy Iyy −Iyz
−Ixz −Iyz Izz

(32)
1 These moments-of-inertia are time-varying because aircraft wing
morphs during flight
2 Should be computed for each state of morphing wing aircraft as well as
transitions between them. How to compute them in real-time?
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 52 / 69
Dynamical Modeling
Time derivative of angular momentum vector h is,
ḣ
N
= Iω̇B
+ İ
B
ω + ω × Iω
=








Ixx ṗ − Ixy q̇ − Ixz ˙
r + ˙
Ixx p − ˙
Ixy q − ˙
Ixzr+p(−Ixzq + Ixy r)
+ q(−Iyzq − Iyy r) + r(Izzq + Iyzr)
−Ixy ṗ − Iyy q̇ − Iyz ˙
r − ˙
Ixy p + ˙
Iyy q − ˙
Iyzr+p(Ixx r + Ixzp)
+ q(−Ixy r + Iyzp) + r(−Ixzr − Izzp)
−Ixzṗ − Iyzq̇ + Izz ˙
r − ˙
Ixzp − ˙
Iyzq + ˙
Izzr+p(−Ixy p − Ixx q)
+q(Iyy p + Ixy q) + r(−Iyzp + Ixzq)








(33)
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 53 / 69
Dynamical Modeling
Time derivative of angular momentum vector h is,
ḣ
N
= Iω̇B
+ İ
B
ω + ω × Iω
=








Ixx ṗ − Ixy q̇ − Ixz ˙
r + ˙
Ixx p − ˙
Ixy q − ˙
Ixzr+p(−Ixzq + Ixy r)
+ q(−Iyzq − Iyy r) + r(Izzq + Iyzr)
−Ixy ṗ − Iyy q̇ − Iyz ˙
r − ˙
Ixy p + ˙
Iyy q − ˙
Iyzr+p(Ixx r + Ixzp)
+ q(−Ixy r + Iyzp) + r(−Ixzr − Izzp)
−Ixzṗ − Iyzq̇ + Izz ˙
r − ˙
Ixzp − ˙
Iyzq + ˙
Izzr+p(−Ixy p − Ixx q)
+q(Iyy p + Ixy q) + r(−Iyzp + Ixzq)








(33)
9 inertia terms that vary in temporal dimension
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 53 / 69
Dynamical Modeling
Time derivative of angular momentum vector h is,
ḣ
N
= Iω̇B
+ İ
B
ω + ω × Iω
=








Ixx ṗ − Ixy q̇ − Ixz ˙
r + ˙
Ixx p − ˙
Ixy q − ˙
Ixzr+p(−Ixzq + Ixy r)
+ q(−Iyzq − Iyy r) + r(Izzq + Iyzr)
−Ixy ṗ − Iyy q̇ − Iyz ˙
r − ˙
Ixy p + ˙
Iyy q − ˙
Iyzr+p(Ixx r + Ixzp)
+ q(−Ixy r + Iyzp) + r(−Ixzr − Izzp)
−Ixzṗ − Iyzq̇ + Izz ˙
r − ˙
Ixzp − ˙
Iyzq + ˙
Izzr+p(−Ixy p − Ixx q)
+q(Iyy p + Ixy q) + r(−Iyzp + Ixzq)








(33)
9 inertia terms that vary in temporal dimension
Is it prudent to neglect time-varying inertia due to morphing?
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 53 / 69
Dynamical Modeling
Moments are due to ?
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
Dynamical Modeling
Moments are due to ?
Aerodynamic forces
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
Dynamical Modeling
Moments are due to ?
Aerodynamic forces
where aerodynamic forces are calculated by ?
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
Dynamical Modeling
Moments are due to ?
Aerodynamic forces
where aerodynamic forces are calculated by ?
Constant strength source-doublet panel method
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
Dynamical Modeling
Moments are due to ?
Aerodynamic forces
where aerodynamic forces are calculated by ?
Constant strength source-doublet panel method
Aerodynamic moment vector is,
L =


LA
MA
NA

 (34)
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
Dynamical Modeling
Moments are due to ?
Aerodynamic forces
where aerodynamic forces are calculated by ?
Constant strength source-doublet panel method
Aerodynamic moment vector is,
L =


LA
MA
NA

 (34)
LA: Roll moment MA: Pitch moment NA: Yaw moment
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
Dynamical Modeling
Moments are due to ?
Aerodynamic forces
where aerodynamic forces are calculated by ?
Constant strength source-doublet panel method
Aerodynamic moment vector is,
L =


LA
MA
NA

 (34)
LA: Roll moment MA: Pitch moment NA: Yaw moment
Why not account for moment produced by propulsive forces?
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
Dynamical Modeling
Moments are due to ?
Aerodynamic forces
where aerodynamic forces are calculated by ?
Constant strength source-doublet panel method
Aerodynamic moment vector is,
L =


LA
MA
NA

 (34)
LA: Roll moment MA: Pitch moment NA: Yaw moment
Why not account for moment produced by propulsive forces?
Because thrust force is assumed to act along the line parallel to
body’s x-axis at C-of-G location
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
Rotational Equation of Motion for Morphing Wing
Using Euler’s equation L = ḣ
N
,
LA = Ixx ṗ − Ixy q̇ − Ixz ˙
r + ˙
Ixx p − ˙
Ixy q − ˙
Ixzr+p(−Ixzq + Ixy r)
+ q(−Iyzq − Iyy r) + r(Izzq + Iyzr)
(35)
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 55 / 69
Rotational Equation of Motion for Morphing Wing
Using Euler’s equation L = ḣ
N
,
LA = Ixx ṗ − Ixy q̇ − Ixz ˙
r + ˙
Ixx p − ˙
Ixy q − ˙
Ixzr+p(−Ixzq + Ixy r)
+ q(−Iyzq − Iyy r) + r(Izzq + Iyzr)
(35)
MA = −Ixy ṗ + Iyy q̇ − Iyz ˙
r − ˙
Ixy p + ˙
Iyy q − ˙
Iyzr+p(Ixx r + Ixzp)
+ q(−Ixy r + Iyzp) + r(−Ixzr − Izzp)
(36)
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 55 / 69
Rotational Equation of Motion for Morphing Wing
Using Euler’s equation L = ḣ
N
,
LA = Ixx ṗ − Ixy q̇ − Ixz ˙
r + ˙
Ixx p − ˙
Ixy q − ˙
Ixzr+p(−Ixzq + Ixy r)
+ q(−Iyzq − Iyy r) + r(Izzq + Iyzr)
(35)
MA = −Ixy ṗ + Iyy q̇ − Iyz ˙
r − ˙
Ixy p + ˙
Iyy q − ˙
Iyzr+p(Ixx r + Ixzp)
+ q(−Ixy r + Iyzp) + r(−Ixzr − Izzp)
(36)
NA = −Ixzṗ − Iyzq̇ + Izz ˙
r − ˙
Ixzp − ˙
Iyzq + ˙
Izzr+p(−Ixy p − Ixx q)
+ q(Iyy p + Ixy q) + r(−Iyzp + Ixzq)
(37)
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 55 / 69
Rotational Equation of Motion for Morphing Wing
Using Euler’s equation L = ḣ
N
,
LA = Ixx ṗ − Ixy q̇ − Ixz ˙
r + ˙
Ixx p − ˙
Ixy q − ˙
Ixzr+p(−Ixzq + Ixy r)
+ q(−Iyzq − Iyy r) + r(Izzq + Iyzr)
(35)
MA = −Ixy ṗ + Iyy q̇ − Iyz ˙
r − ˙
Ixy p + ˙
Iyy q − ˙
Iyzr+p(Ixx r + Ixzp)
+ q(−Ixy r + Iyzp) + r(−Ixzr − Izzp)
(36)
NA = −Ixzṗ − Iyzq̇ + Izz ˙
r − ˙
Ixzp − ˙
Iyzq + ˙
Izzr+p(−Ixy p − Ixx q)
+ q(Iyy p + Ixy q) + r(−Iyzp + Ixzq)
(37)
Time-varying inertias are present in roll, pitch and yaw moments.
Why neglect them?
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 55 / 69
Exercise - Inertia of an aircraft
Ixx = N/m2 Ixy = N/m2 Ixz = N/m2
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 56 / 69
Exercise - Inertia of an aircraft
Ixx = N/m2 Ixy = N/m2 Ixz = N/m2
Iyx = N/m2 Iyy = N/m2 Iyz = N/m2
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 56 / 69
Exercise - Inertia of an aircraft
Ixx = N/m2 Ixy = N/m2 Ixz = N/m2
Iyx = N/m2 Iyy = N/m2 Iyz = N/m2
Izz = N/m2 Izy = N/m2 Izz = N/m2
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 56 / 69
Exercise - Inertia of an aircraft
Ixx = N/m2 Ixy = N/m2 Ixz = N/m2
Iyx = N/m2 Iyy = N/m2 Iyz = N/m2
Izz = N/m2 Izy = N/m2 Izz = N/m2
Rudder Fault =⇒ Transience in Inertia - Is ˙
Iij negligible?
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 56 / 69
Dynamical Modeling
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 57 / 69
Dynamical Modeling
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 57 / 69
Dynamical Modeling
Aerodynamic angles α and β are,
tan(α) =
w
VT
(38)
tan(β) =
v
VT
(39)
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 57 / 69
Dynamical Modeling
Aerodynamic angles α and β are,
tan(α) =
w
VT
(38)
tan(β) =
v
VT
(39)
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 57 / 69
Dynamical Modeling
Aerodynamic angles α and β are,
tan(α) =
w
VT
(38)
tan(β) =
v
VT
(39)
Body axis system
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 57 / 69
Dynamical Modeling
Aerodynamic angles α and β are,
tan(α) =
w
VT
(38)
tan(β) =
v
VT
(39)
Body axis system
w: velocity along body z-axis
v: velocity along body y-axis
VT : total body velocity
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 57 / 69
Dynamical Modeling
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 58 / 69
Dynamical Modeling
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 58 / 69
Dynamical Modeling
Aerodynamic angles change at a rate
of,
α̇ =
1
VT
(ẇ cos α − u̇ sin α) (40)
β̇ =
1
VT
(v̇ cos β − u̇ sin β) (41)
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 58 / 69
Dynamical Modeling
Aerodynamic angles change at a rate
of,
α̇ =
1
VT
(ẇ cos α − u̇ sin α) (40)
β̇ =
1
VT
(v̇ cos β − u̇ sin β) (41)
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 58 / 69
Dynamical Modeling
Aerodynamic angles change at a rate
of,
α̇ =
1
VT
(ẇ cos α − u̇ sin α) (40)
β̇ =
1
VT
(v̇ cos β − u̇ sin β) (41)
Body axis system
Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 58 / 69
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control
Morphing UAV Intelligent Shape and Flight Control

More Related Content

Similar to Morphing UAV Intelligent Shape and Flight Control

Lec 03(VDIdr shady)
Lec 03(VDIdr shady)Lec 03(VDIdr shady)
Lec 03(VDIdr shady)Mohamed Atef
 
Aircraft pitch control design using LQG controller based on genetic algorithm
Aircraft pitch control design using LQG controller based on genetic algorithmAircraft pitch control design using LQG controller based on genetic algorithm
Aircraft pitch control design using LQG controller based on genetic algorithmTELKOMNIKA JOURNAL
 
Design of imc based controller for industrial purpose
Design of imc based controller for industrial purposeDesign of imc based controller for industrial purpose
Design of imc based controller for industrial purpose375ankit
 
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...IJERD Editor
 
Modelling and Simulation of DC-Motor Electric Drive Control System with Varia...
Modelling and Simulation of DC-Motor Electric Drive Control System with Varia...Modelling and Simulation of DC-Motor Electric Drive Control System with Varia...
Modelling and Simulation of DC-Motor Electric Drive Control System with Varia...IDES Editor
 
Project-Mohit Suri
Project-Mohit Suri Project-Mohit Suri
Project-Mohit Suri Mohit Suri
 
Kamalasadan presentation02
Kamalasadan presentation02Kamalasadan presentation02
Kamalasadan presentation02Vamsi Kattamuri
 
IRJET- Hybrid Intelligent Controller for Interior Permanent Magnet Synchr...
IRJET-  	  Hybrid Intelligent Controller for Interior Permanent Magnet Synchr...IRJET-  	  Hybrid Intelligent Controller for Interior Permanent Magnet Synchr...
IRJET- Hybrid Intelligent Controller for Interior Permanent Magnet Synchr...IRJET Journal
 
Intelligent swarm algorithms for optimizing nonlinear sliding mode controller...
Intelligent swarm algorithms for optimizing nonlinear sliding mode controller...Intelligent swarm algorithms for optimizing nonlinear sliding mode controller...
Intelligent swarm algorithms for optimizing nonlinear sliding mode controller...IJECEIAES
 
Non-Linear Controller for Electric Vehicle based on Indian Road Conditions
Non-Linear Controller for Electric Vehicle based on Indian Road ConditionsNon-Linear Controller for Electric Vehicle based on Indian Road Conditions
Non-Linear Controller for Electric Vehicle based on Indian Road ConditionsIDES Editor
 
dokumen.tips_waterfall-model-in-software-engineering.pptx
dokumen.tips_waterfall-model-in-software-engineering.pptxdokumen.tips_waterfall-model-in-software-engineering.pptx
dokumen.tips_waterfall-model-in-software-engineering.pptxRudranilDas11
 
Control System’s Coursework Project Overview
Control System’s Coursework Project OverviewControl System’s Coursework Project Overview
Control System’s Coursework Project OverviewSeth Block
 
Feasibility Study and Design of Electromagnetic Suspension Systems: A Review
Feasibility Study and Design of Electromagnetic Suspension Systems: A ReviewFeasibility Study and Design of Electromagnetic Suspension Systems: A Review
Feasibility Study and Design of Electromagnetic Suspension Systems: A ReviewIRJET Journal
 
Optimal Design of Super Twisting Control with PSO Algorithm for Robotic Manip...
Optimal Design of Super Twisting Control with PSO Algorithm for Robotic Manip...Optimal Design of Super Twisting Control with PSO Algorithm for Robotic Manip...
Optimal Design of Super Twisting Control with PSO Algorithm for Robotic Manip...CSCJournals
 
Speed_Control_of_DC_Motor_Using_Particle_Swarm_Opt.pdf
Speed_Control_of_DC_Motor_Using_Particle_Swarm_Opt.pdfSpeed_Control_of_DC_Motor_Using_Particle_Swarm_Opt.pdf
Speed_Control_of_DC_Motor_Using_Particle_Swarm_Opt.pdfThienMai14
 
An Algorithm Based Simulation Modeling For Control of Production Systems
An Algorithm Based Simulation Modeling For Control of Production SystemsAn Algorithm Based Simulation Modeling For Control of Production Systems
An Algorithm Based Simulation Modeling For Control of Production SystemsIJMER
 
Design and analysis of compensator for missile
Design and analysis of compensator for missileDesign and analysis of compensator for missile
Design and analysis of compensator for missile178r1a04l3
 

Similar to Morphing UAV Intelligent Shape and Flight Control (20)

SE18_Lec 03_ RUP
SE18_Lec 03_ RUPSE18_Lec 03_ RUP
SE18_Lec 03_ RUP
 
Lec 03(VDIdr shady)
Lec 03(VDIdr shady)Lec 03(VDIdr shady)
Lec 03(VDIdr shady)
 
Gh3112071212
Gh3112071212Gh3112071212
Gh3112071212
 
Aircraft pitch control design using LQG controller based on genetic algorithm
Aircraft pitch control design using LQG controller based on genetic algorithmAircraft pitch control design using LQG controller based on genetic algorithm
Aircraft pitch control design using LQG controller based on genetic algorithm
 
Design of imc based controller for industrial purpose
Design of imc based controller for industrial purposeDesign of imc based controller for industrial purpose
Design of imc based controller for industrial purpose
 
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
 
Modelling and Simulation of DC-Motor Electric Drive Control System with Varia...
Modelling and Simulation of DC-Motor Electric Drive Control System with Varia...Modelling and Simulation of DC-Motor Electric Drive Control System with Varia...
Modelling and Simulation of DC-Motor Electric Drive Control System with Varia...
 
Project-Mohit Suri
Project-Mohit Suri Project-Mohit Suri
Project-Mohit Suri
 
Kamalasadan presentation02
Kamalasadan presentation02Kamalasadan presentation02
Kamalasadan presentation02
 
IRJET- Hybrid Intelligent Controller for Interior Permanent Magnet Synchr...
IRJET-  	  Hybrid Intelligent Controller for Interior Permanent Magnet Synchr...IRJET-  	  Hybrid Intelligent Controller for Interior Permanent Magnet Synchr...
IRJET- Hybrid Intelligent Controller for Interior Permanent Magnet Synchr...
 
Intelligent swarm algorithms for optimizing nonlinear sliding mode controller...
Intelligent swarm algorithms for optimizing nonlinear sliding mode controller...Intelligent swarm algorithms for optimizing nonlinear sliding mode controller...
Intelligent swarm algorithms for optimizing nonlinear sliding mode controller...
 
Non-Linear Controller for Electric Vehicle based on Indian Road Conditions
Non-Linear Controller for Electric Vehicle based on Indian Road ConditionsNon-Linear Controller for Electric Vehicle based on Indian Road Conditions
Non-Linear Controller for Electric Vehicle based on Indian Road Conditions
 
dokumen.tips_waterfall-model-in-software-engineering.pptx
dokumen.tips_waterfall-model-in-software-engineering.pptxdokumen.tips_waterfall-model-in-software-engineering.pptx
dokumen.tips_waterfall-model-in-software-engineering.pptx
 
Control System’s Coursework Project Overview
Control System’s Coursework Project OverviewControl System’s Coursework Project Overview
Control System’s Coursework Project Overview
 
Feasibility Study and Design of Electromagnetic Suspension Systems: A Review
Feasibility Study and Design of Electromagnetic Suspension Systems: A ReviewFeasibility Study and Design of Electromagnetic Suspension Systems: A Review
Feasibility Study and Design of Electromagnetic Suspension Systems: A Review
 
Optimal Design of Super Twisting Control with PSO Algorithm for Robotic Manip...
Optimal Design of Super Twisting Control with PSO Algorithm for Robotic Manip...Optimal Design of Super Twisting Control with PSO Algorithm for Robotic Manip...
Optimal Design of Super Twisting Control with PSO Algorithm for Robotic Manip...
 
Thesis1
Thesis1Thesis1
Thesis1
 
Speed_Control_of_DC_Motor_Using_Particle_Swarm_Opt.pdf
Speed_Control_of_DC_Motor_Using_Particle_Swarm_Opt.pdfSpeed_Control_of_DC_Motor_Using_Particle_Swarm_Opt.pdf
Speed_Control_of_DC_Motor_Using_Particle_Swarm_Opt.pdf
 
An Algorithm Based Simulation Modeling For Control of Production Systems
An Algorithm Based Simulation Modeling For Control of Production SystemsAn Algorithm Based Simulation Modeling For Control of Production Systems
An Algorithm Based Simulation Modeling For Control of Production Systems
 
Design and analysis of compensator for missile
Design and analysis of compensator for missileDesign and analysis of compensator for missile
Design and analysis of compensator for missile
 

Recently uploaded

IMPLICATIONS OF THE ABOVE HOLISTIC UNDERSTANDING OF HARMONY ON PROFESSIONAL E...
IMPLICATIONS OF THE ABOVE HOLISTIC UNDERSTANDING OF HARMONY ON PROFESSIONAL E...IMPLICATIONS OF THE ABOVE HOLISTIC UNDERSTANDING OF HARMONY ON PROFESSIONAL E...
IMPLICATIONS OF THE ABOVE HOLISTIC UNDERSTANDING OF HARMONY ON PROFESSIONAL E...RajaP95
 
result management system report for college project
result management system report for college projectresult management system report for college project
result management system report for college projectTonystark477637
 
Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...
Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...
Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...Dr.Costas Sachpazis
 
UNIT-II FMM-Flow Through Circular Conduits
UNIT-II FMM-Flow Through Circular ConduitsUNIT-II FMM-Flow Through Circular Conduits
UNIT-II FMM-Flow Through Circular Conduitsrknatarajan
 
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...ranjana rawat
 
MANUFACTURING PROCESS-II UNIT-5 NC MACHINE TOOLS
MANUFACTURING PROCESS-II UNIT-5 NC MACHINE TOOLSMANUFACTURING PROCESS-II UNIT-5 NC MACHINE TOOLS
MANUFACTURING PROCESS-II UNIT-5 NC MACHINE TOOLSSIVASHANKAR N
 
MANUFACTURING PROCESS-II UNIT-2 LATHE MACHINE
MANUFACTURING PROCESS-II UNIT-2 LATHE MACHINEMANUFACTURING PROCESS-II UNIT-2 LATHE MACHINE
MANUFACTURING PROCESS-II UNIT-2 LATHE MACHINESIVASHANKAR N
 
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130Suhani Kapoor
 
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur EscortsHigh Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escortsranjana rawat
 
Porous Ceramics seminar and technical writing
Porous Ceramics seminar and technical writingPorous Ceramics seminar and technical writing
Porous Ceramics seminar and technical writingrakeshbaidya232001
 
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...Christo Ananth
 
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICSHARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICSRajkumarAkumalla
 
Software Development Life Cycle By Team Orange (Dept. of Pharmacy)
Software Development Life Cycle By  Team Orange (Dept. of Pharmacy)Software Development Life Cycle By  Team Orange (Dept. of Pharmacy)
Software Development Life Cycle By Team Orange (Dept. of Pharmacy)Suman Mia
 
(ANVI) Koregaon Park Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANVI) Koregaon Park Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(ANVI) Koregaon Park Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANVI) Koregaon Park Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...ranjana rawat
 
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...Soham Mondal
 
Call Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service Nashik
Call Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service NashikCall Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service Nashik
Call Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service NashikCall Girls in Nagpur High Profile
 
The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...
The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...
The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...ranjana rawat
 
Introduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxIntroduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxupamatechverse
 
Coefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptxCoefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptxAsutosh Ranjan
 

Recently uploaded (20)

IMPLICATIONS OF THE ABOVE HOLISTIC UNDERSTANDING OF HARMONY ON PROFESSIONAL E...
IMPLICATIONS OF THE ABOVE HOLISTIC UNDERSTANDING OF HARMONY ON PROFESSIONAL E...IMPLICATIONS OF THE ABOVE HOLISTIC UNDERSTANDING OF HARMONY ON PROFESSIONAL E...
IMPLICATIONS OF THE ABOVE HOLISTIC UNDERSTANDING OF HARMONY ON PROFESSIONAL E...
 
result management system report for college project
result management system report for college projectresult management system report for college project
result management system report for college project
 
Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...
Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...
Structural Analysis and Design of Foundations: A Comprehensive Handbook for S...
 
UNIT-II FMM-Flow Through Circular Conduits
UNIT-II FMM-Flow Through Circular ConduitsUNIT-II FMM-Flow Through Circular Conduits
UNIT-II FMM-Flow Through Circular Conduits
 
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANJALI) Dange Chowk Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
 
MANUFACTURING PROCESS-II UNIT-5 NC MACHINE TOOLS
MANUFACTURING PROCESS-II UNIT-5 NC MACHINE TOOLSMANUFACTURING PROCESS-II UNIT-5 NC MACHINE TOOLS
MANUFACTURING PROCESS-II UNIT-5 NC MACHINE TOOLS
 
MANUFACTURING PROCESS-II UNIT-2 LATHE MACHINE
MANUFACTURING PROCESS-II UNIT-2 LATHE MACHINEMANUFACTURING PROCESS-II UNIT-2 LATHE MACHINE
MANUFACTURING PROCESS-II UNIT-2 LATHE MACHINE
 
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
VIP Call Girls Service Hitech City Hyderabad Call +91-8250192130
 
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur EscortsHigh Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
High Profile Call Girls Nagpur Isha Call 7001035870 Meet With Nagpur Escorts
 
★ CALL US 9953330565 ( HOT Young Call Girls In Badarpur delhi NCR
★ CALL US 9953330565 ( HOT Young Call Girls In Badarpur delhi NCR★ CALL US 9953330565 ( HOT Young Call Girls In Badarpur delhi NCR
★ CALL US 9953330565 ( HOT Young Call Girls In Badarpur delhi NCR
 
Porous Ceramics seminar and technical writing
Porous Ceramics seminar and technical writingPorous Ceramics seminar and technical writing
Porous Ceramics seminar and technical writing
 
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
Call for Papers - African Journal of Biological Sciences, E-ISSN: 2663-2187, ...
 
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICSHARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
HARDNESS, FRACTURE TOUGHNESS AND STRENGTH OF CERAMICS
 
Software Development Life Cycle By Team Orange (Dept. of Pharmacy)
Software Development Life Cycle By  Team Orange (Dept. of Pharmacy)Software Development Life Cycle By  Team Orange (Dept. of Pharmacy)
Software Development Life Cycle By Team Orange (Dept. of Pharmacy)
 
(ANVI) Koregaon Park Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANVI) Koregaon Park Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...(ANVI) Koregaon Park Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
(ANVI) Koregaon Park Call Girls Just Call 7001035870 [ Cash on Delivery ] Pun...
 
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
OSVC_Meta-Data based Simulation Automation to overcome Verification Challenge...
 
Call Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service Nashik
Call Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service NashikCall Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service Nashik
Call Girls Service Nashik Vaishnavi 7001305949 Independent Escort Service Nashik
 
The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...
The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...
The Most Attractive Pune Call Girls Budhwar Peth 8250192130 Will You Miss Thi...
 
Introduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptxIntroduction to IEEE STANDARDS and its different types.pptx
Introduction to IEEE STANDARDS and its different types.pptx
 
Coefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptxCoefficient of Thermal Expansion and their Importance.pptx
Coefficient of Thermal Expansion and their Importance.pptx
 

Morphing UAV Intelligent Shape and Flight Control

  • 1. Morphing UAV Intelligent Shape and Flight Control - I Kumar Digvijay Mishra Control of Morphing Aerostructures kumardigvijaymishra@yahoo.co.uk January 11, 2021 Kumar Digvijay Mishra January 11, 2021 1 / 69
  • 2. Overview 1 Introduction 2 A-RLC Architecture Functionality 3 Learning Air Vehicle Shape Changes Overview of RL Implementation of Shape Change Learning Agent 4 Mathematical Modeling of Morphing Air Vehicle Aerodynamic Modeling Constitutive Equations Model Grid Dynamical Modeling Reference Trajectory Shape Memory Alloy Actuator Dynamics Control Effectors on Morphing Wing 5 Morphing Control Law 6 Numerical Examples 7 Conclusion Kumar Digvijay Mishra January 11, 2021 2 / 69
  • 3. Introduction Definition Morphing refers to both small-scale and large-scale shape changes or transconfigurations Two classes of morphing in context of flight vehicles: 1 Morphing for Control A small-scale or component level, in-flight physical or virtual shape change, used to achieve multiple control objectives such as noise suppression, flutter suppression, load alleviation and active flow separation control Kumar Digvijay Mishra 1. Introduction January 11, 2021 3 / 69
  • 4. Introduction Definition Morphing refers to both small-scale and large-scale shape changes or transconfigurations Two classes of morphing in context of flight vehicles: 1 Morphing for Control A small-scale or component level, in-flight physical or virtual shape change, used to achieve multiple control objectives such as noise suppression, flutter suppression, load alleviation and active flow separation control 2 Morphing for Mission Adaptation (MMA) A large-scale, relatively slow, in-flight shape change to enable a single vehicle to perform multiple diverse mission profiles Kumar Digvijay Mishra 1. Introduction January 11, 2021 3 / 69
  • 5. Introduction Essential functionalities of practical MMA capability are: 1 When to reconfigure: Driven by mission priorities/tasks, and leads to optimal shape being a system parameter Kumar Digvijay Mishra 1. Introduction January 11, 2021 4 / 69
  • 6. Introduction Essential functionalities of practical MMA capability are: 1 When to reconfigure: Driven by mission priorities/tasks, and leads to optimal shape being a system parameter 2 How to reconfigure: Problem of sensing, actuation, and control Kumar Digvijay Mishra 1. Introduction January 11, 2021 4 / 69
  • 7. Introduction Essential functionalities of practical MMA capability are: 1 When to reconfigure: Driven by mission priorities/tasks, and leads to optimal shape being a system parameter 2 How to reconfigure: Problem of sensing, actuation, and control 3 Learning to reconfigure: Actuation scheme(s) to produce most optimal shape Kumar Digvijay Mishra 1. Introduction January 11, 2021 4 / 69
  • 8. Introduction Previous control design research1: ◦ Vortex lattice method used to calculate aerodynamics for the case of variable wing dihedral2 ◦ Desired dynamics chosen for each mission phase ◦ Developed H∞ model-following controllers 1 Boothe et al. (2005) and Abdulrahim and Lind(2005) 2 inclination of aircraft’s wing from horizontal, esp upwards, away from fuselage Kumar Digvijay Mishra 1. Introduction January 11, 2021 5 / 69
  • 9. Introduction Previous control design research1: ◦ Vortex lattice method used to calculate aerodynamics for the case of variable wing dihedral2 ◦ Desired dynamics chosen for each mission phase ◦ Developed H∞ model-following controllers Current control design approach: ◦ High-level description of functionality of A-RLC architecture ◦ Development of learning agent, air vehicle model and simulation, smart actuators, and adaptive controller 1 Boothe et al. (2005) and Abdulrahim and Lind(2005) 2 inclination of aircraft’s wing from horizontal, esp upwards, away from fuselage Kumar Digvijay Mishra 1. Introduction January 11, 2021 5 / 69
  • 10. A-RLC Architecture Functionality Adaptive-Reinforcement Learning Control composed of: 1 Reinforcement Learning (RL) Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 6 / 69
  • 11. A-RLC Architecture Functionality Adaptive-Reinforcement Learning Control composed of: 1 Reinforcement Learning (RL) 2 Structured Adaptive Model Inversion (SAMI) Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 6 / 69
  • 12. A-RLC Architecture Functionality Adaptive-Reinforcement Learning Control composed of: 1 Reinforcement Learning (RL) 2 Structured Adaptive Model Inversion (SAMI) Both sub-systems interact significantly during: Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 6 / 69
  • 13. A-RLC Architecture Functionality Adaptive-Reinforcement Learning Control composed of: 1 Reinforcement Learning (RL) 2 Structured Adaptive Model Inversion (SAMI) Both sub-systems interact significantly during: 1 Episodic learning stage: when optimal shape change policy is learned Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 6 / 69
  • 14. A-RLC Architecture Functionality Adaptive-Reinforcement Learning Control composed of: 1 Reinforcement Learning (RL) 2 Structured Adaptive Model Inversion (SAMI) Both sub-systems interact significantly during: 1 Episodic learning stage: when optimal shape change policy is learned 2 Operational stage: when the plant morphs, and tracks a trajectory Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 6 / 69
  • 15. A-RLC Architecture Functionality Adaptive-Reinforcement Learning Control composed of: 1 Reinforcement Learning (RL) 2 Structured Adaptive Model Inversion (SAMI) Both sub-systems interact significantly during: 1 Episodic learning stage: when optimal shape change policy is learned 2 Operational stage: when the plant morphs, and tracks a trajectory A-RLC addresses optimal shape changing of air vehicle, rather than isolated components or individual actuation. Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 6 / 69
  • 16. A-RLC Architecture Functionality Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 7 / 69
  • 17. RL Module of A-RLC Architecture 1 Initially commands an arbitrary action from the set of admissible actions Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 8 / 69
  • 18. RL Module of A-RLC Architecture 1 Initially commands an arbitrary action from the set of admissible actions 2 This action is sent to the plant, which produces a shape change Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 8 / 69
  • 19. RL Module of A-RLC Architecture 1 Initially commands an arbitrary action from the set of admissible actions 2 This action is sent to the plant, which produces a shape change 3 Cost associated with resultant shape change in terms of system states, parameters, and user-defined performance measures is evaluated with the cost function and then passed to the agent Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 8 / 69
  • 20. RL Module of A-RLC Architecture 1 Initially commands an arbitrary action from the set of admissible actions 2 This action is sent to the plant, which produces a shape change 3 Cost associated with resultant shape change in terms of system states, parameters, and user-defined performance measures is evaluated with the cost function and then passed to the agent 4 Agent modifies its action-value function based on Q-learning algorithm Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 8 / 69
  • 21. RL Module of A-RLC Architecture 1 Initially commands an arbitrary action from the set of admissible actions 2 This action is sent to the plant, which produces a shape change 3 Cost associated with resultant shape change in terms of system states, parameters, and user-defined performance measures is evaluated with the cost function and then passed to the agent 4 Agent modifies its action-value function based on Q-learning algorithm 5 For the next episode, the agent chooses a new action based on the current policy and its updated action-value function, and the sequence repeats itself Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 8 / 69
  • 22. RL Module of A-RLC Architecture 1 Initially commands an arbitrary action from the set of admissible actions 2 This action is sent to the plant, which produces a shape change 3 Cost associated with resultant shape change in terms of system states, parameters, and user-defined performance measures is evaluated with the cost function and then passed to the agent 4 Agent modifies its action-value function based on Q-learning algorithm 5 For the next episode, the agent chooses a new action based on the current policy and its updated action-value function, and the sequence repeats itself 6 RL sub-system improved by applying Sequential Function Approximation to generalize learning from previously experienced quantized states and actions to the continuous state-action space. Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 8 / 69
  • 23. SAMI Module of A-RLC Architecture 1 Shape changes in the plant due to actions generated by RL agent cause plant dynamics to change Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 9 / 69
  • 24. SAMI Module of A-RLC Architecture 1 Shape changes in the plant due to actions generated by RL agent cause plant dynamics to change 2 By calculating commanded moments, SAMI controller tracks a reference trajectory irrespective of the changing dynamics of the vehicle caused by these shape changes Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 9 / 69
  • 25. SAMI Module of A-RLC Architecture 1 Shape changes in the plant due to actions generated by RL agent cause plant dynamics to change 2 By calculating commanded moments, SAMI controller tracks a reference trajectory irrespective of the changing dynamics of the vehicle caused by these shape changes 3 SAMI is a non-linear technique based on concepts of Feedback Linearization, Dynamic Inversion, and Structured Model Reference Adaptive Control Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 9 / 69
  • 26. SAMI Module of A-RLC Architecture 1 Shape changes in the plant due to actions generated by RL agent cause plant dynamics to change 2 By calculating commanded moments, SAMI controller tracks a reference trajectory irrespective of the changing dynamics of the vehicle caused by these shape changes 3 SAMI is a non-linear technique based on concepts of Feedback Linearization, Dynamic Inversion, and Structured Model Reference Adaptive Control 4 Dynamic Inversion is approx since system parameters are inaccurately modeled Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 9 / 69
  • 27. SAMI Module of A-RLC Architecture 1 Shape changes in the plant due to actions generated by RL agent cause plant dynamics to change 2 By calculating commanded moments, SAMI controller tracks a reference trajectory irrespective of the changing dynamics of the vehicle caused by these shape changes 3 SAMI is a non-linear technique based on concepts of Feedback Linearization, Dynamic Inversion, and Structured Model Reference Adaptive Control 4 Dynamic Inversion is approx since system parameters are inaccurately modeled 5 Adaptive control structure is wrapped around the dynamic inverter to account for uncertainties in system parameters Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 9 / 69
  • 28. SAMI Module of A-RLC Architecture 1 Shape changes in the plant due to actions generated by RL agent cause plant dynamics to change 2 By calculating commanded moments, SAMI controller tracks a reference trajectory irrespective of the changing dynamics of the vehicle caused by these shape changes 3 SAMI is a non-linear technique based on concepts of Feedback Linearization, Dynamic Inversion, and Structured Model Reference Adaptive Control 4 Dynamic Inversion is approx since system parameters are inaccurately modeled 5 Adaptive control structure is wrapped around the dynamic inverter to account for uncertainties in system parameters 6 Applications: tracking reference trajectories of spacecraft, planetary entry vehicles, and aircraft; handling of actuator failures; facilitation of correct adaptation in presence of actuator saturation Kumar Digvijay Mishra 2. A-RLC Architecture Functionality January 11, 2021 9 / 69
  • 29. 3.1 Overview of Reinforcement Learning Definition RL is a method of learning from interaction between an agent and its environment to achieve a specified goal. Kumar Digvijay Mishra 3. Learning Air Vehicle Shape Changes January 11, 2021 10 / 69
  • 30. 3.1 Overview of Reinforcement Learning Definition RL is a method of learning from interaction between an agent and its environment to achieve a specified goal. ◦ Agent is the learner and decision-maker with the goal of maximizing total reward it receives over the long run. Kumar Digvijay Mishra 3. Learning Air Vehicle Shape Changes January 11, 2021 10 / 69
  • 31. 3.1 Overview of Reinforcement Learning Definition RL is a method of learning from interaction between an agent and its environment to achieve a specified goal. ◦ Agent is the learner and decision-maker with the goal of maximizing total reward it receives over the long run. ◦ Environment is everything outside the agent, that it interacts with. Kumar Digvijay Mishra 3. Learning Air Vehicle Shape Changes January 11, 2021 10 / 69
  • 32. 3.1 Overview of Reinforcement Learning Definition RL is a method of learning from interaction between an agent and its environment to achieve a specified goal. ◦ Agent is the learner and decision-maker with the goal of maximizing total reward it receives over the long run. ◦ Environment is everything outside the agent, that it interacts with. ◦ Agent’s Policy is the mapping from states to probabilities of selecting a specified possible action at a time step Kumar Digvijay Mishra 3. Learning Air Vehicle Shape Changes January 11, 2021 10 / 69
  • 33. Overview of Reinforcement Learning ◦ Agent interacts with its environment at discrete time steps, t = 0, 1, 2, 3.... Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 11 / 69
  • 34. Overview of Reinforcement Learning ◦ Agent interacts with its environment at discrete time steps, t = 0, 1, 2, 3.... ◦ At each time step t, agent receives some representation of environment’s state, st ∈ S, where S is a set of possible states, and on that basis it selects an action, at ∈ A(st), where A(st) is a set of actions available in state st. Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 11 / 69
  • 35. Overview of Reinforcement Learning ◦ Agent interacts with its environment at discrete time steps, t = 0, 1, 2, 3.... ◦ At each time step t, agent receives some representation of environment’s state, st ∈ S, where S is a set of possible states, and on that basis it selects an action, at ∈ A(st), where A(st) is a set of actions available in state st. ◦ One time step later, as a result of agent’s action, it receives a numerical reward, rt+1 = R, and finds itself in a new state, st+1. Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 11 / 69
  • 36. Overview of Reinforcement Learning ◦ Agent interacts with its environment at discrete time steps, t = 0, 1, 2, 3.... ◦ At each time step t, agent receives some representation of environment’s state, st ∈ S, where S is a set of possible states, and on that basis it selects an action, at ∈ A(st), where A(st) is a set of actions available in state st. ◦ One time step later, as a result of agent’s action, it receives a numerical reward, rt+1 = R, and finds itself in a new state, st+1. ◦ Here agent’s policy πt(s, a) indicates the probability that agent chose action at = a for the environment in given state of st = s at time t. Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 11 / 69
  • 37. Overview of Reinforcement Learning RL algos are based on estimating value functions for a policy π: State-value function V π(s) estimates how good it is, under policy π, for the agent to be in state s. Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 12 / 69
  • 38. Overview of Reinforcement Learning RL algos are based on estimating value functions for a policy π: State-value function V π(s) estimates how good it is, under policy π, for the agent to be in state s. ◦ Defined as the expected return starting from s and thereafter following π V π (st) ≡ E[ ∞ X k=0 γk rt+k] (1) where γ is discount factor, rt+k is sequence of rewards Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 12 / 69
  • 39. Overview of Reinforcement Learning RL algos are based on estimating value functions for a policy π: State-value function V π(s) estimates how good it is, under policy π, for the agent to be in state s. ◦ Defined as the expected return starting from s and thereafter following π V π (st) ≡ E[ ∞ X k=0 γk rt+k] (1) where γ is discount factor, rt+k is sequence of rewards Action-value function Qπ(s, a) estimates how good it is, under policy π, for the agent to perform action a in state s. Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 12 / 69
  • 40. Overview of Reinforcement Learning RL algos are based on estimating value functions for a policy π: State-value function V π(s) estimates how good it is, under policy π, for the agent to be in state s. ◦ Defined as the expected return starting from s and thereafter following π V π (st) ≡ E[ ∞ X k=0 γk rt+k] (1) where γ is discount factor, rt+k is sequence of rewards Action-value function Qπ(s, a) estimates how good it is, under policy π, for the agent to perform action a in state s. ◦ Defined as the expected return starting from s, taking action a, and thereafter following policy π Qπ (s, a) ≡ r(s, a) + γV ∗ (δ(s, a)) (2) where δ(s, a) denotes the state resulting from applying action a to state s & V ∗(δ(s, a)) is state-value function for optimal policy π∗ Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 12 / 69
  • 41. Overview of Reinforcement Learning ◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a) Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
  • 42. Overview of Reinforcement Learning ◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a) ◦ Policy Improvement: For a state s, an action can be chosen from all possible actions to improve π to π‘ s.t. V π(s) or Qπ(s, a) are maximized Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
  • 43. Overview of Reinforcement Learning ◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a) ◦ Policy Improvement: For a state s, an action can be chosen from all possible actions to improve π to π‘ s.t. V π(s) or Qπ(s, a) are maximized Evaluate new V π‘ (s) or Qπ‘ (s, a) for improved policy π‘ Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
  • 44. Overview of Reinforcement Learning ◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a) ◦ Policy Improvement: For a state s, an action can be chosen from all possible actions to improve π to π‘ s.t. V π(s) or Qπ(s, a) are maximized Evaluate new V π‘ (s) or Qπ‘ (s, a) for improved policy π‘ Policy π‘ can be further improved to π“ Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
  • 45. Overview of Reinforcement Learning ◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a) ◦ Policy Improvement: For a state s, an action can be chosen from all possible actions to improve π to π‘ s.t. V π(s) or Qπ(s, a) are maximized Evaluate new V π‘ (s) or Qπ‘ (s, a) for improved policy π‘ Policy π‘ can be further improved to π“ ◦ Optimal Policy: A policy π∗ that has : Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
  • 46. Overview of Reinforcement Learning ◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a) ◦ Policy Improvement: For a state s, an action can be chosen from all possible actions to improve π to π‘ s.t. V π(s) or Qπ(s, a) are maximized Evaluate new V π‘ (s) or Qπ‘ (s, a) for improved policy π‘ Policy π‘ can be further improved to π“ ◦ Optimal Policy: A policy π∗ that has : Either an optimal state-value function V ∗ (s) = maxπV π (s) Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
  • 47. Overview of Reinforcement Learning ◦ Policy Evaluation: Process of computing V π(s) or Qπ(s, a) ◦ Policy Improvement: For a state s, an action can be chosen from all possible actions to improve π to π‘ s.t. V π(s) or Qπ(s, a) are maximized Evaluate new V π‘ (s) or Qπ‘ (s, a) for improved policy π‘ Policy π‘ can be further improved to π“ ◦ Optimal Policy: A policy π∗ that has : Either an optimal state-value function V ∗ (s) = maxπV π (s) Or an optimal action-value function Q∗ (s, a) = maxπQπ (s, a) Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 13 / 69
  • 48. Overview of Reinforcement Learning Goal of RL is to find optimal policy π∗ by policy iteration ◦ Optimal action-value function Q∗ in terms of optimal state-value function V ∗ : Q∗ (s, a) ≡ E[r(s, a) + γV ∗ (δ(s, a))] ≡ E[r(s, a)] + E[γV ∗ (δ(s, a))] ≡ E[r(s, a)] + γ X s0 P(s 0 |s, a)V ∗ (s 0 ) (3) where P(s‘|s, a) is probability of taking action a in state s to produce next state s‘ Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 14 / 69
  • 49. Overview of Reinforcement Learning Q∗ (s, a) ≡ E[r(s, a)] + γ X s0 P(s 0 |s, a)V ∗ (s 0 ) Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 15 / 69
  • 50. Overview of Reinforcement Learning Q∗ (s, a) ≡ E[r(s, a)] + γ X s0 P(s 0 |s, a)V ∗ (s 0 ) Optimal state-value function V ∗(s‘) is action-value function Q(s‘, a‘) maximized for action a‘ Q∗ (s, a) ≡ E[r(s, a)] + γ X s0 P(s 0 |s, a) max a0 Q(s 0 , a 0 ) (4) Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 15 / 69
  • 51. Overview of Reinforcement Learning Q∗ (s, a) ≡ E[r(s, a)] + γ X s0 P(s 0 |s, a)V ∗ (s 0 ) Optimal state-value function V ∗(s‘) is action-value function Q(s‘, a‘) maximized for action a‘ Q∗ (s, a) ≡ E[r(s, a)] + γ X s0 P(s 0 |s, a) max a0 Q(s 0 , a 0 ) (4) Iterative update of each action-value function Q(s,a) to converge to optimal Q∗(s, a) gives following training rule Qn(s, a) ← (1 − α)Qn−1(s, a) + α[r + γ max a0 Qn−1(s 0 , a 0 )] (5) Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 15 / 69
  • 52. Overview of Reinforcement Learning Q∗ (s, a) ≡ E[r(s, a)] + γ X s0 P(s 0 |s, a)V ∗ (s 0 ) Optimal state-value function V ∗(s‘) is action-value function Q(s‘, a‘) maximized for action a‘ Q∗ (s, a) ≡ E[r(s, a)] + γ X s0 P(s 0 |s, a) max a0 Q(s 0 , a 0 ) (4) Iterative update of each action-value function Q(s,a) to converge to optimal Q∗(s, a) gives following training rule Qn(s, a) ← (1 − α)Qn−1(s, a) + α[r + γ max a0 Qn−1(s 0 , a 0 )] (5) For Q-learning algorithm, Q(s, a) ← Q(s, a) + α[r + γ max a0 Q(s 0 , a 0 ) − Q(s, a)] (6) Kumar Digvijay Mishra 3.1 Learning Air Vehicle Shape Changes January 11, 2021 15 / 69
  • 53. Policy Iteration Major methods for PI: 1 Dynamic Programming Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 16 / 69
  • 54. Policy Iteration Major methods for PI: 1 Dynamic Programming 2 Monte Carlo Method Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 16 / 69
  • 55. Policy Iteration Major methods for PI: 1 Dynamic Programming 2 Monte Carlo Method 3 Temporal-Difference Learning Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 16 / 69
  • 56. Policy Iteration Major methods for PI: 1 Dynamic Programming 2 Monte Carlo Method 3 Temporal-Difference Learning 4 Q-Learning Technique Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 16 / 69
  • 57. Major Methods of Policy Iteration 1. Dynamic Programming Refers to a collection of algos to compute optimal policies given a perfect model of the environment as a Markov decision process Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 17 / 69
  • 58. Major Methods of Policy Iteration 1. Dynamic Programming Refers to a collection of algos to compute optimal policies given a perfect model of the environment as a Markov decision process Use value functions to organize and structure the search for good policies Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 17 / 69
  • 59. Major Methods of Policy Iteration 1. Dynamic Programming Refers to a collection of algos to compute optimal policies given a perfect model of the environment as a Markov decision process Use value functions to organize and structure the search for good policies Limited utility in RL because of Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 17 / 69
  • 60. Major Methods of Policy Iteration 1. Dynamic Programming Refers to a collection of algos to compute optimal policies given a perfect model of the environment as a Markov decision process Use value functions to organize and structure the search for good policies Limited utility in RL because of 1 Requires model of environment’s dynamics to learn optimal behavior Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 17 / 69
  • 61. Major Methods of Policy Iteration 1. Dynamic Programming Refers to a collection of algos to compute optimal policies given a perfect model of the environment as a Markov decision process Use value functions to organize and structure the search for good policies Limited utility in RL because of 1 Requires model of environment’s dynamics to learn optimal behavior 2 their assumption of a perfect model, and Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 17 / 69
  • 62. Major Methods of Policy Iteration 1. Dynamic Programming Refers to a collection of algos to compute optimal policies given a perfect model of the environment as a Markov decision process Use value functions to organize and structure the search for good policies Limited utility in RL because of 1 Requires model of environment’s dynamics to learn optimal behavior 2 their assumption of a perfect model, and 3 computational expense Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 17 / 69
  • 63. Major Methods of Policy Iteration 2. Monte Carlo Methods Estimates functions using an iterative, incremental procedure Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
  • 64. Major Methods of Policy Iteration 2. Monte Carlo Methods Estimates functions using an iterative, incremental procedure Solution based on averaging sample returns Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
  • 65. Major Methods of Policy Iteration 2. Monte Carlo Methods Estimates functions using an iterative, incremental procedure Solution based on averaging sample returns To ensure availability of well-defined returns, returns are defined only for episodic tasks Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
  • 66. Major Methods of Policy Iteration 2. Monte Carlo Methods Estimates functions using an iterative, incremental procedure Solution based on averaging sample returns To ensure availability of well-defined returns, returns are defined only for episodic tasks Upon completion of an episode, value estimates and policies are changed Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
  • 67. Major Methods of Policy Iteration 2. Monte Carlo Methods Estimates functions using an iterative, incremental procedure Solution based on averaging sample returns To ensure availability of well-defined returns, returns are defined only for episodic tasks Upon completion of an episode, value estimates and policies are changed Advantages: Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
  • 68. Major Methods of Policy Iteration 2. Monte Carlo Methods Estimates functions using an iterative, incremental procedure Solution based on averaging sample returns To ensure availability of well-defined returns, returns are defined only for episodic tasks Upon completion of an episode, value estimates and policies are changed Advantages: 1 Contrary to DP, MC methods can be used to learn optimal behavior directly from interaction with the environment with no model of environment’s dynamics Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
  • 69. Major Methods of Policy Iteration 2. Monte Carlo Methods Estimates functions using an iterative, incremental procedure Solution based on averaging sample returns To ensure availability of well-defined returns, returns are defined only for episodic tasks Upon completion of an episode, value estimates and policies are changed Advantages: 1 Contrary to DP, MC methods can be used to learn optimal behavior directly from interaction with the environment with no model of environment’s dynamics 2 Can be used with simulation Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
  • 70. Major Methods of Policy Iteration 2. Monte Carlo Methods Estimates functions using an iterative, incremental procedure Solution based on averaging sample returns To ensure availability of well-defined returns, returns are defined only for episodic tasks Upon completion of an episode, value estimates and policies are changed Advantages: 1 Contrary to DP, MC methods can be used to learn optimal behavior directly from interaction with the environment with no model of environment’s dynamics 2 Can be used with simulation 3 Easy and efficient to focus MC methods on a small subset of states Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
  • 71. Major Methods of Policy Iteration 2. Monte Carlo Methods Estimates functions using an iterative, incremental procedure Solution based on averaging sample returns To ensure availability of well-defined returns, returns are defined only for episodic tasks Upon completion of an episode, value estimates and policies are changed Advantages: 1 Contrary to DP, MC methods can be used to learn optimal behavior directly from interaction with the environment with no model of environment’s dynamics 2 Can be used with simulation 3 Easy and efficient to focus MC methods on a small subset of states Limitation: Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
  • 72. Major Methods of Policy Iteration 2. Monte Carlo Methods Estimates functions using an iterative, incremental procedure Solution based on averaging sample returns To ensure availability of well-defined returns, returns are defined only for episodic tasks Upon completion of an episode, value estimates and policies are changed Advantages: 1 Contrary to DP, MC methods can be used to learn optimal behavior directly from interaction with the environment with no model of environment’s dynamics 2 Can be used with simulation 3 Easy and efficient to focus MC methods on a small subset of states Limitation: Convergence properties not well-understood for RL Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 18 / 69
  • 73. Major Methods of Policy Iteration 3. Temporal-Difference Methods Sutton’s3method of TD is a form of policy evaluation method in DP which attempts to choose a control policy π0 3 Sutton (1988) 4 Williams and Baird 1993 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
  • 74. Major Methods of Policy Iteration 3. Temporal-Difference Methods Sutton’s3method of TD is a form of policy evaluation method in DP which attempts to choose a control policy π0 Expected discounted rewards, V π(i), for each state i in S are predicted using policy π0 to improve to new policy π1 3 Sutton (1988) 4 Williams and Baird 1993 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
  • 75. Major Methods of Policy Iteration 3. Temporal-Difference Methods Sutton’s3method of TD is a form of policy evaluation method in DP which attempts to choose a control policy π0 Expected discounted rewards, V π(i), for each state i in S are predicted using policy π0 to improve to new policy π1 Advantages: 3 Sutton (1988) 4 Williams and Baird 1993 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
  • 76. Major Methods of Policy Iteration 3. Temporal-Difference Methods Sutton’s3method of TD is a form of policy evaluation method in DP which attempts to choose a control policy π0 Expected discounted rewards, V π(i), for each state i in S are predicted using policy π0 to improve to new policy π1 Advantages: 1 Less computation 3 Sutton (1988) 4 Williams and Baird 1993 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
  • 77. Major Methods of Policy Iteration 3. Temporal-Difference Methods Sutton’s3method of TD is a form of policy evaluation method in DP which attempts to choose a control policy π0 Expected discounted rewards, V π(i), for each state i in S are predicted using policy π0 to improve to new policy π1 Advantages: 1 Less computation 2 No assumption of perfect model of environment 3 Sutton (1988) 4 Williams and Baird 1993 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
  • 78. Major Methods of Policy Iteration 3. Temporal-Difference Methods Sutton’s3method of TD is a form of policy evaluation method in DP which attempts to choose a control policy π0 Expected discounted rewards, V π(i), for each state i in S are predicted using policy π0 to improve to new policy π1 Advantages: 1 Less computation 2 No assumption of perfect model of environment 3 Howard’s4 algo converges to some policy 3 Sutton (1988) 4 Williams and Baird 1993 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
  • 79. Major Methods of Policy Iteration 3. Temporal-Difference Methods Sutton’s3method of TD is a form of policy evaluation method in DP which attempts to choose a control policy π0 Expected discounted rewards, V π(i), for each state i in S are predicted using policy π0 to improve to new policy π1 Advantages: 1 Less computation 2 No assumption of perfect model of environment 3 Howard’s4 algo converges to some policy Limitations: 3 Sutton (1988) 4 Williams and Baird 1993 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
  • 80. Major Methods of Policy Iteration 3. Temporal-Difference Methods Sutton’s3method of TD is a form of policy evaluation method in DP which attempts to choose a control policy π0 Expected discounted rewards, V π(i), for each state i in S are predicted using policy π0 to improve to new policy π1 Advantages: 1 Less computation 2 No assumption of perfect model of environment 3 Howard’s4 algo converges to some policy Limitations: 1 Can’t learn optimal functions directly. Why ? 3 Sutton (1988) 4 Williams and Baird 1993 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
  • 81. Major Methods of Policy Iteration 3. Temporal-Difference Methods Sutton’s3method of TD is a form of policy evaluation method in DP which attempts to choose a control policy π0 Expected discounted rewards, V π(i), for each state i in S are predicted using policy π0 to improve to new policy π1 Advantages: 1 Less computation 2 No assumption of perfect model of environment 3 Howard’s4 algo converges to some policy Limitations: 1 Can’t learn optimal functions directly. Why ? Because it fixes the policy and then determines corresponding value functions 3 Sutton (1988) 4 Williams and Baird 1993 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning contd... January 11, 2021 19 / 69
  • 82. Major Methods of Policy Iteration 4. Q-Learning Technique A form of successive approximation technique of DP 5 Watkins and Davan 1989 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
  • 83. Major Methods of Policy Iteration 4. Q-Learning Technique A form of successive approximation technique of DP Proposed and developed by Watkins5 5 Watkins and Davan 1989 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
  • 84. Major Methods of Policy Iteration 4. Q-Learning Technique A form of successive approximation technique of DP Proposed and developed by Watkins5 Learns optimal value functions directly, as opposed to fixing a policy and determining corresponding value functions, like TD 5 Watkins and Davan 1989 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
  • 85. Major Methods of Policy Iteration 4. Q-Learning Technique A form of successive approximation technique of DP Proposed and developed by Watkins5 Learns optimal value functions directly, as opposed to fixing a policy and determining corresponding value functions, like TD Advantages: 5 Watkins and Davan 1989 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
  • 86. Major Methods of Policy Iteration 4. Q-Learning Technique A form of successive approximation technique of DP Proposed and developed by Watkins5 Learns optimal value functions directly, as opposed to fixing a policy and determining corresponding value functions, like TD Advantages: 1 Learns optimal value functions directly 5 Watkins and Davan 1989 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
  • 87. Major Methods of Policy Iteration 4. Q-Learning Technique A form of successive approximation technique of DP Proposed and developed by Watkins5 Learns optimal value functions directly, as opposed to fixing a policy and determining corresponding value functions, like TD Advantages: 1 Learns optimal value functions directly 2 Avoids need to sweep over the state-action space 5 Watkins and Davan 1989 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
  • 88. Major Methods of Policy Iteration 4. Q-Learning Technique A form of successive approximation technique of DP Proposed and developed by Watkins5 Learns optimal value functions directly, as opposed to fixing a policy and determining corresponding value functions, like TD Advantages: 1 Learns optimal value functions directly 2 Avoids need to sweep over the state-action space 3 First provably convergent direct adaptive optimal control algo 5 Watkins and Davan 1989 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
  • 89. Major Methods of Policy Iteration 4. Q-Learning Technique A form of successive approximation technique of DP Proposed and developed by Watkins5 Learns optimal value functions directly, as opposed to fixing a policy and determining corresponding value functions, like TD Advantages: 1 Learns optimal value functions directly 2 Avoids need to sweep over the state-action space 3 First provably convergent direct adaptive optimal control algo 4 Less computations 5 Watkins and Davan 1989 Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 20 / 69
  • 90. Reinforcement Learning ◦ Has been applied to wide variety of physical control tasks, both real and simulated Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
  • 91. Reinforcement Learning ◦ Has been applied to wide variety of physical control tasks, both real and simulated ◦ When state-space is too large to enumerate value functions, function approximators are used including, Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
  • 92. Reinforcement Learning ◦ Has been applied to wide variety of physical control tasks, both real and simulated ◦ When state-space is too large to enumerate value functions, function approximators are used including, 1 Neural networks Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
  • 93. Reinforcement Learning ◦ Has been applied to wide variety of physical control tasks, both real and simulated ◦ When state-space is too large to enumerate value functions, function approximators are used including, 1 Neural networks 2 Clustering Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
  • 94. Reinforcement Learning ◦ Has been applied to wide variety of physical control tasks, both real and simulated ◦ When state-space is too large to enumerate value functions, function approximators are used including, 1 Neural networks 2 Clustering 3 Nearest-neighbor methods Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
  • 95. Reinforcement Learning ◦ Has been applied to wide variety of physical control tasks, both real and simulated ◦ When state-space is too large to enumerate value functions, function approximators are used including, 1 Neural networks 2 Clustering 3 Nearest-neighbor methods 4 Tile coding, and Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
  • 96. Reinforcement Learning ◦ Has been applied to wide variety of physical control tasks, both real and simulated ◦ When state-space is too large to enumerate value functions, function approximators are used including, 1 Neural networks 2 Clustering 3 Nearest-neighbor methods 4 Tile coding, and 5 Cerebellar model articulator controller Kumar Digvijay Mishra 3.1 Overview of Reinforcement Learning January 11, 2021 21 / 69
  • 97. 3.2 Implementation of Shape Change Learning Agent For morphing wing problem: RL agent independently maneuvers wing from some initial state to final state, characterized by aerodynamic properties of the wing Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
  • 98. 3.2 Implementation of Shape Change Learning Agent For morphing wing problem: RL agent independently maneuvers wing from some initial state to final state, characterized by aerodynamic properties of the wing Agent learns from its interaction with the environment to command a series of actions that changes the states of morphing wing’s thickness or camber toward an optimal shape - optimal policy Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
  • 99. 3.2 Implementation of Shape Change Learning Agent For morphing wing problem: RL agent independently maneuvers wing from some initial state to final state, characterized by aerodynamic properties of the wing Agent learns from its interaction with the environment to command a series of actions that changes the states of morphing wing’s thickness or camber toward an optimal shape - optimal policy Environment is the resulting aerodynamics to which the wing is subjected to Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
  • 100. 3.2 Implementation of Shape Change Learning Agent For morphing wing problem: RL agent independently maneuvers wing from some initial state to final state, characterized by aerodynamic properties of the wing Agent learns from its interaction with the environment to command a series of actions that changes the states of morphing wing’s thickness or camber toward an optimal shape - optimal policy Environment is the resulting aerodynamics to which the wing is subjected to RL agent has no prior knowledge of relationship between actions and thickness and camber of morphing wing Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
  • 101. 3.2 Implementation of Shape Change Learning Agent For morphing wing problem: RL agent independently maneuvers wing from some initial state to final state, characterized by aerodynamic properties of the wing Agent learns from its interaction with the environment to command a series of actions that changes the states of morphing wing’s thickness or camber toward an optimal shape - optimal policy Environment is the resulting aerodynamics to which the wing is subjected to RL agent has no prior knowledge of relationship between actions and thickness and camber of morphing wing RL agent knows all possible actions that can be applied Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
  • 102. 3.2 Implementation of Shape Change Learning Agent For morphing wing problem: RL agent independently maneuvers wing from some initial state to final state, characterized by aerodynamic properties of the wing Agent learns from its interaction with the environment to command a series of actions that changes the states of morphing wing’s thickness or camber toward an optimal shape - optimal policy Environment is the resulting aerodynamics to which the wing is subjected to RL agent has no prior knowledge of relationship between actions and thickness and camber of morphing wing RL agent knows all possible actions that can be applied RL agent has accurate, real-time information of morphing wing shape, present aerodynamics, and current reward provided by environment Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
  • 103. 3.2 Implementation of Shape Change Learning Agent For morphing wing problem: RL agent independently maneuvers wing from some initial state to final state, characterized by aerodynamic properties of the wing Agent learns from its interaction with the environment to command a series of actions that changes the states of morphing wing’s thickness or camber toward an optimal shape - optimal policy Environment is the resulting aerodynamics to which the wing is subjected to RL agent has no prior knowledge of relationship between actions and thickness and camber of morphing wing RL agent knows all possible actions that can be applied RL agent has accurate, real-time information of morphing wing shape, present aerodynamics, and current reward provided by environment ◦ Sensory info from SMA Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 22 / 69
  • 104. 3.2 Implementation of Shape Change Learning Agent RL agent uses a 1-step Q-learning method - an off-policy TD control algo Q(s, a) ← Q(s, a) + α[r + γ max a0 Q(s 0 , a 0 ) − Q(s, a)] Initialize Q(s,a) arbitrarily Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 23 / 69
  • 105. 3.2 Implementation of Shape Change Learning Agent RL agent uses a 1-step Q-learning method - an off-policy TD control algo Q(s, a) ← Q(s, a) + α[r + γ max a0 Q(s 0 , a 0 ) − Q(s, a)] Initialize Q(s,a) arbitrarily Repeat (for each episode) - Initialize s - Repeat (for each step of the episode) * Choose a from s using policy derived from Q(s,a) (e.g. -Greedy Policy) * Take action a, observe reward r, state s’ * Q(s, a) ← Q(s, a) + α[r + γ maxa0 Q(s 0 , a 0 ) − Q(s, a)] * s ← s’ - until s is terminal Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 23 / 69
  • 106. 3.2 Implementation of Shape Change Learning Agent RL agent uses a 1-step Q-learning method - an off-policy TD control algo Q(s, a) ← Q(s, a) + α[r + γ max a0 Q(s 0 , a 0 ) − Q(s, a)] Initialize Q(s,a) arbitrarily Repeat (for each episode) - Initialize s - Repeat (for each step of the episode) * Choose a from s using policy derived from Q(s,a) (e.g. -Greedy Policy) * Take action a, observe reward r, state s’ * Q(s, a) ← Q(s, a) + α[r + γ maxa0 Q(s 0 , a 0 ) − Q(s, a)] * s ← s’ - until s is terminal return Q(s,a) Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 23 / 69
  • 107. Implementation of Shape Change Learning Agent Q(s,a) can be updated using: 1 a random policy, with each action having the same probability of selection Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 24 / 69
  • 108. Implementation of Shape Change Learning Agent Q(s,a) can be updated using: 1 a random policy, with each action having the same probability of selection 2 an -greedy policy, where is a small value Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 24 / 69
  • 109. Implementation of Shape Change Learning Agent Q(s,a) can be updated using: 1 a random policy, with each action having the same probability of selection 2 an -greedy policy, where is a small value Action a with the maximum Q(s,a) is selected with probability 1 − , otherwise a random action is selected Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 24 / 69
  • 110. Implementation of Shape Change Learning Agent Q(s,a) can be updated using: 1 a random policy, with each action having the same probability of selection 2 an -greedy policy, where is a small value Action a with the maximum Q(s,a) is selected with probability 1 − , otherwise a random action is selected RL agent learns -greedy policy defined as: if (probability 1 − ) a = argmaxaQ(s, a) else a = rand(ai ) (7) Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 24 / 69
  • 111. Implementation of Shape Change Learning Agent As # of learning episodes increase, learned action-value function Q(s,a) converges asymptotically to optimal action-value function Q∗(s, a) Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 25 / 69
  • 112. Implementation of Shape Change Learning Agent As # of learning episodes increase, learned action-value function Q(s,a) converges asymptotically to optimal action-value function Q∗(s, a) Q-Learning method is an off-policy one because it evaluates target policy (the greedy policy) while following another policy Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 25 / 69
  • 113. Implementation of Shape Change Learning Agent With fewer # of states actions of RL problem, action-value function Q(s,a) can be tabulated, where an entity in the table is action value for each state-action pair Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 26 / 69
  • 114. Implementation of Shape Change Learning Agent With fewer # of states actions of RL problem, action-value function Q(s,a) can be tabulated, where an entity in the table is action value for each state-action pair States in RL problem for morphing vehicle are shape of wings Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 26 / 69
  • 115. Implementation of Shape Change Learning Agent With fewer # of states actions of RL problem, action-value function Q(s,a) can be tabulated, where an entity in the table is action value for each state-action pair States in RL problem for morphing vehicle are shape of wings States in morphing vehicle are continuous there are an infinite # of state-action pairs Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 26 / 69
  • 116. Implementation of Shape Change Learning Agent With fewer # of states actions of RL problem, action-value function Q(s,a) can be tabulated, where an entity in the table is action value for each state-action pair States in RL problem for morphing vehicle are shape of wings States in morphing vehicle are continuous there are an infinite # of state-action pairs To reduce # of state-action pairs, artificially quantize states into discrete sets while maintaining integrity of learned action-value function Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 26 / 69
  • 117. Implementation of Shape Change Learning Agent With fewer # of states actions of RL problem, action-value function Q(s,a) can be tabulated, where an entity in the table is action value for each state-action pair States in RL problem for morphing vehicle are shape of wings States in morphing vehicle are continuous there are an infinite # of state-action pairs To reduce # of state-action pairs, artificially quantize states into discrete sets while maintaining integrity of learned action-value function Depth of quantization is determined through experiments Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 26 / 69
  • 118. Implementation of Shape Change Learning Agent With fewer # of states actions of RL problem, action-value function Q(s,a) can be tabulated, where an entity in the table is action value for each state-action pair States in RL problem for morphing vehicle are shape of wings States in morphing vehicle are continuous there are an infinite # of state-action pairs To reduce # of state-action pairs, artificially quantize states into discrete sets while maintaining integrity of learned action-value function Depth of quantization is determined through experiments # of state-action pairs must be kept manageable while adding more morphing state variables to existing wing states of thickness camber Kumar Digvijay Mishra 3.2 Learning Air Vehicle Shape Changes January 11, 2021 26 / 69
  • 119. 4.1 Aerodynamic Modeling of Morphing Air Vehicle Constant strength doublet-source panel6model used to calculate aerodynamic properties of various wing configurations 6 Niksch 2009 Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
  • 120. 4.1 Aerodynamic Modeling of Morphing Air Vehicle Constant strength doublet-source panel6model used to calculate aerodynamic properties of various wing configurations Better than CFD methods 6 Niksch 2009 Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
  • 121. 4.1 Aerodynamic Modeling of Morphing Air Vehicle Constant strength doublet-source panel6model used to calculate aerodynamic properties of various wing configurations Better than CFD methods Assumption: Flow is incompressible and inviscid 6 Niksch 2009 Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
  • 122. 4.1 Aerodynamic Modeling of Morphing Air Vehicle Constant strength doublet-source panel6model used to calculate aerodynamic properties of various wing configurations Better than CFD methods Assumption: Flow is incompressible and inviscid =⇒ Model is valid for linear range of angle-of-attack 6 Niksch 2009 Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
  • 123. 4.1 Aerodynamic Modeling of Morphing Air Vehicle Constant strength doublet-source panel6model used to calculate aerodynamic properties of various wing configurations Better than CFD methods Assumption: Flow is incompressible and inviscid =⇒ Model is valid for linear range of angle-of-attack Limitations of the model: 6 Niksch 2009 Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
  • 124. 4.1 Aerodynamic Modeling of Morphing Air Vehicle Constant strength doublet-source panel6model used to calculate aerodynamic properties of various wing configurations Better than CFD methods Assumption: Flow is incompressible and inviscid =⇒ Model is valid for linear range of angle-of-attack Limitations of the model: 1 Sensitive to the grid size, location of panels, and # of panels 6 Niksch 2009 Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
  • 125. 4.1 Aerodynamic Modeling of Morphing Air Vehicle Constant strength doublet-source panel6model used to calculate aerodynamic properties of various wing configurations Better than CFD methods Assumption: Flow is incompressible and inviscid =⇒ Model is valid for linear range of angle-of-attack Limitations of the model: 1 Sensitive to the grid size, location of panels, and # of panels because model uses a panel method to determine aerodynamics 6 Niksch 2009 Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
  • 126. 4.1 Aerodynamic Modeling of Morphing Air Vehicle Constant strength doublet-source panel6model used to calculate aerodynamic properties of various wing configurations Better than CFD methods Assumption: Flow is incompressible and inviscid =⇒ Model is valid for linear range of angle-of-attack Limitations of the model: 1 Sensitive to the grid size, location of panels, and # of panels because model uses a panel method to determine aerodynamics 2 Only NACA 4-Digit Series airfoils considered ◦ Have explicit equations that describe upper and lower geometries of airfoil sections as a function of camber and thickness ◦ Easier to examine and optimize to achieve best possible wing shape ◦ Have blunt leading edges for thick/thin airfoil sections ◦ Are ideal for subsonic flight conditions 6 Niksch 2009 Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 27 / 69
  • 127. Versatile Aerodynamic Modeling of Morphing Air Vehicle Morphing DOF and flight condition parameters in model are: 1 Wing thickness, t 2 Maximum camber 3 Location of maximum camber 4 Root chord, cr 5 Tip chord, ct 6 Sweep angle, Λ 7 Dihedral angle, Γ 8 Wing span, b 9 Wing angle-of-attack, α Kumar Digvijay Mishra 4.1 Mathematical Modeling January 11, 2021 28 / 69
  • 128. 4.2 Constitutive Equations Constant strength doublet-source panel method is used for modeling wing 7 Katz and Plotkin 2001 Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 29 / 69
  • 129. 4.2 Constitutive Equations Constant strength doublet-source panel method is used for modeling wing Basic potential flow theory7used to obtain equations for aerodynamic forces on wing 52 φ = 0 (8) Laplace equation where 52 is the Laplace operator and φ is the velocity potential 7 Katz and Plotkin 2001 Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 29 / 69
  • 130. 4.2 Constitutive Equations Constant strength doublet-source panel method is used for modeling wing Basic potential flow theory7used to obtain equations for aerodynamic forces on wing 52 φ = 0 (8) Laplace equation where 52 is the Laplace operator and φ is the velocity potential Assumption: Incompressible and inviscid flow 7 Katz and Plotkin 2001 Kumar Digvijay Mishra 4. Mathematical Modeling January 11, 2021 29 / 69
  • 131. Constitutive Equations Using Green’s identity, solution to Laplace equation is formed with a sum of source σ and doublet µ distributions along the boundary SB Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 30 / 69
  • 132. Constitutive Equations Using Green’s identity, solution to Laplace equation is formed with a sum of source σ and doublet µ distributions along the boundary SB φ = − 1 4π Z SB [σ( 1 r ) − µn · 5( 1 r )]dS + φ∞ (9) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 30 / 69
  • 133. Constitutive Equations Using Green’s identity, solution to Laplace equation is formed with a sum of source σ and doublet µ distributions along the boundary SB φ = − 1 4π Z SB [σ( 1 r ) − µn · 5( 1 r )]dS + φ∞ (9) Assuming wake convects at trailing edge of wing as a set of thin doublets, (9) becomes Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 30 / 69
  • 134. Constitutive Equations Using Green’s identity, solution to Laplace equation is formed with a sum of source σ and doublet µ distributions along the boundary SB φ = − 1 4π Z SB [σ( 1 r ) − µn · 5( 1 r )]dS + φ∞ (9) Assuming wake convects at trailing edge of wing as a set of thin doublets, (9) becomes φ = 1 4π Z Body+Wake µn · 5( 1 r )dS − 1 4π Z Body σ( 1 r ) + φ∞ (10) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 30 / 69
  • 135. Boundary Conditions BC is: No-penetration condition - requires normal velocity of flow at surface to be zero Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 31 / 69
  • 136. Boundary Conditions BC is: No-penetration condition - requires normal velocity of flow at surface to be zero Specified by: Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 31 / 69
  • 137. Boundary Conditions BC is: No-penetration condition - requires normal velocity of flow at surface to be zero Specified by: 1 Direct formulation/Neumann condition: forces normal velocity component of flow at surface to be zero Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 31 / 69
  • 138. Boundary Conditions BC is: No-penetration condition - requires normal velocity of flow at surface to be zero Specified by: 1 Direct formulation/Neumann condition: forces normal velocity component of flow at surface to be zero 2 Indirect formulation/Dirichlet condition: specifies a value for the potential function, and indirectly satisfies zero normal flow condition on boundary Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 31 / 69
  • 139. Constitutive Equations φ = 1 4π Z Body+Wake µn · 5( 1 r )dS − 1 4π Z Body σ( 1 r ) + φ∞ Using Dirichlet BC, potential is specified at all points on the boundary Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 32 / 69
  • 140. Constitutive Equations φ = 1 4π Z Body+Wake µn · 5( 1 r )dS − 1 4π Z Body σ( 1 r ) + φ∞ Using Dirichlet BC, potential is specified at all points on the boundary If a point is placed inside the surface, inner potential, φi , is defined by the singularity distributions along the surface φi = 1 4π Z Body+Wake µ ∂ ∂n 1 r dS − 1 4π Z Body σ 1 r + φ∞ (11) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 32 / 69
  • 141. Constitutive Equations φ = 1 4π Z Body+Wake µn · 5( 1 r )dS − 1 4π Z Body σ( 1 r ) + φ∞ Using Dirichlet BC, potential is specified at all points on the boundary If a point is placed inside the surface, inner potential, φi , is defined by the singularity distributions along the surface φi = 1 4π Z Body+Wake µ ∂ ∂n 1 r dS − 1 4π Z Body σ 1 r + φ∞ (11) Do you see any problem in the integral ? Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 32 / 69
  • 142. Constitutive Equations φ = 1 4π Z Body+Wake µn · 5( 1 r )dS − 1 4π Z Body σ( 1 r ) + φ∞ Using Dirichlet BC, potential is specified at all points on the boundary If a point is placed inside the surface, inner potential, φi , is defined by the singularity distributions along the surface φi = 1 4π Z Body+Wake µ ∂ ∂n 1 r dS − 1 4π Z Body σ 1 r + φ∞ (11) Do you see any problem in the integral ? Singularity as r → 0 Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 32 / 69
  • 143. Constitutive Equations φi = 1 4π Z Body+Wake µ ∂ ∂n 1 r dS − 1 4π Z Body σ 1 r + φ∞ To evaluate integrals near singular points, no-penetration BC is enforced by setting φi to a constant value Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 33 / 69
  • 144. Constitutive Equations φi = 1 4π Z Body+Wake µ ∂ ∂n 1 r dS − 1 4π Z Body σ 1 r + φ∞ To evaluate integrals near singular points, no-penetration BC is enforced by setting φi to a constant value By choosing φi to be φ∞, we get 1 4π Z Body+Wake µ ∂ ∂n 1 r dS − 1 4π Z Body σ 1 r = 0 (12) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 33 / 69
  • 145. Constitutive Equations φi = 1 4π Z Body+Wake µ ∂ ∂n 1 r dS − 1 4π Z Body σ 1 r + φ∞ To evaluate integrals near singular points, no-penetration BC is enforced by setting φi to a constant value By choosing φi to be φ∞, we get 1 4π Z Body+Wake µ ∂ ∂n 1 r dS − 1 4π Z Body σ 1 r = 0 (12) Integral equations are reduced to a set of linear algebraic equations Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 33 / 69
  • 146. Constitutive Equations Integral equations are reduced to a set of linear algebraic equations Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 34 / 69
  • 147. Constitutive Equations Integral equations are reduced to a set of linear algebraic equations System divided into N panels for surface Nw panels for wake Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 34 / 69
  • 148. Constitutive Equations Integral equations are reduced to a set of linear algebraic equations System divided into N panels for surface Nw panels for wake BC specified at a collocation point, which for the Dirichlet BC is inside body and at center of panel Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 34 / 69
  • 149. Constitutive Equations 1 4π Z Body+Wake µ ∂ ∂n 1 r dS − 1 4π Z Body σ 1 r = 0 For N collocation points for N panels, above equation becomes, N X k=1 1 4π Z BodyPanel µn · 5 1 r dS + Nw X l=1 1 4π Z WakePanel µn · 5 1 r dS − N X k=1 1 4π Z BodyPanel σ 1 r dS = 0 (13) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 35 / 69
  • 150. Constitutive Equations N X k=1 1 4π Z BodyPanel µn · 5 1 r dS + Nw X l=1 1 4π Z WakePanel µn · 5 1 r dS − N X k=1 1 4π Z BodyPanel σ 1 r dS = 0 For each collocation point inside body, Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 36 / 69
  • 151. Constitutive Equations N X k=1 1 4π Z BodyPanel µn · 5 1 r dS + Nw X l=1 1 4π Z WakePanel µn · 5 1 r dS − N X k=1 1 4π Z BodyPanel σ 1 r dS = 0 For each collocation point inside body, N X k=1 Akµk + Nw X l=1 Al µl − N X k=1 Bkσk = 0 (14) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 36 / 69
  • 152. Constitutive Equations N X k=1 1 4π Z BodyPanel µn · 5 1 r dS + Nw X l=1 1 4π Z WakePanel µn · 5 1 r dS − N X k=1 1 4π Z BodyPanel σ 1 r dS = 0 For each collocation point inside body, N X k=1 Akµk + Nw X l=1 Al µl − N X k=1 Bkσk = 0 (14) A is doublet influence coefficient, and Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 36 / 69
  • 153. Constitutive Equations N X k=1 1 4π Z BodyPanel µn · 5 1 r dS + Nw X l=1 1 4π Z WakePanel µn · 5 1 r dS − N X k=1 1 4π Z BodyPanel σ 1 r dS = 0 For each collocation point inside body, N X k=1 Akµk + Nw X l=1 Al µl − N X k=1 Bkσk = 0 (14) A is doublet influence coefficient, and B is source influence coefficient Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 36 / 69
  • 154. Constitutive Equations N X k=1 Akµk + Nw X l=1 Al µl − N X k=1 Bkσk = 0 Kutta condition(KC): No circulation at the trailing edge of a wing section Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 37 / 69
  • 155. Constitutive Equations N X k=1 Akµk + Nw X l=1 Al µl − N X k=1 Bkσk = 0 Kutta condition(KC): No circulation at the trailing edge of a wing section Using KC, doublet strength of wake panel ≡ difference between trailing edge panels on upper surface and lower surface Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 37 / 69
  • 156. Constitutive Equations N X k=1 Akµk + Nw X l=1 Al µl − N X k=1 Bkσk = 0 Kutta condition(KC): No circulation at the trailing edge of a wing section Using KC, doublet strength of wake panel ≡ difference between trailing edge panels on upper surface and lower surface KC eliminates wake contribution in above equation, Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 37 / 69
  • 157. Constitutive Equations N X k=1 Akµk + Nw X l=1 Al µl − N X k=1 Bkσk = 0 Kutta condition(KC): No circulation at the trailing edge of a wing section Using KC, doublet strength of wake panel ≡ difference between trailing edge panels on upper surface and lower surface KC eliminates wake contribution in above equation,    a11 . . . a1N . . . ... . . . aN1 . . . aNN       µ1 . . . µN    = −    b11 . . . b1N . . . ... . . . bN1 . . . bNN       σ1 . . . σN    (15) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 37 / 69
  • 158. Constitutive Equations    a11 . . . a1N . . . ... . . . aN1 . . . aNN       µ1 . . . µN    = −    b11 . . . b1N . . . ... . . . bN1 . . . bNN       σ1 . . . σN    Set of N equations with N unknown doublet strengths (µ’s) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 38 / 69
  • 159. Constitutive Equations    a11 . . . a1N . . . ... . . . aN1 . . . aNN       µ1 . . . µN    = −    b11 . . . b1N . . . ... . . . bN1 . . . bNN       σ1 . . . σN    Set of N equations with N unknown doublet strengths (µ’s) Solvable by matrix inversion    µ1 . . . µN    = −    a11 . . . a1N . . . ... . . . aN1 . . . aNN    −1    b11 . . . b1N . . . ... . . . bN1 . . . bNN       σ1 . . . σN    Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 38 / 69
  • 160. Determination of Aerodynamic Forces After computing doublet strengths, aerodynamic forces acting on each panel are found Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 39 / 69
  • 161. Determination of Aerodynamic Forces After computing doublet strengths, aerodynamic forces acting on each panel are found 1 Determine tangential (l,m) and normal (n) perturbation velocity components for each of the panels Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 39 / 69
  • 162. Determination of Aerodynamic Forces After computing doublet strengths, aerodynamic forces acting on each panel are found 1 Determine tangential (l,m) and normal (n) perturbation velocity components for each of the panels ql = − ∂µ ∂l , qm = − ∂µ ∂m , qn = −σ (16) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 39 / 69
  • 163. Determination of Aerodynamic Forces After computing doublet strengths, aerodynamic forces acting on each panel are found 1 Determine tangential (l,m) and normal (n) perturbation velocity components for each of the panels ql = − ∂µ ∂l , qm = − ∂µ ∂m , qn = −σ (16) 2 Total velocity of each panel k is computed with Qk = (Q∞l , Q∞m, Q∞n)k + (ql , qm, qn)k (17) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 39 / 69
  • 164. Determination of Aerodynamic Forces Qk = (Q∞l , Q∞m, Q∞n)k + (ql , qm, qn)k These velocities are expressed in local panel coordinate system Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 40 / 69
  • 165. Determination of Aerodynamic Forces Qk = (Q∞l , Q∞m, Q∞n)k + (ql , qm, qn)k These velocities are expressed in local panel coordinate system 3 Pressure coefficient at each panel is found using modified form of Bernoulli’s equation Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 40 / 69
  • 166. Determination of Aerodynamic Forces Qk = (Q∞l , Q∞m, Q∞n)k + (ql , qm, qn)k These velocities are expressed in local panel coordinate system 3 Pressure coefficient at each panel is found using modified form of Bernoulli’s equation Cpk = 1 − Q2 k Q2 ∞ (18) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 40 / 69
  • 167. Determination of Aerodynamic Forces Qk = (Q∞l , Q∞m, Q∞n)k + (ql , qm, qn)k These velocities are expressed in local panel coordinate system 3 Pressure coefficient at each panel is found using modified form of Bernoulli’s equation Cpk = 1 − Q2 k Q2 ∞ (18) 4 Summation of aerodynamic forces from each panel gives total aerodynamic force Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 40 / 69
  • 168. Determination of Aerodynamic Forces Qk = (Q∞l , Q∞m, Q∞n)k + (ql , qm, qn)k These velocities are expressed in local panel coordinate system 3 Pressure coefficient at each panel is found using modified form of Bernoulli’s equation Cpk = 1 − Q2 k Q2 ∞ (18) 4 Summation of aerodynamic forces from each panel gives total aerodynamic force X CFk = − Cpk 4S S · nk (19) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 40 / 69
  • 169. Determination of Aerodynamic Forces 5 Parasitic drag coefficient CD0 is calculated based on wing area S and equivalent parasite area f Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 41 / 69
  • 170. Determination of Aerodynamic Forces 5 Parasitic drag coefficient CD0 is calculated based on wing area S and equivalent parasite area f CD0 = f S (20) Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 41 / 69
  • 171. Determination of Aerodynamic Forces 5 Parasitic drag coefficient CD0 is calculated based on wing area S and equivalent parasite area f CD0 = f S (20) where equivalent parasite area is related to wetted area by log10f = a + b log10Swet (21) a, b - correlation coefficients related to equivalent skin friction of aircraft Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 41 / 69
  • 172. Determination of Aerodynamic Forces 5 Parasitic drag coefficient CD0 is calculated based on wing area S and equivalent parasite area f CD0 = f S (20) where equivalent parasite area is related to wetted area by log10f = a + b log10Swet (21) a, b - correlation coefficients related to equivalent skin friction of aircraft 6 Oswald efficiency factor is, e = 1.78(1 − 0.045 AR0.68 ) − 0.64 (22) valid if sweep angle of leading edge of wing 300 Kumar Digvijay Mishra 4.2 Mathematical Modeling January 11, 2021 41 / 69
  • 173. 4.3 Model Grid Cosine spacing used to generate grid for both chordwise and spanwise paneling8 xvi = c(y) 2 1 − cos iπ N + 1 ; i = 1 : N (23) 7 Moran 1984 Kumar Digvijay Mishra 4. Mathematical Modeling of Morphing Air Vehicle January 11, 2021 42 / 69
  • 174. 4.3 Model Grid Cosine spacing used to generate grid for both chordwise and spanwise paneling8 xvi = c(y) 2 1 − cos iπ N + 1 ; i = 1 : N (23) Cosine spacing gives more panels near leading and trailing edges as well as root and tip of the wing where many aerodynamic changes occur 7 Moran 1984 Kumar Digvijay Mishra 4. Mathematical Modeling of Morphing Air Vehicle January 11, 2021 42 / 69
  • 175. 4.3 Model Grid Straight tapered wing configuration with cosine spacing Kumar Digvijay Mishra 4. Mathematical Modeling of Morphing Air Vehicle January 11, 2021 43 / 69
  • 176. 4.3 Model Grid Swept wing configuration with cosine spacing More panels placed where many aerodynamic changes occur: 1 Leading and trailing edges as well as root and tip of the wing 2 Four corner points of the wing Kumar Digvijay Mishra 4. Mathematical Modeling of Morphing Air Vehicle January 11, 2021 44 / 69
  • 177. 4.3 Model Grid Delta wing configuration with cosine spacing More panels placed where many aerodynamic changes occur: 1 Leading and trailing edges as well as root and tip of the wing 2 Four corner points of the wing Kumar Digvijay Mishra 4. Mathematical Modeling of Morphing Air Vehicle January 11, 2021 45 / 69
  • 178. Model Grid Accuracy As # of panels decrease, accuracy of model also decreases Kumar Digvijay Mishra 4.3 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 46 / 69
  • 179. Model Grid Accuracy As # of panels decrease, accuracy of model also decreases Balance between accuracy and computational time is achieved by defining set # of panels beyond which accuracy does not increase appreciably Kumar Digvijay Mishra 4.3 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 46 / 69
  • 180. Model Grid Accuracy As # of panels decrease, accuracy of model also decreases Balance between accuracy and computational time is achieved by defining set # of panels beyond which accuracy does not increase appreciably Ex: Doubling no. of panels is acceptable if accuracy of the model increased by 10% Kumar Digvijay Mishra 4.3 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 46 / 69
  • 181. Model Grid Accuracy As # of panels decrease, accuracy of model also decreases Balance between accuracy and computational time is achieved by defining set # of panels beyond which accuracy does not increase appreciably Ex: Doubling no. of panels is acceptable if accuracy of the model increased by 10% 50% increase in the set of panels is unnecessary, if accuracy is increased by less than 1% Kumar Digvijay Mishra 4.3 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 46 / 69
  • 182. 4.4 Dynamical Modeling Kumar Digvijay Mishra 4. Mathematical Modeling of Morphing Air Vehicle January 11, 2021 47 / 69
  • 183. Dynamical Modeling Velocity vector v with body-axis components is v =   u v w   (24) Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 48 / 69
  • 184. Dynamical Modeling Velocity vector v with body-axis components is v =   u v w   (24) Angular velocity vector ω with roll(p), pitch(q), and yaw(r) components in body-axis coordinate system is ω =   p q r   (25) Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 48 / 69
  • 185. Dynamical Modeling Derivative of velocity vector v with respect to time yields v̇N = v̇B + ω × v =   u̇ + qw − rv v̇ + ru − pw ẇ + pv − qu   (26) Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 49 / 69
  • 186. Dynamical Modeling Derivative of velocity vector v with respect to time yields v̇N = v̇B + ω × v =   u̇ + qw − rv v̇ + ru − pw ẇ + pv − qu   (26) Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 49 / 69
  • 187. Dynamical Modeling Derivative of velocity vector v with respect to time yields v̇N = v̇B + ω × v =   u̇ + qw − rv v̇ + ru − pw ẇ + pv − qu   (26) Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 49 / 69
  • 188. Dynamical Modeling Derivative of velocity vector v with respect to time yields v̇N = v̇B + ω × v =   u̇ + qw − rv v̇ + ru − pw ẇ + pv − qu   (26) u̇, v̇, and ẇ: acceleration of body along forward, lateral, and down/up-ward direction Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 49 / 69
  • 189. Dynamical Modeling Derivative of velocity vector v with respect to time yields v̇N = v̇B + ω × v =   u̇ + qw − rv v̇ + ru − pw ẇ + pv − qu   (26) u̇, v̇, and ẇ: acceleration of body along forward, lateral, and down/up-ward direction There is always forward acceleration, but no lateral acceleration for longitudinal maneuvers Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 49 / 69
  • 190. Dynamical Modeling Derivative of velocity vector v with respect to time yields v̇N = v̇B + ω × v =   u̇ + qw − rv v̇ + ru − pw ẇ + pv − qu   (26) u̇, v̇, and ẇ: acceleration of body along forward, lateral, and down/up-ward direction There is always forward acceleration, but no lateral acceleration for longitudinal maneuvers Notice association of {roll(p), pitch(q), yaw(r)} components of angular velocity with translational velocity components add to acceleration components (u̇, v̇, ẇ) Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 49 / 69
  • 191. Dynamical Modeling Force vector is, F =   X + Tx + Gx Y + Ty + Gy Z + Tz + Gz   =   −mg sin(θ) − D cos(α)cos(β) + L sin(α)cos(β) + T mg sin(φ)cos(θ) + D cos(α)sin(β) − L sin(α)sin(β) mg cos(φ)cos(θ) − D sin(α) − L cos(α)   (27) Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 50 / 69
  • 192. Dynamical Modeling Force vector is, F =   X + Tx + Gx Y + Ty + Gy Z + Tz + Gz   =   −mg sin(θ) − D cos(α)cos(β) + L sin(α)cos(β) + T mg sin(φ)cos(θ) + D cos(α)sin(β) − L sin(α)sin(β) mg cos(φ)cos(θ) − D sin(α) − L cos(α)   (27) G: Gravitational force; T: Propulsive(thrust) force Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 50 / 69
  • 193. Dynamical Modeling Force vector is, F =   X + Tx + Gx Y + Ty + Gy Z + Tz + Gz   =   −mg sin(θ) − D cos(α)cos(β) + L sin(α)cos(β) + T mg sin(φ)cos(θ) + D cos(α)sin(β) − L sin(α)sin(β) mg cos(φ)cos(θ) − D sin(α) − L cos(α)   (27) G: Gravitational force; T: Propulsive(thrust) force D: Drag force; L: Lift force Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 50 / 69
  • 194. Dynamical Modeling Force vector is, F =   X + Tx + Gx Y + Ty + Gy Z + Tz + Gz   =   −mg sin(θ) − D cos(α)cos(β) + L sin(α)cos(β) + T mg sin(φ)cos(θ) + D cos(α)sin(β) − L sin(α)sin(β) mg cos(φ)cos(θ) − D sin(α) − L cos(α)   (27) G: Gravitational force; T: Propulsive(thrust) force D: Drag force; L: Lift force (X, Y, Z): Aerodynamic forces calculated using constant strength source-doublet panel method Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 50 / 69
  • 195. Dynamical Modeling Force vector is, F =   X + Tx + Gx Y + Ty + Gy Z + Tz + Gz   =   −mg sin(θ) − D cos(α)cos(β) + L sin(α)cos(β) + T mg sin(φ)cos(θ) + D cos(α)sin(β) − L sin(α)sin(β) mg cos(φ)cos(θ) − D sin(α) − L cos(α)   (27) G: Gravitational force; T: Propulsive(thrust) force D: Drag force; L: Lift force (X, Y, Z): Aerodynamic forces calculated using constant strength source-doublet panel method ψ, θ, φ: Yaw, pitch roll angles Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 50 / 69
  • 196. Dynamical Modeling Force vector is, F =   X + Tx + Gx Y + Ty + Gy Z + Tz + Gz   =   −mg sin(θ) − D cos(α)cos(β) + L sin(α)cos(β) + T mg sin(φ)cos(θ) + D cos(α)sin(β) − L sin(α)sin(β) mg cos(φ)cos(θ) − D sin(α) − L cos(α)   (27) G: Gravitational force; T: Propulsive(thrust) force D: Drag force; L: Lift force (X, Y, Z): Aerodynamic forces calculated using constant strength source-doublet panel method ψ, θ, φ: Yaw, pitch roll angles (α, β): Aerodynamic angles (AoA, sideslip) Kumar Digvijay Mishra 4.4 Mathematical Modeling of Morphing Air Vehicle January 11, 2021 50 / 69
  • 197. Dynamical Modeling From Newton’s 2nd law, translational equations of motion are: m(u̇ +qw −rv) = −mgsin(θ)−Dcos(α)cos(β)+Lsin(α)cos(β)+T (28) m(v̇ +ru−pw) = −mgsin(φ)cos(θ)−Dcos(α)sin(β)−Lsin(α)sin(β) (29) m(ẇ + pv − qu) = −mgcos(φ)cos(θ) − Dsin(α) − Lcos(α) (30) p, q, r: roll, pitch, yaw components of angular velocity vector, ω u, v, w: components of body-axis translational velocity vector, v ψ, θ, φ: yaw(unused), pitch, roll angles α, β: aerodynamic angles T: Thrust; D: Drag ; L: Lift Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 51 / 69
  • 198. Dynamical Modeling Euler’s equation L = ḣ N gives relationship between vector sum of moments L and angular momentum vector h Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 52 / 69
  • 199. Dynamical Modeling Euler’s equation L = ḣ N gives relationship between vector sum of moments L and angular momentum vector h Angular momentum vector related to inertia matrix I and body-axis angular velocity vector ω by h = Iω (31) Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 52 / 69
  • 200. Dynamical Modeling Euler’s equation L = ḣ N gives relationship between vector sum of moments L and angular momentum vector h Angular momentum vector related to inertia matrix I and body-axis angular velocity vector ω by h = Iω (31) Inertia matrix is, I = Ixx −Ixy −Ixz −Ixy Iyy −Iyz −Ixz −Iyz Izz (32) Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 52 / 69
  • 201. Dynamical Modeling Euler’s equation L = ḣ N gives relationship between vector sum of moments L and angular momentum vector h Angular momentum vector related to inertia matrix I and body-axis angular velocity vector ω by h = Iω (31) Inertia matrix is, I = Ixx −Ixy −Ixz −Ixy Iyy −Iyz −Ixz −Iyz Izz (32) Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 52 / 69
  • 202. Dynamical Modeling Euler’s equation L = ḣ N gives relationship between vector sum of moments L and angular momentum vector h Angular momentum vector related to inertia matrix I and body-axis angular velocity vector ω by h = Iω (31) Inertia matrix is, I = Ixx −Ixy −Ixz −Ixy Iyy −Iyz −Ixz −Iyz Izz (32) 1 These moments-of-inertia are time-varying because aircraft wing morphs during flight Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 52 / 69
  • 203. Dynamical Modeling Euler’s equation L = ḣ N gives relationship between vector sum of moments L and angular momentum vector h Angular momentum vector related to inertia matrix I and body-axis angular velocity vector ω by h = Iω (31) Inertia matrix is, I = Ixx −Ixy −Ixz −Ixy Iyy −Iyz −Ixz −Iyz Izz (32) 1 These moments-of-inertia are time-varying because aircraft wing morphs during flight 2 Should be computed for each state of morphing wing aircraft as well as transitions between them. How to compute them in real-time? Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 52 / 69
  • 204. Dynamical Modeling Time derivative of angular momentum vector h is, ḣ N = Iω̇B + İ B ω + ω × Iω =         Ixx ṗ − Ixy q̇ − Ixz ˙ r + ˙ Ixx p − ˙ Ixy q − ˙ Ixzr+p(−Ixzq + Ixy r) + q(−Iyzq − Iyy r) + r(Izzq + Iyzr) −Ixy ṗ − Iyy q̇ − Iyz ˙ r − ˙ Ixy p + ˙ Iyy q − ˙ Iyzr+p(Ixx r + Ixzp) + q(−Ixy r + Iyzp) + r(−Ixzr − Izzp) −Ixzṗ − Iyzq̇ + Izz ˙ r − ˙ Ixzp − ˙ Iyzq + ˙ Izzr+p(−Ixy p − Ixx q) +q(Iyy p + Ixy q) + r(−Iyzp + Ixzq)         (33) Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 53 / 69
  • 205. Dynamical Modeling Time derivative of angular momentum vector h is, ḣ N = Iω̇B + İ B ω + ω × Iω =         Ixx ṗ − Ixy q̇ − Ixz ˙ r + ˙ Ixx p − ˙ Ixy q − ˙ Ixzr+p(−Ixzq + Ixy r) + q(−Iyzq − Iyy r) + r(Izzq + Iyzr) −Ixy ṗ − Iyy q̇ − Iyz ˙ r − ˙ Ixy p + ˙ Iyy q − ˙ Iyzr+p(Ixx r + Ixzp) + q(−Ixy r + Iyzp) + r(−Ixzr − Izzp) −Ixzṗ − Iyzq̇ + Izz ˙ r − ˙ Ixzp − ˙ Iyzq + ˙ Izzr+p(−Ixy p − Ixx q) +q(Iyy p + Ixy q) + r(−Iyzp + Ixzq)         (33) 9 inertia terms that vary in temporal dimension Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 53 / 69
  • 206. Dynamical Modeling Time derivative of angular momentum vector h is, ḣ N = Iω̇B + İ B ω + ω × Iω =         Ixx ṗ − Ixy q̇ − Ixz ˙ r + ˙ Ixx p − ˙ Ixy q − ˙ Ixzr+p(−Ixzq + Ixy r) + q(−Iyzq − Iyy r) + r(Izzq + Iyzr) −Ixy ṗ − Iyy q̇ − Iyz ˙ r − ˙ Ixy p + ˙ Iyy q − ˙ Iyzr+p(Ixx r + Ixzp) + q(−Ixy r + Iyzp) + r(−Ixzr − Izzp) −Ixzṗ − Iyzq̇ + Izz ˙ r − ˙ Ixzp − ˙ Iyzq + ˙ Izzr+p(−Ixy p − Ixx q) +q(Iyy p + Ixy q) + r(−Iyzp + Ixzq)         (33) 9 inertia terms that vary in temporal dimension Is it prudent to neglect time-varying inertia due to morphing? Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 53 / 69
  • 207. Dynamical Modeling Moments are due to ? Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
  • 208. Dynamical Modeling Moments are due to ? Aerodynamic forces Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
  • 209. Dynamical Modeling Moments are due to ? Aerodynamic forces where aerodynamic forces are calculated by ? Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
  • 210. Dynamical Modeling Moments are due to ? Aerodynamic forces where aerodynamic forces are calculated by ? Constant strength source-doublet panel method Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
  • 211. Dynamical Modeling Moments are due to ? Aerodynamic forces where aerodynamic forces are calculated by ? Constant strength source-doublet panel method Aerodynamic moment vector is, L =   LA MA NA   (34) Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
  • 212. Dynamical Modeling Moments are due to ? Aerodynamic forces where aerodynamic forces are calculated by ? Constant strength source-doublet panel method Aerodynamic moment vector is, L =   LA MA NA   (34) LA: Roll moment MA: Pitch moment NA: Yaw moment Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
  • 213. Dynamical Modeling Moments are due to ? Aerodynamic forces where aerodynamic forces are calculated by ? Constant strength source-doublet panel method Aerodynamic moment vector is, L =   LA MA NA   (34) LA: Roll moment MA: Pitch moment NA: Yaw moment Why not account for moment produced by propulsive forces? Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
  • 214. Dynamical Modeling Moments are due to ? Aerodynamic forces where aerodynamic forces are calculated by ? Constant strength source-doublet panel method Aerodynamic moment vector is, L =   LA MA NA   (34) LA: Roll moment MA: Pitch moment NA: Yaw moment Why not account for moment produced by propulsive forces? Because thrust force is assumed to act along the line parallel to body’s x-axis at C-of-G location Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 54 / 69
  • 215. Rotational Equation of Motion for Morphing Wing Using Euler’s equation L = ḣ N , LA = Ixx ṗ − Ixy q̇ − Ixz ˙ r + ˙ Ixx p − ˙ Ixy q − ˙ Ixzr+p(−Ixzq + Ixy r) + q(−Iyzq − Iyy r) + r(Izzq + Iyzr) (35) Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 55 / 69
  • 216. Rotational Equation of Motion for Morphing Wing Using Euler’s equation L = ḣ N , LA = Ixx ṗ − Ixy q̇ − Ixz ˙ r + ˙ Ixx p − ˙ Ixy q − ˙ Ixzr+p(−Ixzq + Ixy r) + q(−Iyzq − Iyy r) + r(Izzq + Iyzr) (35) MA = −Ixy ṗ + Iyy q̇ − Iyz ˙ r − ˙ Ixy p + ˙ Iyy q − ˙ Iyzr+p(Ixx r + Ixzp) + q(−Ixy r + Iyzp) + r(−Ixzr − Izzp) (36) Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 55 / 69
  • 217. Rotational Equation of Motion for Morphing Wing Using Euler’s equation L = ḣ N , LA = Ixx ṗ − Ixy q̇ − Ixz ˙ r + ˙ Ixx p − ˙ Ixy q − ˙ Ixzr+p(−Ixzq + Ixy r) + q(−Iyzq − Iyy r) + r(Izzq + Iyzr) (35) MA = −Ixy ṗ + Iyy q̇ − Iyz ˙ r − ˙ Ixy p + ˙ Iyy q − ˙ Iyzr+p(Ixx r + Ixzp) + q(−Ixy r + Iyzp) + r(−Ixzr − Izzp) (36) NA = −Ixzṗ − Iyzq̇ + Izz ˙ r − ˙ Ixzp − ˙ Iyzq + ˙ Izzr+p(−Ixy p − Ixx q) + q(Iyy p + Ixy q) + r(−Iyzp + Ixzq) (37) Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 55 / 69
  • 218. Rotational Equation of Motion for Morphing Wing Using Euler’s equation L = ḣ N , LA = Ixx ṗ − Ixy q̇ − Ixz ˙ r + ˙ Ixx p − ˙ Ixy q − ˙ Ixzr+p(−Ixzq + Ixy r) + q(−Iyzq − Iyy r) + r(Izzq + Iyzr) (35) MA = −Ixy ṗ + Iyy q̇ − Iyz ˙ r − ˙ Ixy p + ˙ Iyy q − ˙ Iyzr+p(Ixx r + Ixzp) + q(−Ixy r + Iyzp) + r(−Ixzr − Izzp) (36) NA = −Ixzṗ − Iyzq̇ + Izz ˙ r − ˙ Ixzp − ˙ Iyzq + ˙ Izzr+p(−Ixy p − Ixx q) + q(Iyy p + Ixy q) + r(−Iyzp + Ixzq) (37) Time-varying inertias are present in roll, pitch and yaw moments. Why neglect them? Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 55 / 69
  • 219. Exercise - Inertia of an aircraft Ixx = N/m2 Ixy = N/m2 Ixz = N/m2 Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 56 / 69
  • 220. Exercise - Inertia of an aircraft Ixx = N/m2 Ixy = N/m2 Ixz = N/m2 Iyx = N/m2 Iyy = N/m2 Iyz = N/m2 Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 56 / 69
  • 221. Exercise - Inertia of an aircraft Ixx = N/m2 Ixy = N/m2 Ixz = N/m2 Iyx = N/m2 Iyy = N/m2 Iyz = N/m2 Izz = N/m2 Izy = N/m2 Izz = N/m2 Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 56 / 69
  • 222. Exercise - Inertia of an aircraft Ixx = N/m2 Ixy = N/m2 Ixz = N/m2 Iyx = N/m2 Iyy = N/m2 Iyz = N/m2 Izz = N/m2 Izy = N/m2 Izz = N/m2 Rudder Fault =⇒ Transience in Inertia - Is ˙ Iij negligible? Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 56 / 69
  • 223. Dynamical Modeling Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 57 / 69
  • 224. Dynamical Modeling Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 57 / 69
  • 225. Dynamical Modeling Aerodynamic angles α and β are, tan(α) = w VT (38) tan(β) = v VT (39) Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 57 / 69
  • 226. Dynamical Modeling Aerodynamic angles α and β are, tan(α) = w VT (38) tan(β) = v VT (39) Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 57 / 69
  • 227. Dynamical Modeling Aerodynamic angles α and β are, tan(α) = w VT (38) tan(β) = v VT (39) Body axis system Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 57 / 69
  • 228. Dynamical Modeling Aerodynamic angles α and β are, tan(α) = w VT (38) tan(β) = v VT (39) Body axis system w: velocity along body z-axis v: velocity along body y-axis VT : total body velocity Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 57 / 69
  • 229. Dynamical Modeling Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 58 / 69
  • 230. Dynamical Modeling Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 58 / 69
  • 231. Dynamical Modeling Aerodynamic angles change at a rate of, α̇ = 1 VT (ẇ cos α − u̇ sin α) (40) β̇ = 1 VT (v̇ cos β − u̇ sin β) (41) Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 58 / 69
  • 232. Dynamical Modeling Aerodynamic angles change at a rate of, α̇ = 1 VT (ẇ cos α − u̇ sin α) (40) β̇ = 1 VT (v̇ cos β − u̇ sin β) (41) Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 58 / 69
  • 233. Dynamical Modeling Aerodynamic angles change at a rate of, α̇ = 1 VT (ẇ cos α − u̇ sin α) (40) β̇ = 1 VT (v̇ cos β − u̇ sin β) (41) Body axis system Kumar Digvijay Mishra 4.4 Modeling of Morphing Air Vehicle January 11, 2021 58 / 69