Infotech@Aerospace
19 - 22 August 2013
Boston, Massachusetts
Your systems. Working as one.

An Extensible Architecture for Avionics
Sensor Health Assessment Using DDS

Sumant Tambe, Ph.D.
Senior Software Research Engineer
Real-Time Innovations, Inc.

12/3/2013

Acknowledgement: This research is funded by the Air Force Research Laboratory, WPAFB, Dayton, OH under Small Business Innovation
Research (SBIR) contract # FA8650-11-C-1054. Manuscript approved for publication by WPAFB: PA Approval Number: 88ABW-2013-3209.
12/3/2013

NASA K10 Rover:
Surface Telerobotics
Uses RTI Connext DDS

© 2012 RTI • COMPANY CONFIDENTIAL

2
Why NASA Uses RTI Connext DDS
• NASA was looking for a software architecture and
communications infrastructure that enabled reliable,
standards-based messaging between the
International Space Station and Earth.
• NASA relied on the RTI Connext DDS solution because
of its ability to tolerate time delay and loss of signal.
• A common, flexible, interoperable data
communications interface that would readily
integrate across each robot’s disparate applications
and operating systems.
12/3/2013

© 2013 Real-time Innovations, Inc.

3
Data-Centric Architecture of DDS
Sensor Health
Assessment

Airborne
Sensor
DataWriter

publish

DDS Global Data Space
Sensor ID

Airborne
Sensor

12.35

35.45

0x2

1.34

6.78

ID

Roll

10

35.5
35.5

DataReader

-45

10.1

subscribe

Yaw

0x4

Airborne
Sensor

Pitch

0x3

DataWriter

DataReader

Velocity Y

0x1

publish

Velocity X

subscribe

-30

subscribe
publish

DataReader

DataWriter

12/3/2013

© 2013 Real-time Innovations, Inc.

4
Research: Cyber Situational Awareness

• Research is funded by the Air Force Research
Laboratory, WPAFB, Dayton, OH

12/3/2013

© 2013 Real-time Innovations, Inc.

5
Context: Cyber Situational Awareness
• Enterprise Security Situational Awareness
• Sensor Health Management
Security Situational
Awareness

Sensor Health Management

Goal

Situation (Attack) recognition, comprehension,
projection

Fault Detection, Isolation, Mitigation

Sensors

E.g., Host-based and network intrusion detection
systems (OSSEC, Snort, Ganglia), malware
detectors, firewalls, operating system, etc.

E.g., Accelerometer, fuel-flow sensors, altimeter,
speedometer, etc.

Input to
Analysis

Events and Alerts: log analysis, file integrity
checking, policy monitoring, rootkit detection, realtime alerting and active response

Sensor readings, software status signals, software
quality signals, operating system information

Analysis
Techniques

Complex Event Processing, Classification (Naïve
Bayes, SVM, linear classifiers), clustering (Kmeans), pattern matching etc.

Bayesian Networks, Hybrid Bayesian, Timed Fault
Propagation Graphs (TFPG),

Tools/Librari
es

R, Weka

Smile and Genie

Visualization

Histograms, heat maps, time series

Time series

12/3/2013

© 2013 Real-time Innovations, Inc.

6
Sensor Health Management

12/3/2013

© 2013 Real-time Innovations, Inc.

7
Cyber Health Analysis of Airborne Sensors
Propulsion

Comms

Engine

Electrical

Navigation

(formerly Predator B)
Mechanical
Payload

STANAG 4586 Message #1101: Subsystem Status Report
0 = No Status; 1 = Nominal; 2 = Caution; 3 = Warning; 4 = Emergency; 5 = Failed
Hardware-in-the-loop Simulation
Visualization

Sensor Health
Estimates
BeagleBone

Bayesian Network

Orientation
Sensor
(IMU)
Sensor Health
Analysis using
Bayesian Network

Pitch/Roll/Yaw
updates over TTL UART1

EngineOperatingStates

Smile library

Simulated UAV
Engine Data
(STANAG 4586)

DDS
Simulated MQ-9 Reaper UAV
12/3/2013

© 2013 Real-time Innovations, Inc.

9
Flight Simulator
• Flight Simulator
– E.g., Flight-Sim , X-Plane
– Realistic aerodynamic simulation
– Large number of model planes
including UAVs
• E.g., GD MQ-9 Reaper

– Simulates sensor failures
•

E.g., Engine fire

• Using Simulator data
– Publish using DDS
– Drive the Bayesian Network
– Inject artificial failure to evaluate
BaysNet
12/3/2013

© 2012 RTI • COMPANY CONFIDENTIAL

10
Hardware-in-the-loop Simulation using
Orientation Sensors
• Hardware Description
– Beaglebone Rev A5 (700 MHz,
256 MB RAM)
– CHR-6dm AHRS Orientation
Sensor
– Communication over TTL (3.3V)
UART at 115200 Baud
– Onboard Extended Kalman Filter
(EKF) produces yaw, pitch, and
roll angle estimates

12/3/2013

BeagleBone with CHR-6dm (above)
CHR-6dm orientation Sensor (below)

© 2013 Real-time Innovations, Inc.

11
Sensor Health Analysis Using Bayesian
Networks
• Bayesian Network based on STANAG 4586
• Sensor dependencies are modeled as an acyclic
graphs
• Can learn the structure and parameters from
raw data
• Each node has a probability distribution
function with respect to its parents
• Raw values of sensor readings are discretized
and plugged in
• Health nodes (H_*) give instantaneous
probability of sensor being faulty
–
–

Health Nodes
Sensor Nodes

A real value between 0 to 1
A value above or below a threshold trigger an alert

12/3/2013

© 2013 Real-time Innovations, Inc.

12
IMU-in-the-loop Simulation (Failure Scenarios)
1

Sub-Scenario #1
High Engine Thrust

Replay
Starts
[0:00]

Replay
Ends
[5:00]

(Change Pitch and Roll Physically)

2

Replay
Starts [0:00]

Sub-Scenario #2
Very Low Engine Thrust @ [1:30]
(Change Pitch and Roll Physically)

Engine
Catches Fire
[0:57]

12/3/2013

Very Low Engine
Thrust [1:30]

© 2013 Real-time Innovations, Inc.

Replay Ends [5:00]

13
Extensibility Facilitates System Evolution
• Evolution
– Hardware evolves  Software evolves  Data-types evolve
– Evolved system components must continue to communicate with
unevolved components
– E.g., 2d point (X, Y) evolves to 3d point (X, Y, Z)
– New radar that produces 3d points must work with old display that
expects 2d

• Solution: OMG Extensible and Dynamic Topic Types
Specification
struct EngineOperatingStates {
double timestamp;
string vehicleId; //@key
long
engineStatus;
double engineSpeed;
double enginePower;
double exhaustGasTemperature;
double engineBodyTemperature;
double outputPower;
long
fireDetectionSensor;
};// Extensibility(EXTENSIBLE)
12/3/2013

struct EngineOperatingStatesSensorsHealth {
double p_engineSpeedSensor_good;
double p_engineSpeedSensor_bad;
double p_exhaustGasTempSensor_good;
double p_exhaustGasTempSensor_bad;
double p_fireDetectionSensor_good;
double p_fireDetectionSensor_bad;
. . .
};// Extensibility(EXTENSIBLE)

© 2013 Real-time Innovations, Inc.

14
Take Home Points
• Bayesian Networks
– A flexible and capable platform for fault
detection and diagnosis
– Fine-grain fault detection within a
subsystem and across subsystems
– Large number of COTS tools

• RTI Connext DDS
– Has small footprint
• Runs on a ARM processors (700 MHz,
256MB RAM)

– Has low latency
• Supports hardware-in-the-loop simulation
as well as Smart Systems

– Standards-based support for system
evolution
12/3/2013

© 2013 Real-time Innovations, Inc.

15
Future Work
1. Expanded Cyber Situational
Awareness
–

Using independent ground-based
sensors

2. Integrate portable and truly
interoperable airborne software
components
–

Use RTI Transport Services Segment
(TSS) implementation

3. Semantic Interoperability
–
–

12/3/2013

Use UAS Control Segment (UCS)
Protocol Interoperability using DDS
(RTPS)

© 2013 Real-time Innovations, Inc.

16
Thank You!

12/3/2013

© 2013 Real-time Innovations, Inc.

17

An Extensible Architecture for Avionics Sensor Health Assessment Using DDS

  • 1.
    Infotech@Aerospace 19 - 22August 2013 Boston, Massachusetts Your systems. Working as one. An Extensible Architecture for Avionics Sensor Health Assessment Using DDS Sumant Tambe, Ph.D. Senior Software Research Engineer Real-Time Innovations, Inc. 12/3/2013 Acknowledgement: This research is funded by the Air Force Research Laboratory, WPAFB, Dayton, OH under Small Business Innovation Research (SBIR) contract # FA8650-11-C-1054. Manuscript approved for publication by WPAFB: PA Approval Number: 88ABW-2013-3209.
  • 2.
    12/3/2013 NASA K10 Rover: SurfaceTelerobotics Uses RTI Connext DDS © 2012 RTI • COMPANY CONFIDENTIAL 2
  • 3.
    Why NASA UsesRTI Connext DDS • NASA was looking for a software architecture and communications infrastructure that enabled reliable, standards-based messaging between the International Space Station and Earth. • NASA relied on the RTI Connext DDS solution because of its ability to tolerate time delay and loss of signal. • A common, flexible, interoperable data communications interface that would readily integrate across each robot’s disparate applications and operating systems. 12/3/2013 © 2013 Real-time Innovations, Inc. 3
  • 4.
    Data-Centric Architecture ofDDS Sensor Health Assessment Airborne Sensor DataWriter publish DDS Global Data Space Sensor ID Airborne Sensor 12.35 35.45 0x2 1.34 6.78 ID Roll 10 35.5 35.5 DataReader -45 10.1 subscribe Yaw 0x4 Airborne Sensor Pitch 0x3 DataWriter DataReader Velocity Y 0x1 publish Velocity X subscribe -30 subscribe publish DataReader DataWriter 12/3/2013 © 2013 Real-time Innovations, Inc. 4
  • 5.
    Research: Cyber SituationalAwareness • Research is funded by the Air Force Research Laboratory, WPAFB, Dayton, OH 12/3/2013 © 2013 Real-time Innovations, Inc. 5
  • 6.
    Context: Cyber SituationalAwareness • Enterprise Security Situational Awareness • Sensor Health Management Security Situational Awareness Sensor Health Management Goal Situation (Attack) recognition, comprehension, projection Fault Detection, Isolation, Mitigation Sensors E.g., Host-based and network intrusion detection systems (OSSEC, Snort, Ganglia), malware detectors, firewalls, operating system, etc. E.g., Accelerometer, fuel-flow sensors, altimeter, speedometer, etc. Input to Analysis Events and Alerts: log analysis, file integrity checking, policy monitoring, rootkit detection, realtime alerting and active response Sensor readings, software status signals, software quality signals, operating system information Analysis Techniques Complex Event Processing, Classification (Naïve Bayes, SVM, linear classifiers), clustering (Kmeans), pattern matching etc. Bayesian Networks, Hybrid Bayesian, Timed Fault Propagation Graphs (TFPG), Tools/Librari es R, Weka Smile and Genie Visualization Histograms, heat maps, time series Time series 12/3/2013 © 2013 Real-time Innovations, Inc. 6
  • 7.
    Sensor Health Management 12/3/2013 ©2013 Real-time Innovations, Inc. 7
  • 8.
    Cyber Health Analysisof Airborne Sensors Propulsion Comms Engine Electrical Navigation (formerly Predator B) Mechanical Payload STANAG 4586 Message #1101: Subsystem Status Report 0 = No Status; 1 = Nominal; 2 = Caution; 3 = Warning; 4 = Emergency; 5 = Failed
  • 9.
    Hardware-in-the-loop Simulation Visualization Sensor Health Estimates BeagleBone BayesianNetwork Orientation Sensor (IMU) Sensor Health Analysis using Bayesian Network Pitch/Roll/Yaw updates over TTL UART1 EngineOperatingStates Smile library Simulated UAV Engine Data (STANAG 4586) DDS Simulated MQ-9 Reaper UAV 12/3/2013 © 2013 Real-time Innovations, Inc. 9
  • 10.
    Flight Simulator • FlightSimulator – E.g., Flight-Sim , X-Plane – Realistic aerodynamic simulation – Large number of model planes including UAVs • E.g., GD MQ-9 Reaper – Simulates sensor failures • E.g., Engine fire • Using Simulator data – Publish using DDS – Drive the Bayesian Network – Inject artificial failure to evaluate BaysNet 12/3/2013 © 2012 RTI • COMPANY CONFIDENTIAL 10
  • 11.
    Hardware-in-the-loop Simulation using OrientationSensors • Hardware Description – Beaglebone Rev A5 (700 MHz, 256 MB RAM) – CHR-6dm AHRS Orientation Sensor – Communication over TTL (3.3V) UART at 115200 Baud – Onboard Extended Kalman Filter (EKF) produces yaw, pitch, and roll angle estimates 12/3/2013 BeagleBone with CHR-6dm (above) CHR-6dm orientation Sensor (below) © 2013 Real-time Innovations, Inc. 11
  • 12.
    Sensor Health AnalysisUsing Bayesian Networks • Bayesian Network based on STANAG 4586 • Sensor dependencies are modeled as an acyclic graphs • Can learn the structure and parameters from raw data • Each node has a probability distribution function with respect to its parents • Raw values of sensor readings are discretized and plugged in • Health nodes (H_*) give instantaneous probability of sensor being faulty – – Health Nodes Sensor Nodes A real value between 0 to 1 A value above or below a threshold trigger an alert 12/3/2013 © 2013 Real-time Innovations, Inc. 12
  • 13.
    IMU-in-the-loop Simulation (FailureScenarios) 1 Sub-Scenario #1 High Engine Thrust Replay Starts [0:00] Replay Ends [5:00] (Change Pitch and Roll Physically) 2 Replay Starts [0:00] Sub-Scenario #2 Very Low Engine Thrust @ [1:30] (Change Pitch and Roll Physically) Engine Catches Fire [0:57] 12/3/2013 Very Low Engine Thrust [1:30] © 2013 Real-time Innovations, Inc. Replay Ends [5:00] 13
  • 14.
    Extensibility Facilitates SystemEvolution • Evolution – Hardware evolves  Software evolves  Data-types evolve – Evolved system components must continue to communicate with unevolved components – E.g., 2d point (X, Y) evolves to 3d point (X, Y, Z) – New radar that produces 3d points must work with old display that expects 2d • Solution: OMG Extensible and Dynamic Topic Types Specification struct EngineOperatingStates { double timestamp; string vehicleId; //@key long engineStatus; double engineSpeed; double enginePower; double exhaustGasTemperature; double engineBodyTemperature; double outputPower; long fireDetectionSensor; };// Extensibility(EXTENSIBLE) 12/3/2013 struct EngineOperatingStatesSensorsHealth { double p_engineSpeedSensor_good; double p_engineSpeedSensor_bad; double p_exhaustGasTempSensor_good; double p_exhaustGasTempSensor_bad; double p_fireDetectionSensor_good; double p_fireDetectionSensor_bad; . . . };// Extensibility(EXTENSIBLE) © 2013 Real-time Innovations, Inc. 14
  • 15.
    Take Home Points •Bayesian Networks – A flexible and capable platform for fault detection and diagnosis – Fine-grain fault detection within a subsystem and across subsystems – Large number of COTS tools • RTI Connext DDS – Has small footprint • Runs on a ARM processors (700 MHz, 256MB RAM) – Has low latency • Supports hardware-in-the-loop simulation as well as Smart Systems – Standards-based support for system evolution 12/3/2013 © 2013 Real-time Innovations, Inc. 15
  • 16.
    Future Work 1. ExpandedCyber Situational Awareness – Using independent ground-based sensors 2. Integrate portable and truly interoperable airborne software components – Use RTI Transport Services Segment (TSS) implementation 3. Semantic Interoperability – – 12/3/2013 Use UAS Control Segment (UCS) Protocol Interoperability using DDS (RTPS) © 2013 Real-time Innovations, Inc. 16
  • 17.
    Thank You! 12/3/2013 © 2013Real-time Innovations, Inc. 17