High-Performance Interoperable
Architecture for Information
Dominance




Gordon A. Hunt – RTI Webinar
Chief Engineer, RTI • UCS WG Sub-Committee Chair • Commander USN-R
Agenda

• High Performance
  – It really is about time…
• Interoperable Architecture
  – What is Interoperability
  – Architecture foundations
• [for] Information Dominance
  – Putting it all together
  – Right time – right data – right place
So, a little about Performance.

     Measure the right thing
      for the right reason.
Performance
• What can we measure?
  –   Message rates
  –   Numbers of messages
  –   Size of messages
  –   Time to process messages
  –   Number of clients, number of servers
  –   Throughput of a broker/server

• Do the measurements matter?
  – Consider that “Messages” are about something.
  – What I really care about, is how long it takes to know
    that something.
Performance is often Defined by…
       Point-to-point, sockets, RPC, RMI
       Concerned with an applications individual throughput/latency
       Bottleneck at the slowest application
       Little correlation between bus/fabric speeds and system performance

                                    Centralized, DB, ESBs
                                    Measure the aggregate performance
                     Broker         Replication rates, transaction rates
                      ESB           Number of supported clients
                     DBMS
                                    Generally not measure client
                                     performance

                       Decentralized, Data Centric
                       Measures data as presented to the application
                       Measures data update rates (1-1, 1-many, etc.)
                       Measures time to get the consistent state
Performance Does Matter
• Impacts Scale
   – How big is the System-of Systems going to be?
   – Physically can‟t send all the data everywhere.
   – Can‟t revisit existing application when adding new capability.

• Impacts flexibility of Implementation
   – Not arbitrarily constrained to messages rates/sizes.
   – Capability and distribution can be added.

• However, the following is required
   – Applications need to manage their performance requirements
   – Applications need to manage their behavior explicitly
Application Behavior Concerns
• What data is valid
   – Do I have the current value(s), from everyone

• How much of the data do I need
   – Reliably, only get me date that changes by a little bit
   – Send me data no faster than this rate

• What happens if
   – I expected an update to data at …, what do you do.
   – I want an acknowledgement, wait for this long, for this many

• I‟m too busy
   – Old data that I am just getting around to looking at
   – Are the updates I‟m about to send even valid anymore?
Managed Performance for
 Information Dominance
         Application                            Application
      Quality of Service                     Quality of Service
    (Desired Behaviors,..)                 (Desired Behaviors,..)




             Integration Bus for Information Dominance

Operational Systems / Tactical Systems   Information Technology (IT)
                                           Command & Control (C2)
       DATA-IN_MOTION                        DATA-AT-REST
An Integration Capability that
• Supports Integration of applications and
  delivery of consistent behaviors when
  –   Environment is ADHOC
  –   Performance concerns are mixed
  –   Scale is unknown
  –   Technologies are mixed

• Enables Information Dominance via
  – Architectural approach that supports a rigorous yet
    flexible integration methodology that is interoperable
    across data form/function boundaries
Interoperable Architecture

 How Do We „Do‟ Interoperability?
What does the Architecture look like?
Current Approaches

• Protocol Definitions & Standards
  – Tell me the messages


• Open Architecture Mandates
  – Interoperability on Commonality


• Service Oriented Architecture
  – Stateless services
Is Current Practice Working
• Recent studies have shown a growth in interoperability
  policy issuance in DoD
   – Thousands of pages of directives, instructions, and mandates
   – Numerous standards and architecture bodies in the DoD

• No Correlation between Increased Interoperability and
  Standards
   – Standards are necessary, but not sufficient for interoperability

• Conventional means of developing platform, unit
  command, and theater architectures are complex,
  manpower intensive, and time consuming.
   – Achieving Interoperability increases complexity
   – Complexity of systems-of-systems not understood or well
     managed
   – Can‟t make complexity go away, just move where it is
Pause: What are we Trying to
            Achieve?
Driver
Interchangeability   To put each of (two things) in the place of the other, or to be used in place of
                     each other.


Integratability      To form, coordinate, or blend into a functioning or unified whole. To incorporate
                     into a larger, functioning or unified whole.


Replaceability       One thing or person taking the place of another especially as a substitute or
                     successor.


Extensibility        The ability to add new components, subsystems, and capabilities to a system.


Interoperability     The ability of systems, units, or forces to provide services to and accept services
                     from other systems, units, or forces, and to use the services so exchanged to
                     enable them to operate effectively together.


 Open Architecture Requires Interoperability at a Higher Level
                    Than Key Interfaces.
Levels of Interoperability
            -- Technical --
System Behavior                Non-Functional Need
Communication protocol             Interchangeability,
for exchanging                        Integrateability,
data. Bits & Bytes                      Replaceability,
are exchanged in                          Extensibility,
an unambiguous                        and Meaningful
manner.            Technical           Interoperability
Levels of Interoperability
            -- Syntactic --
System Behavior                Non-Functional Need
Common structure                   Interchangeability,
or data format                        Integrateability,
defined for                             Replaceability,
                   Syntactic
exchanging                                Extensibility,
information.
                                      And Meaningful
This format
                   Technical           Interoperability
must be
unambiguously
defined.
Levels of Interoperability
           -- Semantic --
System Behavior               Non-Functional Need
Meaning of data   Semantic
                                  Interchangeability,
is exchanged                         Integrateability,
via common                             Replaceability,
                  Syntactic
information                              Extensibility,
model.                               And Meaningful
                                      Interoperability
The meaning       Technical
of information
is shared and
unambiguously
defined.
What‟s the Difference?
• Semantic definition captures concepts of
  – Structure (Content)
  – Relationships (Context)
  – Time (Behaviors)
• This makes the state of the system and the
  application‟s data
  – Explicit
  – Directly Observable
  – Manageable


• Everything has state…
Why is this hard?

 Consider a measure of
complexity on application
development approaches.
How to Architect Systems to
      Achieve Interoperability
1. Application Centric: Building Interoperable Apps
   – Application manages Technical, Syntactic, and Semantic Interoperability


2. Message Centric: Building Interoperable Messaging Systems
   – Delegates Syntactic Interoperability to Messaging Middleware
   – Application manages Semantic Interoperability


3. Data Centric: Building Truly Data Centric Systems
   – Delegates Semantic Interoperability to Middleware
   – Open Architecture Requires a Data Centric Approach.




                                                                               19
Application Centric Development
 • Scales O(n) only if
     – fully known, distributed, and homogeneous
       knowledge of interfaces and state

                                       Data State: Color denote uniqueness

Node/Service




                 Interface: Colors denote uniqueness
                                                                       20
Application Centric Development

• Scales O(n2) only if
  – each System invokes the Specified Interface of the
    System that it wishes to communicate with and no
    application state
  – n is the number of interfaces




                                                         21
Application Centric Development

• Scales O(n3)
  – Each system must understand the interaction
    patterns and the remote data state….
  – n is the number of data states




                                                  22
Application Centric Development

• Small systems aren‟t the problem, and
  they give a false sense of hope.
  – This is ~3.5 time more „complex‟ than 2 nodes




                                                    23
Application Centric Development

• O(n3) Scaling
  – ~8 times more
    complex




                                  24
Message Centric Development

• Delegate syntactic
  interoperability
  – Scales O(n2)
  – State is still in Apps




                               25
Data-Centric Development
• Delegate to middleware
  Syntactic and Semantic
  Interoperability
  – Scales O(n)
  – State synced with middleware




  – State IN the middleware
    and infrastructure
  – Data-Centric Pub/Sub

                                   26
An Interoperable Architecture

    HAS a Data-Centric bus and is
       Semantically Defined.
What is State?
• Things have attributes and
  characteristics
   – The meeting will run 1:00–2:00 in the
                                              “State” (“data”) is a
     conference room.                         snapshot of these
   – My friend‟s phone number is 555-1234
     and he‟s currently grooming his cat.     attributes and
   – The car is blue and is traveling north
     from Sunnyvale at 65 mph.
                                              characteristics.
   – The UAS is performing this mission,
     with these goals and resources.
                                              Best Practice:
…whether they exist in the real               operate on
world, in the computer, or both
                                              state, not dialogs
…whether or not we observe or                 about state.
acknowledge them
Data-Centric is Explicit
       Understanding of State



Weather ‘Client’                      Weather Service
-   Asks for a Weather Report         -   Stateless Service
-   Get the current prediction        -   Provides current weather
                                          only when asked
              State
              -   The fact that I asked,
              -   Where, at a certain time
              -   ‘Somebody’ has to keep it current…
Example: Data-Centric FlightPlan
          Start with a Semantic understanding of FlightPlans
          Content is understood – can represent data efficiently
          Context is understood, know what‟s what….



             Dispose        New         Update        New




                                                                   Subscribe
Publish




            “AA123”      “DL987”       “AA123”      “AA123”
                         65.4          56.7         45.6
               X         32.1          89.0         78.9
Example: Data-Centric FlightPlan
                                      Quality of Service
                 FlightPlan
                FlightPlan
               FlightPlan
              FlightPlan             History
                                     Deadline
                                     Time-Based Filter
           • Once infrastructure understands objects, can attach
             behavior (QoS) contracts to them
           • Data-in-motion behavior includes time perspective




                                                                          Subscribe
Publish




           • “Keep only the latest value” or “I need updates at
             this rate” make no sense unless per-object
              – Flight AA123 updates shouldn‟t overwrite DL987,
                even if AA123 is updated more frequently
              – Update rate for one track shouldn‟t change just because
                another track appeared
[for] Information Dominance

  Making it Happen, Steps you
           can take…



                                32
Challenges for Information
           Dominance
• Consistent Application Performance
  – Regardless of Scale
  – Numbers of nodes, amount of data, data rates…
• Dynamic systems‟ topology
  – No system always on, nodes come and go
  – Capabilities derived from rapid integration
• Disparate Technologies
  – Different deployment concerns
  – Different interaction patterns and behaviors
• Data-in-Motion and Data-at-Rest
  – Making at all actionable at the right place & time
Data-Centric Infrastructure
Supports Information Dominance
            Application                            Application
          Quality of Service                     Quality of Service
       (Time, Performance,..)                 (Time, Performance,..)




                Integration Bus for Information Dominance

   Operational Systems / Tactical Systems   Information Technology (IT)
                                              Command & Control (C2)
          DATA-IN_MOTION                        DATA-AT-REST
RTI Connext :
    Edge-to-Enterprise Real-Time SOA
                          Connext Integrator
                 Operational Systems           Information
                                             Technology (IT)




             Connext   Connext    Connext
              Micro     DDS      Messaging




Facilitates cross-organizational integration:
•    Well-defined and discoverable information models;
•    Standard data-centric operations
•    Standard wire interoperability protocol
•    Mediation: integrate with any interface, expose via any interface
RTI Connext™: A Data-Centric
       Infrastructure
                                                                Discrete OT & IT
  Small Device                             General-Purpose       Apps/Systems
                         DDS Apps
     Apps                                  Real-Time Apps

  Pub/Sub API          Pub/Sub API            Messaging API        Adapters

   Connext               Connext               Connext             Connext
    Micro                 DDS                 Messaging           Integrator

RTI DataBus™

        Administration              Recording             Federation

          Monitoring                 Replay             Transformation

            Logging             Visualization            Persistence

                 Common Tools and Infrastructure Services
Summary
• Information Dominance is enabled by
  – Access to the right information
  – Managed expectations with regards to performance
• Access to the right information is facilitated by
  – The right architecture and description of data
  – Content, Context, and Behavior
• Leveraging the data is facilitated by
  – Having that data presented to the application in the
    right form and function
  – Decoupling the applications view from the
    infrastructures management of the state
Where is State
Point-to-point, sockets, RPC, RMI
State is in the applications
Each application maintains its view


                              Centralized, DB, ESBs
                              State is in the Database
               Broker         Managed interactions with state
                ESB
               DBMS




                 Decentralized, Data Centric
                 State is in the bus
                 Stateless clients/services
                 State has explicit properties to manage its behavior
Download
Connext
Free Trial
NOW




 www.rti.com/downloads

High-Performance Interoperable Architecture for Information Dominance

  • 1.
    High-Performance Interoperable Architecture forInformation Dominance Gordon A. Hunt – RTI Webinar Chief Engineer, RTI • UCS WG Sub-Committee Chair • Commander USN-R
  • 2.
    Agenda • High Performance – It really is about time… • Interoperable Architecture – What is Interoperability – Architecture foundations • [for] Information Dominance – Putting it all together – Right time – right data – right place
  • 3.
    So, a littleabout Performance. Measure the right thing for the right reason.
  • 4.
    Performance • What canwe measure? – Message rates – Numbers of messages – Size of messages – Time to process messages – Number of clients, number of servers – Throughput of a broker/server • Do the measurements matter? – Consider that “Messages” are about something. – What I really care about, is how long it takes to know that something.
  • 5.
    Performance is oftenDefined by… Point-to-point, sockets, RPC, RMI Concerned with an applications individual throughput/latency Bottleneck at the slowest application Little correlation between bus/fabric speeds and system performance Centralized, DB, ESBs Measure the aggregate performance Broker Replication rates, transaction rates ESB Number of supported clients DBMS Generally not measure client performance Decentralized, Data Centric Measures data as presented to the application Measures data update rates (1-1, 1-many, etc.) Measures time to get the consistent state
  • 6.
    Performance Does Matter •Impacts Scale – How big is the System-of Systems going to be? – Physically can‟t send all the data everywhere. – Can‟t revisit existing application when adding new capability. • Impacts flexibility of Implementation – Not arbitrarily constrained to messages rates/sizes. – Capability and distribution can be added. • However, the following is required – Applications need to manage their performance requirements – Applications need to manage their behavior explicitly
  • 7.
    Application Behavior Concerns •What data is valid – Do I have the current value(s), from everyone • How much of the data do I need – Reliably, only get me date that changes by a little bit – Send me data no faster than this rate • What happens if – I expected an update to data at …, what do you do. – I want an acknowledgement, wait for this long, for this many • I‟m too busy – Old data that I am just getting around to looking at – Are the updates I‟m about to send even valid anymore?
  • 8.
    Managed Performance for Information Dominance Application Application Quality of Service Quality of Service (Desired Behaviors,..) (Desired Behaviors,..) Integration Bus for Information Dominance Operational Systems / Tactical Systems Information Technology (IT) Command & Control (C2) DATA-IN_MOTION DATA-AT-REST
  • 9.
    An Integration Capabilitythat • Supports Integration of applications and delivery of consistent behaviors when – Environment is ADHOC – Performance concerns are mixed – Scale is unknown – Technologies are mixed • Enables Information Dominance via – Architectural approach that supports a rigorous yet flexible integration methodology that is interoperable across data form/function boundaries
  • 10.
    Interoperable Architecture HowDo We „Do‟ Interoperability? What does the Architecture look like?
  • 11.
    Current Approaches • ProtocolDefinitions & Standards – Tell me the messages • Open Architecture Mandates – Interoperability on Commonality • Service Oriented Architecture – Stateless services
  • 12.
    Is Current PracticeWorking • Recent studies have shown a growth in interoperability policy issuance in DoD – Thousands of pages of directives, instructions, and mandates – Numerous standards and architecture bodies in the DoD • No Correlation between Increased Interoperability and Standards – Standards are necessary, but not sufficient for interoperability • Conventional means of developing platform, unit command, and theater architectures are complex, manpower intensive, and time consuming. – Achieving Interoperability increases complexity – Complexity of systems-of-systems not understood or well managed – Can‟t make complexity go away, just move where it is
  • 13.
    Pause: What arewe Trying to Achieve? Driver Interchangeability To put each of (two things) in the place of the other, or to be used in place of each other. Integratability To form, coordinate, or blend into a functioning or unified whole. To incorporate into a larger, functioning or unified whole. Replaceability One thing or person taking the place of another especially as a substitute or successor. Extensibility The ability to add new components, subsystems, and capabilities to a system. Interoperability The ability of systems, units, or forces to provide services to and accept services from other systems, units, or forces, and to use the services so exchanged to enable them to operate effectively together. Open Architecture Requires Interoperability at a Higher Level Than Key Interfaces.
  • 14.
    Levels of Interoperability -- Technical -- System Behavior Non-Functional Need Communication protocol Interchangeability, for exchanging Integrateability, data. Bits & Bytes Replaceability, are exchanged in Extensibility, an unambiguous and Meaningful manner. Technical Interoperability
  • 15.
    Levels of Interoperability -- Syntactic -- System Behavior Non-Functional Need Common structure Interchangeability, or data format Integrateability, defined for Replaceability, Syntactic exchanging Extensibility, information. And Meaningful This format Technical Interoperability must be unambiguously defined.
  • 16.
    Levels of Interoperability -- Semantic -- System Behavior Non-Functional Need Meaning of data Semantic Interchangeability, is exchanged Integrateability, via common Replaceability, Syntactic information Extensibility, model. And Meaningful Interoperability The meaning Technical of information is shared and unambiguously defined.
  • 17.
    What‟s the Difference? •Semantic definition captures concepts of – Structure (Content) – Relationships (Context) – Time (Behaviors) • This makes the state of the system and the application‟s data – Explicit – Directly Observable – Manageable • Everything has state…
  • 18.
    Why is thishard? Consider a measure of complexity on application development approaches.
  • 19.
    How to ArchitectSystems to Achieve Interoperability 1. Application Centric: Building Interoperable Apps – Application manages Technical, Syntactic, and Semantic Interoperability 2. Message Centric: Building Interoperable Messaging Systems – Delegates Syntactic Interoperability to Messaging Middleware – Application manages Semantic Interoperability 3. Data Centric: Building Truly Data Centric Systems – Delegates Semantic Interoperability to Middleware – Open Architecture Requires a Data Centric Approach. 19
  • 20.
    Application Centric Development • Scales O(n) only if – fully known, distributed, and homogeneous knowledge of interfaces and state Data State: Color denote uniqueness Node/Service Interface: Colors denote uniqueness 20
  • 21.
    Application Centric Development •Scales O(n2) only if – each System invokes the Specified Interface of the System that it wishes to communicate with and no application state – n is the number of interfaces 21
  • 22.
    Application Centric Development •Scales O(n3) – Each system must understand the interaction patterns and the remote data state…. – n is the number of data states 22
  • 23.
    Application Centric Development •Small systems aren‟t the problem, and they give a false sense of hope. – This is ~3.5 time more „complex‟ than 2 nodes 23
  • 24.
    Application Centric Development •O(n3) Scaling – ~8 times more complex 24
  • 25.
    Message Centric Development •Delegate syntactic interoperability – Scales O(n2) – State is still in Apps 25
  • 26.
    Data-Centric Development • Delegateto middleware Syntactic and Semantic Interoperability – Scales O(n) – State synced with middleware – State IN the middleware and infrastructure – Data-Centric Pub/Sub 26
  • 27.
    An Interoperable Architecture HAS a Data-Centric bus and is Semantically Defined.
  • 28.
    What is State? •Things have attributes and characteristics – The meeting will run 1:00–2:00 in the “State” (“data”) is a conference room. snapshot of these – My friend‟s phone number is 555-1234 and he‟s currently grooming his cat. attributes and – The car is blue and is traveling north from Sunnyvale at 65 mph. characteristics. – The UAS is performing this mission, with these goals and resources. Best Practice: …whether they exist in the real operate on world, in the computer, or both state, not dialogs …whether or not we observe or about state. acknowledge them
  • 29.
    Data-Centric is Explicit Understanding of State Weather ‘Client’ Weather Service - Asks for a Weather Report - Stateless Service - Get the current prediction - Provides current weather only when asked State - The fact that I asked, - Where, at a certain time - ‘Somebody’ has to keep it current…
  • 30.
    Example: Data-Centric FlightPlan Start with a Semantic understanding of FlightPlans Content is understood – can represent data efficiently Context is understood, know what‟s what…. Dispose New Update New Subscribe Publish “AA123” “DL987” “AA123” “AA123” 65.4 56.7 45.6 X 32.1 89.0 78.9
  • 31.
    Example: Data-Centric FlightPlan Quality of Service FlightPlan FlightPlan FlightPlan FlightPlan History Deadline Time-Based Filter • Once infrastructure understands objects, can attach behavior (QoS) contracts to them • Data-in-motion behavior includes time perspective Subscribe Publish • “Keep only the latest value” or “I need updates at this rate” make no sense unless per-object – Flight AA123 updates shouldn‟t overwrite DL987, even if AA123 is updated more frequently – Update rate for one track shouldn‟t change just because another track appeared
  • 32.
    [for] Information Dominance Making it Happen, Steps you can take… 32
  • 33.
    Challenges for Information Dominance • Consistent Application Performance – Regardless of Scale – Numbers of nodes, amount of data, data rates… • Dynamic systems‟ topology – No system always on, nodes come and go – Capabilities derived from rapid integration • Disparate Technologies – Different deployment concerns – Different interaction patterns and behaviors • Data-in-Motion and Data-at-Rest – Making at all actionable at the right place & time
  • 34.
    Data-Centric Infrastructure Supports InformationDominance Application Application Quality of Service Quality of Service (Time, Performance,..) (Time, Performance,..) Integration Bus for Information Dominance Operational Systems / Tactical Systems Information Technology (IT) Command & Control (C2) DATA-IN_MOTION DATA-AT-REST
  • 35.
    RTI Connext : Edge-to-Enterprise Real-Time SOA Connext Integrator Operational Systems Information Technology (IT) Connext Connext Connext Micro DDS Messaging Facilitates cross-organizational integration: • Well-defined and discoverable information models; • Standard data-centric operations • Standard wire interoperability protocol • Mediation: integrate with any interface, expose via any interface
  • 36.
    RTI Connext™: AData-Centric Infrastructure Discrete OT & IT Small Device General-Purpose Apps/Systems DDS Apps Apps Real-Time Apps Pub/Sub API Pub/Sub API Messaging API Adapters Connext Connext Connext Connext Micro DDS Messaging Integrator RTI DataBus™ Administration Recording Federation Monitoring Replay Transformation Logging Visualization Persistence Common Tools and Infrastructure Services
  • 37.
    Summary • Information Dominanceis enabled by – Access to the right information – Managed expectations with regards to performance • Access to the right information is facilitated by – The right architecture and description of data – Content, Context, and Behavior • Leveraging the data is facilitated by – Having that data presented to the application in the right form and function – Decoupling the applications view from the infrastructures management of the state
  • 38.
    Where is State Point-to-point,sockets, RPC, RMI State is in the applications Each application maintains its view Centralized, DB, ESBs State is in the Database Broker Managed interactions with state ESB DBMS Decentralized, Data Centric State is in the bus Stateless clients/services State has explicit properties to manage its behavior
  • 39.

Editor's Notes

  • #14 Using a Quality Attribute Methodology, that supports the Business, Non-functional Requirements
  • #24 As systems are added, the number of Interfaces that must be considered increases exponentiallyAs systems are added, the number of Data States that must be considered increases exponentiallyInteroperability between 3 systems is ~3.5 times more complex than between 2 systems