The Real-Time
Middleware Experts
Easing Integration of Large-Scale
Real-Time Systems with DDS
Rick Warren, Principal Engin...
Introduction
 Rick Warren
– Principal Engineer; with RTI since 2001
– Engineering organization; previously Professional S...
RTI: The Global Leader in DDS
 70%+ worldwide DDS market share
 First with…
– DDS API (2004)
– DDS-RTPS interoperability...
© 2010 Real-Time Innovations, Inc. 4
Customer Application Highlights
Weapon System
Lockheed Martin Aegis
Radar, weapons, d...
What do I mean by “large system”?
 Systems of systems
– Modular, hierarchical design
– Legacy components,
subsystems
– Mu...
What matters to integrators of large systems?
 Governance (including over security)
– What information will be exchanged?...
Part 1: Architecture
(we’ll get to technology later)
© 2010 Real-Time Innovations, Inc. 7
System
Component
Component
Component
Class
A Modest Proposition
© 2009 Real-Time Innovations, Inc. 8
State
Behavior
Class
...
Subsystem 2
P P
PP P
Subsystem 1
P P
PP P
Schematic of a Composed System
 Subsystems may have different network environme...
Schematic of a Composed System
 Wash, rinse, repeat
Subsystem 1
P P
PP P
Subsystem 2
P P
PP P
Router/G
ateway
Router/G
at...
Same data model?
Same network env.?
Same lifecycle?
Behavior unaffected?
Understandable?
Apples and Oranges
P P
PP P
Subsy...
Nothing New Under the Sun
© 2009 Real-Time Innovations, Inc. 12
Subsystem
Data Space
Subsystem
Data Space
Subsystem
Data S...
Nothing New Under the Sun
© 2010 Real-Time Innovations, Inc. 13
U.S. Combat
Data Space
Allied Combat
Data Space
U.S. C4I
D...
Nothing New Under the Sun
© 2010 Real-Time Innovations, Inc. 14
NYC Trading
Data Space
London Trading
Data Space
Tokyo Tra...
Nothing New Under the Sun
© 2010 Real-Time Innovations, Inc. 15
Generation
Data Space
Substation #1
Data Space
Substation ...
Architecture Recap
 Solve big problems
like you solve small
ones
– Break them into
pieces
– Define interfaces
between pie...
Fault Tolerance and Scalability
 Fault tolerance: Router/gateway can’t be single point of
failure. 3 answers:
1. Persiste...
Part 2: DDS
© 2010 Real-Time Innovations, Inc. 18
DDS Is Open Architecture
 OMG standard for data-centric
publish-subscribe communication
 Vendor independent
– API for po...
Example: Data-Centric Track Data
© 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 20
Publish
Subscribe
Data Schema
...
Example: Data-Centric Track Data
© 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 21
Publish
Subscribe
Data Schema
...
Analysis by U.S. Armed Services
NESI, P1190 rev 3, http://nesipublic.spawar.navy.mil/nesix/Frames/P1190
© 2010 Real-Time I...
How Does DDS Stack Up?
 Governance
– Structural SLA: data type
– Behavioral SLA: delivery QoS
– Security
– Problem
detect...
DDS Scalability: Two Aspects
1. Application data
– How does performance fall off as # participants, writers,
readers incre...
1. DDS Scalability: Application Data
 DDS-RTPS reliable multicast scales at least:
– …to hundreds of readers per writer
–...
2. DDS Scalability: P2P Discovery Scenarios
 Single domain, symmetric discovery
– Everyone discovers everyone else
– Easi...
Summary: Large-Scale P2P Scalability
 Experimental results: DDS-RTPS allows participant to
talk to ≥ 1-2K others in singl...
Geographic Scalability: WAN Transport
 Site-to-site comms need DDS across WAN(s)
– Including untrusted networks
– Includi...
Geographic Scalability: DIL Transport
 DDS is also great on disadvantaged networks
– DDS QoS provide tunable behavior, pe...
Geographic Scalability: DIL Transport
 DDS is also great on disadvantaged networks
– DDS QoS provide tunable behavior, pe...
RTI Routing Service:
Integrating Non-DDS Applications
 Adapt to other protocols and interfaces
 Adapter SDK with customi...
Recap
 Large systems usually composed of smaller systems
– Developed independently
– …With different network environments...
Recap
 DDS is uniquely qualified to meet these needs
– Strong SLAs for governance
– Multiple topologies for fault-toleran...
Next Steps
 Free software downloads
– www.rti.com/downloads
– Product evaluation:
full RTI Professional Edition
• Free tr...
Upcoming SlideShare
Loading in …5
×

Easing Integration of Large-Scale Real-Time Systems with DDS

1,037 views
926 views

Published on

Webcast (sorry, audio not included) on system integration design patterns from July of 2010 pertaining mostly (but not exclusively) to Data-Distribution Service (DDS) technology.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,037
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
32
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • TRL 9:
    Actual system proven through successful mission operations. Actual application of the technology in its final form and under mission conditions, such as those encountered in operational test and evaluation (OT&E). Examples include using the system under operational mission conditions.
  • When I say “large” in this presentation, I’m primarily talking about complexity. I’m not talking primarily about a unified simple design that happens to have lots of participants in it.

    Same principles apply to defense systems, financial systems, power systems, industrial automation, etc.
  • Care about lots of the same things as any system designers — functionality, performance, … — but care about certain things much more.

    Isolation relates to governance: making sure integration doesn’t violate SLAs
  • This will eventually be a talk about DDS, but we’ll get to that later
  • Integrating two subsystems with different data spaces is not the same as joining them into one data space.

    You will be tempted to just mush things together. (Just connect the network cable; it’s easy, right?) Beware that temptation.
    Are the different subsystems using the same structural and behavioral data model?
    Are the network environments the same?
    Do they evolve together?
    Suppose the answers are both Yes.
    Will they continue to be the same over time as the composed system evolves?
    Will the system behave the same when all the data is going to twice as many consumers?
    Can one team of people understand the design and operation of a now-much-more-complex single subsystem?
  • Same principles apply in every industry
  • Same principles apply in every industry
  • Same principles apply in every industry
  • Same principles apply in every industry
  • 2 problem areas brought up before that we haven’t discussed yet.
    These will be bridge to technology-specific discussion in 2nd half.
  • From the beginning, the data stream is associated with the schema of the data that will be propagated on that stream. Your applications already have some expectations; if you express those to a data-centric infrastructure, it can help you. For example, you can use this schema to automatically transform data into other formats. (This is how the Routing Service and Web Integration Service work.) The infrastructure can also dissect your data to filter on content (for example “give me updates where x > 5”).

    “Key” means “this field establishes the identity of a unique object.” Like the key in a relational database table. In DDS, can be any number of fields of any type(s).

    New track you’ve never seen before. Notice that since type is already known, only need to send field values, not field names or types.
    Update to a track you’ve already seen
    Another new track – notice that the key is different
    A track you’ve seen before has gone away
  • Test results from lab of hundreds of multi-core Linux machines connected by gigabit Ethernet
  • Easing Integration of Large-Scale Real-Time Systems with DDS

    1. 1. The Real-Time Middleware Experts Easing Integration of Large-Scale Real-Time Systems with DDS Rick Warren, Principal Engineer rick.warren@rti.com
    2. 2. Introduction  Rick Warren – Principal Engineer; with RTI since 2001 – Engineering organization; previously Professional Services – Together with CTO, lead RTI’s Object Management Group (OMG) initiatives – Passionate about systems integration, standards, and usability  This Presentation – Large-scale design patterns for integrating heterogeneous distributed systems – Part 1: Architecture and requirements – Part 2: Addressing Part 1 with OMG DDS © 2010 Real-Time Innovations, Inc. 2
    3. 3. RTI: The Global Leader in DDS  70%+ worldwide DDS market share  First with… – DDS API (2004) – DDS-RTPS interoperability protocol (2007) — based on RTI’s native protocol  Most mature solution – 12+ years of commercial availability – Technology Readiness Level (TRL) 9: successful mission operations – Diverse range of industries: defense, finance, medical, industrial control, power generation, communications – 300+ commercial customers, 100+ research projects – 100,000+ licensed copies  Leading OMG standardization – Board of Directors member – Co-chair DDS Special Interest Group – Chair several DDS standard finalization, revision task forces © 2010 Real-Time Innovations, Inc. 3
    4. 4. © 2010 Real-Time Innovations, Inc. 4 Customer Application Highlights Weapon System Lockheed Martin Aegis Radar, weapons, displays, C2 Unmanned Air System Boeing ScanEagle Sensors, ground station Control System Grand Coulee Dam: largest electricity producer in US Controls, high availability Driver Safety Volkswagen Vision systems, analysis, driver information systems Medical Imaging NMR and MRI Sensors, RF generators, user interface, control computers Automated Trading Automated Trading Desk (ATD, now Citigroup) Market data feed handlers, pricing engines, algorithmic trading applications
    5. 5. What do I mean by “large system”?  Systems of systems – Modular, hierarchical design – Legacy components, subsystems – Multiple technologies – Global scale  Decoupled subsystem lifecycles – Independent development – Independent deployment and use – Independent management – Independent revision and retirement  Multiple communities of interest – Different data interest, entitlements – Non-uniform levels of trust © 2009 Real-Time Innovations, Inc. 5 LAN DDS #1 (Initech) JMS (Dunder Mifflin) DDS #2 (Acme) Legacy (COBOL ‘R’ US) Satellite Links
    6. 6. What matters to integrators of large systems?  Governance (including over security) – What information will be exchanged? – Under what conditions will it be exchanged? – Who is allowed to exchange this information? – If these SLAs are violated, can the exchange be prevented? Can I be notified? – (In the past, what has occurred wrt these SLAs?)  Isolation – When I connect A and B, ensure they don’t break (each other) – When I disconnect A, ensure B doesn’t break – When I connect C, don’t change A or B  Scalability (like in any system,  Fault Tolerance but stakes are higher) © 2009 Real-Time Innovations, Inc. 6
    7. 7. Part 1: Architecture (we’ll get to technology later) © 2010 Real-Time Innovations, Inc. 7
    8. 8. System Component Component Component Class A Modest Proposition © 2009 Real-Time Innovations, Inc. 8 State Behavior Class  Fundamental design principles scale – Abstraction: Provide interface based on relevant concepts – Encapsulation: Hide internal implementation, communication – Composition: Combine existing capabilities into new ones
    9. 9. Subsystem 2 P P PP P Subsystem 1 P P PP P Schematic of a Composed System  Subsystems may have different network environments  Integration may have different network environment than subsystems themselves  Data may need to be transformed / cleansed as it moves among subsystems  Routing / gateway services will adapt data types / formats / protocols LAN LAN WAN Router/G ateway Router/G ateway Isolation Additional Governance
    10. 10. Schematic of a Composed System  Wash, rinse, repeat Subsystem 1 P P PP P Subsystem 2 P P PP P Router/G ateway Router/G ateway © 2010 Real-Time Innovations, Inc. 10
    11. 11. Same data model? Same network env.? Same lifecycle? Behavior unaffected? Understandable? Apples and Oranges P P PP P Subsystem 1 + Subsystem 2 P P PP P Subsystem 2 P P PP P Subsystem 1 P P PP P Router/ Gateway Router/ Gateway
    12. 12. Nothing New Under the Sun © 2009 Real-Time Innovations, Inc. 12 Subsystem Data Space Subsystem Data Space Subsystem Data Space Integration Data Space Router/ Gateway Router/ Gateway Router/ Gateway
    13. 13. Nothing New Under the Sun © 2010 Real-Time Innovations, Inc. 13 U.S. Combat Data Space Allied Combat Data Space U.S. C4I Data Space Integration Data Space Router/ Gateway Router/ Gateway Router/ Gateway Defense Industry
    14. 14. Nothing New Under the Sun © 2010 Real-Time Innovations, Inc. 14 NYC Trading Data Space London Trading Data Space Tokyo Trading Data Space Integration Data Space Router/ Gateway Router/ Gateway Router/ Gateway Financial Services Industry
    15. 15. Nothing New Under the Sun © 2010 Real-Time Innovations, Inc. 15 Generation Data Space Substation #1 Data Space Substation #2 Data Space Integration Data Space Router/ Gateway Router/ Gateway Router/ Gateway Power Industry
    16. 16. Architecture Recap  Solve big problems like you solve small ones – Break them into pieces – Define interfaces between pieces – Implement each piece – Integrate along the interfaces  Translation – Define subsystems – Implement them however you like • May have different requirements, therefore technology – Router/gateway defines/enforces interfaces © 2010 Real-Time Innovations, Inc. 16
    17. 17. Fault Tolerance and Scalability  Fault tolerance: Router/gateway can’t be single point of failure. 3 answers: 1. Persistent data survives system failures 2. Segmented/load-balanced configuration limits scope of failures 3. Redundancy allows continued service during failure  Scalability: How big can a subsystem be? 1. How big a subsystem can you understand, test, and debug? 2. How well does infrastructure scale? © 2010 Real-Time Innovations, Inc. 17
    18. 18. Part 2: DDS © 2010 Real-Time Innovations, Inc. 18
    19. 19. DDS Is Open Architecture  OMG standard for data-centric publish-subscribe communication  Vendor independent – API for portability – Network protocol for interoperability  8+ implementations – Commercial, open-source, and hybrid licenses  Supports heterogeneous systems – C, C++, C#/.NET, Java, Ada, … – Windows, Linux and other Unix, embedded, RTOS Real-Time Publish-Subscribe Protocol (DDS-RTPS) Middleware DDS API Cross-vendor portability Cross-vendor interoperability © 2010 Real-Time Innovations, Inc. 19
    20. 20. Example: Data-Centric Track Data © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 20 Publish Subscribe Data Schema x : float y : float id : string (key) New 45.6 78.9 “AA123” Update 56.7 89.0 “AA123” New 65.4 32.1 “DL987” Dispose “AA123” X Map this into XML; rows + cols Express content-based filters Propagate data efficiently
    21. 21. Example: Data-Centric Track Data © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 21 Publish Subscribe Data Schema x : float y : float id : string (key) Quality of Service Deadline Time-Based Filter History  Once infrastructure understands objects, can attach QoS contracts to them  “Keep only the latest value” or “I need updates at this rate” make no sense unless per-object – Flight AA123 updates shouldn’t overwrite DL987, even if AA123 is updated more frequently – Update rate for one track shouldn’t change just because another track appeared
    22. 22. Analysis by U.S. Armed Services NESI, P1190 rev 3, http://nesipublic.spawar.navy.mil/nesix/Frames/P1190 © 2010 Real-Time Innovations, Inc. 22 Adapted from NSWC-DD OA Documentation
    23. 23. How Does DDS Stack Up?  Governance – Structural SLA: data type – Behavioral SLA: delivery QoS – Security – Problem detection/prevention/notification – System monitoring and recording  Isolation – Prevent data “leaks” and side effects – Allow dynamic (dis-|re-)connection – Support independent integration and evolution  Fault tolerance – Data persistence – Segmented/load-balanced routing – Redundant routing  Scalability – WAN, DIL connectivity – Peer-to-peer communication – Brokered/managed communication – Built in – Built in – Products available; future standards – Built in – Products available – Built in: domains, partitions – Built in – Built in, esp. w/ DDS-XTypes spec – Built in – Built in if router uses DDS – Built in if router uses DDS – Products available; future standards – Highly scalable – Protocol support if necessary © 2010 Real-Time Innovations, Inc. 23
    24. 24. DDS Scalability: Two Aspects 1. Application data – How does performance fall off as # participants, writers, readers increase? – Any single writer/reader pair can saturate gig-E: achieving high aggregate throughput is not main issue – Interesting issue: Fan-out – number of readers per writer 2. Discovery – How many DDS applications can discover one another? – Interesting issue: How many applications need to discover one another?  Hard limits, if any, very dependent on DDS implementation, machine configuration, network, etc. – Following results based on RTI on modern commodity HW © 2010 Real-Time Innovations, Inc. 24
    25. 25. 1. DDS Scalability: Application Data  DDS-RTPS reliable multicast scales at least: – …to hundreds of readers per writer – …with very little degradation in throughput. – Larger testing facilities welcomed.  © 2010 Real-Time Innovations, Inc. 25 < 15% decrease 1~900 readers RTI Data Distribution Service 4.3 Red Hat EL 5.0, 32-bit 2.4 GHz processors Gigabit Ethernet UDP/IPv4 Reliable ordered delivery
    26. 26. 2. DDS Scalability: P2P Discovery Scenarios  Single domain, symmetric discovery – Everyone discovers everyone else – Easiest to configure; most challenging wrt scalability – RTI test results: 1,800 participants; 3.2M endpoint matches  Single domain, asymmetric discovery – Each participant only knows of certain others in its domain – Good for partitioning domains in which not everyone talks  Multiple domains – Greatest separation for subsystems that rarely exchange data • DDS-RTPS maps to different IP ports © 2009 Real-Time Innovations, Inc. 26 1 … n
    27. 27. Summary: Large-Scale P2P Scalability  Experimental results: DDS-RTPS allows participant to talk to ≥ 1-2K others in single symmetric domain  Implication: Compose arbitrarily large systems out of subsystems this size or smaller  Large (sub)systems often have requirements for: – Increased governance, e.g. over security boundaries – Data transformation – Protocol mediation – These typically require data brokering anyway © 2009 Real-Time Innovations, Inc. 27
    28. 28. Geographic Scalability: WAN Transport  Site-to-site comms need DDS across WAN(s) – Including untrusted networks – Including firewall, NAT traversal – Challenge: UDP may not be routable, especially if multicast  Standard DDS-RTPS protocol designed for layering atop diverse lower-level protocols – RTI supports: • Shared memory for performance on a single machine • UDP (unicast, multicast) for scalability, determinism w/in LAN • TCP for routing across WAN • (D)TLS for transport-level security • etc. • …simultaneously © 2010 Real-Time Innovations, Inc. 28
    29. 29. Geographic Scalability: DIL Transport  DDS is also great on disadvantaged networks – DDS QoS provide tunable behavior, performance – DDS-RTPS protocol encapsulates data efficiently w/ CDR © 2010 Real-Time Innovations, Inc. 29 Source: • SELEX-SI/ Consorzio SESM/ University of Naples • “Flexible Communication Among DDS Publishers and Subscribers” • July 2008, Real- Time Systems Workshop, Washington, DC 10x more compact than XML or JSON
    30. 30. Geographic Scalability: DIL Transport  DDS is also great on disadvantaged networks – DDS QoS provide tunable behavior, performance – DDS-RTPS protocol encapsulates data efficiently w/ CDR © 2010 Real-Time Innovations, Inc. 30 Source: • SELEX-SI/ Consorzio SESM/ University of Naples • “Flexible Communication Among DDS Publishers and Subscribers” • July 2008, Real- Time Systems Workshop, Washington, DC20x faster
    31. 31. RTI Routing Service: Integrating Non-DDS Applications  Adapt to other protocols and interfaces  Adapter SDK with customizable plugins – JMS – Socket – File  Take advantage of Routing Service capabilities – Data transformation and filtering – Data life-cycle management – Cross networks, transports and platforms – Dynamic discovery DDS Plug-in Adapters © 2010 Real-Time Innovations, Inc. 31
    32. 32. Recap  Large systems usually composed of smaller systems – Developed independently – …With different network environments – …And different data interest/entitlements  Not an accident: hierarchical design is a best practice at any scale – Subsystems managed by router/gateway • Restrict access to internal topics/flows • Mediate internal/external protocols • Transform/cleanse internal/external data types • Impose governance: enforce policy, attach monitoring – Inter-router integration space is itself a “subsystem” for further hierarchical composition © 2010 Real-Time Innovations, Inc. 32
    33. 33. Recap  DDS is uniquely qualified to meet these needs – Strong SLAs for governance – Multiple topologies for fault-tolerant subsystem isolation – High-performance, scalable, deterministic interoperability in various network environments – What other standard technology offers this combination? (Not JMS. Not CORBA. Not AMQP. Not web services. …) © 2010 Real-Time Innovations, Inc. 33
    34. 34. Next Steps  Free software downloads – www.rti.com/downloads – Product evaluation: full RTI Professional Edition • Free trial with comprehensive tutorial • Free licenses for research & academia • Includes: – Leading DDS implementation – Data recording – System monitoring and debugging – Integrates with JMS, databases, Microsoft Excel – More – Shapes demo: no coding required  Videos, webinars, and whitepapers – www.rti.com/resources © 2010 Real-Time Innovations, Inc. 34

    ×