DDS: The data-centric future beyond message-based integration

1,710 views

Published on

High-level introduction to the OMG Data Distribution Service (DDS) standard and how it provides values beyond what is possible with traditional messaging middleware such as JMS or AMQP.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,710
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
71
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

DDS: The data-centric future beyond message-based integration

  1. 1. DDS: The data-centric future beyond message-based integration 2010 NATO-IST Meeting, Madrid, Oct 2010 Gerardo Pardo-Castellote, Ph.D.The Global Leader in DDS Co-chair OMG DDS SIG Chief Technology Officer, Real-Time Innovations, Inc. © 2010 Real-Time Innovations, Inc.
  2. 2. Systems that interact with the Real World Must adapt to changing environment Cannot stop processing the information Live within world-imposed timing Beyond traditional interpretation of real-time© 2010 Real-Time Innovations, Inc. 2
  3. 3. Challenge:More Data, More Speed, More SourcesTRENDS: Growing Information Volume Lowering Decision Latency Increasing System Availability Accelerating technology insertion and deploymentNext-generation systems needs: Performance Scalability Robustness Integration & Evolution © 2010 Real-Time Innovations, Inc. 3
  4. 4. “Real World” Systems are integrated using a Data Model Grounded on the “physics” of the problem domain – Tied to the nature of the sensors and real objects in the system (vehicles, device types, …) Provides governance across disparate teams & organizations – Central authority can define data model necessary for interoperability Increased decoupling from use-cases and components – Avoids over constraining applications Open, Evolvable, Platform-Independent – The use-cases, algorithms might change between missions or versions of the system App App App Realizing this data-model requires a middleware infrastructure© 2010 Real-Time Innovations, Inc. 4
  5. 5. Family of Specifications 2008 2009 2010 2010 2011 UML Profile DDS for DDS DDS-STD-C++ Web-Enabled for DDS Lw CCM X-Types DDS-JAVA5 DDS App 2004 App App DDS Spec DDS 2006 DDS DDS Implementation DDS Implementation Implementation Interoperablity Network / TCP / UDP / IP© 2010 Real-Time Innovations, Inc. 5
  6. 6. DDS adopted by key programs DISR – Mandates DDS for Pub-Sub API – Mandates DDS-RTPS for Pub-Sub Interop US Navy Open Architecture – Mandates DDS for Pub-Sub SPAWAR NESI – Mandates DDS for Pub-Sub SOA European Air Traffic Control – DDS used to interoperate ATC centers UK Generic Vehicle Architecture – Mandates DDS for vehicle comm. – Mandates DDS-RTPS for interop.© 2010 Real-Time Innovations, Inc.
  7. 7. Key A&D Programs Have Adopted DDS Ship Self Defense System Air-Traffic Management Eurocontrol Reagan Class Aircraft Carrier UK, Germany, Spain, Combat Management System France, Italy, Switzerland Advanced Cockpit Ground Control Station Aegis Weapon System Predator and SkyWarrior Lockheed Martin UAS Radar, weapons, displays, C2 General Atomics Korea FFX Frigate Samsung-Thales AWACS Combat Management Radar System system© 2010 Real-Time Innovations, Inc. 7
  8. 8. Data-Centric Qos-Aware Pub-Sub Model Virtual, decentralized global data space Source Latitude Longitude Altitude (Key) UAV1 37.4 -122.0 500.0 UAV2 40.7 -74.0 250.0 UAV3 50.2 -0.7 2000.0 Persistence RecordingCRUD operations Service Service © 2010 Real-Time Innovations, Inc. 8
  9. 9. ShapesDemo Demo: Publish-Subscribe© 2010 Real-Time Innovations, Inc. 9
  10. 10. exampleDDS communications model Data Domain Data DomainNew Writer Participant Reader Participant “Alarm” Got new “Alarm”subscriber! data Offered Requested Listener QoS Listener QoS Participants scope the global data space (domain) Topics define the data-objects (collections of subjects) Writers publish data on Topics Readers subscribe to data on Topics QoS Policies are used configure the system Listeners are used to notify the application of events
  11. 11. ShapesDemo Demo: Real-Time Quality of Service Content filter Time-based filter History Deadline© 2010 Real-Time Innovations, Inc. 11
  12. 12. Real-Time Quality of Service (QoS) QoS Policy QoS Policy DURABILITY USER DATA User QoS HISTORY TOPIC DATA Volatility READER DATA LIFECYCLE GROUP DATA WRITER DATA LIFECYCLE PARTITION Presentation LIFESPAN PRESENTATION Infrastructure ENTITY FACTORY DESTINATION ORDER RESOURCE LIMITS OWNERSHIP Redundancy Transport RELIABILITY OWNERSHIP STRENGTH Delivery TIME BASED FILTER LIVELINESS DEADLINE LATENCY BUDGET CONTENT FILTERS TRANSPORT PRIORITY© 2010 Real-Time Innovations, Inc. 12
  13. 13. 20X Faster than JMS / Broker-based solutions DDS is about 20X faster than JMS or Web Services 60,000 Throughput with a single publisher ( 2KB messages ) 50,000 Point-to-Point Updates per Second 45 40 DDS 40,000 1-1 1-10[1000s sample/s] 35 JMS 30,000 1-24 30 20,000 25 20 10,000 15 10 0 16 32 64 128 256 512 1024 5 Message/Data Size (bytes - without batching) 0 2 4 6 9 11 18 CPU load [%] DDS reliable multicast exhibits near perfect scalabilityPlatform: Linux 2.6 on AMD Athlon, Dual core, 2.2 GHz © 2010 Real-Time Innovations, Inc. 13
  14. 14. Realizing Performance & Scalability DDS Approach Others: Broker-based middleware JMS AMQP ESBs RTPS … DDS operates peer-to-peer, without brokers DDS uses RTPS, an Advanced Multi-Session protocol supporting Reliable Multicast© 2010 Real-Time Innovations, Inc. 14
  15. 15. DDS builds Higher quality, Lower TCO Systems Pre-built components address many challenging use-cases Presence Comp Discovery Comp Comp Content-Based Delivery Scalable pub-sub DDS Global Real-Time QoS Data Space Qos Monitoring Historical Cache Durable Data Messaging Event Database & Caching Processing Bridge SQL Availability Persistence Redundancy Redundancy & Failover Recording & Durability & Failover Security Guard Hooks© 2010 Real-Time Innovations, Inc. 15
  16. 16. ShapesDemo Data Persistence Persistence_svc A standalone service that persists data outside of the context of a DataWriter Data Data Writer Data Can be configured for: Writer Reader • Redundancy • Load balancing Data • Direct for performance Reader Global Data Space • Relay/Transactional • Redundant/ Fault-tolerant Persistence Persistence Service Service Permanent Permanent Storage Storage© 2010 Real-Time Innovations, Inc. 16
  17. 17. Integrating components to genericmiddleware technology Comp Comp Comp Comp Custom Integration Data Model Custom Mapping Middleware ArtifactsAkin to implementing an OO design on a Procedural Language:Requires mapping inheritance, encapsulation, exceptions, … © 2010 Real-Time Innovations, Inc. 17
  18. 18. Integrating components to data-centricmiddleware technology Comp Comp Comp Comp Standard API Data Model Standard Mapping(*) DDS Global Data SpaceNo custom mappings / code necessaryDirect support for data-centric actions: create, dispose, read/take © 2010 Real-Time Innovations, Inc. 18
  19. 19. Example: Message-Centric Legacy Define message-sets / handshakes Nothing to base filters, xforms on Component or Use-case based Error checking dev integration Schema, Self-describing data is verbose Limited QoS) Subscribe “My app 0x00000006 id=“AA123”Publish knows this 4141010203 x=float(45.6) means 0042366666 dispose.” 429DC y=float(78.9) © 2010 Real-Time Innovations, Inc. 19
  20. 20. Example: Modern Data-Centric Design Start with Data Model / Schemas / Meaning Data Schema Map this into XML; rows + cols id : string Express content-based filters (key) x : float Propagate data efficiently y : float Dispose New Update New SubscribePublish “AA123” “DL987” “AA123” “AA123” 65.4 56.7 45.6 X 32.1 89.0 78.9 © 2010 Real-Time Innovations, Inc. 20
  21. 21. Example: Modern Data-Centric Design Attach QoS to Data Model Data Schema Quality of Service id : string History (key) x : float Deadline y : float Time-Based Filter Once infrastructure understands data items, can attach QoS contracts to them SubscribePublish “Keep only the latest value” or “I need updates at this rate” make no sense unless per-item – Flight AA123 updates shouldn’t overwrite DL987, even if AA123 is updated more frequently – Update rate for one track shouldn’t change just because another track appeared © 2010 Real-Time Innovations, Inc. 21
  22. 22. ShapesDemo Demo: Some additional DDS capabilities Detecting presence History cache Deleting objects Detecting Applications© 2010 Real-Time Innovations, Inc. 22
  23. 23. Protocol optimized for disadvantaged networks Full peer-to-peer protocol – No required brokers or servers Adaptable via Qos – Reliability, timeouts, message priority Native multicast support DDS Interoperability Wire – Fully uses transport multicast, if available Protocol adopted in 2007 – Handles reliability, avoids duplicates Supports disconnected media – Based on UDP robust to disconnects Efficient data encapsulation – Binary CDR is 20 X better than XML/SOAP Built-in availability and durability – Durable & Persistent data – Historical cache – Failover support© 2010 Real-Time Innovations, Inc.
  24. 24. Summary Real-World Systems & Systems of Systems facing information volume, velocity, and mgmt. challenges Common solution is integration around a Data Model DDS is a family of OMG specifications that directly supports data-centric publish-subscribe communications DDS includes portable API’s for C, C++, Java, etc. and an Interoperable Wire Protocol Use of DDS results in reduced programming, decreased cost, and lowered risk Cost and Interoperability are the key drivers© 2010 Real-Time Innovations, Inc. 24
  25. 25. Thank You© 2010 Real-Time Innovations, Inc. 25

×