Your SlideShare is downloading. ×
  • Like
The Data Distribution Service
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

The Data Distribution Service

  • 1,504 views
Published

This presentation introduces the key concepts at the foundation of DDS, the data distribution service for real-time systems. Wether you are a new to DDS or a relatively experienced user, you'll find …

This presentation introduces the key concepts at the foundation of DDS, the data distribution service for real-time systems. Wether you are a new to DDS or a relatively experienced user, you'll find this presentation a good source of information.

Published in Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,504
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
151
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The Data Distribution Service [For Real-Time Systems]OpenSplice DDS Angelo CORSARO, Ph.D. Chief Technology Officer OMG DDS Sig Co-Chair PrismTech angelo.corsaro@prismtech.com
  • 2. OpenSplice DDS Background
  • 3. Data Distribution Service For Real-Time Systems Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Introduced in 2004 to address the Data Distribution Real-Timeliness Scale challenges faced by a wide class of Defense and Aerospace ApplicationsOpenSplice DDS Key requirement for the Parallelism Determinism Throughput, Availability Scalability, Persistence, Security ☐ standard were to deliver very Systemic Signal Data Real-Time Information Near Real-Time Fault- Tolerant Information Complex Information high and predictable Processing Processing Management Processing Processing performance while scaling from embedded to ultra- Parallel Systems Distributed Systems large-scale deployments
  • 4. Data Distribution Service For Real-Time Systems Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Recommended by key Real-Timeliness Scale administration worldwide, e.g. DoD, MoD, EUROCAE, etc.OpenSplice DDS Widely adopted across Parallelism Determinism Throughput, Availability Scalability, Persistence, Security ☐ several different domains, Systemic Signal Data Real-Time Information Near Real-Time Fault- Tolerant Information Complex Information e.g., Automated Trading, Processing Processing Management Processing Processing Simulations, SCADA, Telemetry, etc. Parallel Systems Distributed Systems
  • 5. Some Use Cases Copyright  2011,  PrismTech  –    All  Rights  Reserved. Integrated Modular Vetronics Training & Simulation Systems Naval Combat SystemsOpenSplice DDS Air Traffic Control & Management Large Scale SCADA Systems High Frequency Auto-Trading
  • 6. Standards: What For? Copyright  2011,  PrismTech  –    All  Rights  Reserved. Common “Language” Data Model +OpenSplice DDS QoS Requirements Interoperability DDSI Wire Interoperability
  • 7. OpenSplice DDS DDS Application API Standard Standards: What For? Portability Copyright  2011,  PrismTech  –    All  Rights  Reserved.
  • 8. OpenSplice DDS D D BaS ics
  • 9. Data Distribution Service For Real-Time Systems DDS provides a Topic-Based Publish/ Copyright  2011,  PrismTech  –    All  Rights  Reserved. Subscribe abstraction based on: Data Reader Data Writer ☐ Topics: data distribution subject’sOpenSplice DDS Data Reader Data TopicD Writer DataWriters: data producers TopicA ☐ Data TopicB Reader Data Writer ☐ DataReaders: data consumers TopicC ... Data Data Writer Reader DDS Global Data Space
  • 10. Data Distribution Service For Real-Time Systems Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ DataWriters and DataReaders Data Reader are automatically and Data Writer dynamically matched by theOpenSplice DDS Data Reader Data TopicD DDS Dynamic Discovery Writer TopicA Data TopicB Reader Data ☐ A rich set of QoS allows to Writer TopicC ... control existential, temporal, Data Data and spatial properties of data Writer Reader DDS Global Data Space
  • 11. DDS Topics ☐ A Topic defines a class of “TVehicleDynamics” streams Copyright  2011,  PrismTech  –    All  Rights  Reserved. Name ☐ A Topic has associated a user defined extensible type and a Topic Typ set of QoS policies S DURABILITY, QoOpenSplice DDS e DEADLINE, ☐ The Topic name, type and QoS VehicleDynamics PRIORITY, defines the key functional and … non-functional invariants struct VehicleDynamics { IDL ☐ Topics can be discovered or long vid; locally defined long x; long y; long dx; long dy;}; #pragma keylist VehicleDynamics vid
  • 12. Topic Instances ☐ Each unique key value Copyright  2011,  PrismTech  –    All  Rights  Reserved. identifies a unique stream of data vid =701 DDS not only vid =809 struct VehicleStatus { ☐ Topic long vid;OpenSplice DDS long x; long y; long dx; long dy;}; demultiplexes “streams” #pragma keylist VehicleStatus vid vid =977 but provides also lifecycle Instances Instances information ☐ A DDS DataWriter can write multiple instances
  • 13. OpenSplice DDS Anatomy of a DDS Application Copyright  2011,  PrismTech  –    All  Rights  Reserved.
  • 14. Anatomy of a DDS Application [Scala API] Domain Domain Copyright  2011,  PrismTech  –    All  Rights  Reserved. val dp = DomainParticipant(domainId) Participant Publisher Topic SubscriberOpenSplice DDS DataWriter DataReader
  • 15. Anatomy of a DDS Application [Scala API] Domain Domain Copyright  2011,  PrismTech  –    All  Rights  Reserved. val dp = DomainParticipant(domainId) Participant Session // Create a Publisher / Subscriber Publisher Topic SubscriberOpenSplice DDS val pub = Publisher(dp) val sub = Subscriber(dp) // Create a Topic val topic = Topic[VehicleDynamics](dp, “TVehicleDynamics”) DataWriter DataReader
  • 16. Anatomy of a DDS Application [Scala API] Domain Domain Copyright  2011,  PrismTech  –    All  Rights  Reserved. val dp = DomainParticipant(domainId) Participant Session // Create a Publisher / Subscriber Publisher Topic SubscriberOpenSplice DDS val pub = Publisher(dp) val sub = Subscriber(dp) // Create a Topic val topic = Topic[VehicleDynamics](dp, “TVehicleDynamics”) Reader/Writers for User Defined for Types DataWriter DataReader // Create a DataWriter/DataWriter val writer = DataWriter[TempSensor](pub, topic) Reader/Writer for val reader = DataReader[TempSensor](sub, topic) application defined Topic Types
  • 17. Anatomy of a DDS Application [Scala API] Domain Domain Copyright  2011,  PrismTech  –    All  Rights  Reserved. val dp = DomainParticipant(domainId) Participant Session // Create a Publisher / Subscriber Publisher Topic SubscriberOpenSplice DDS val pub = Publisher(dp) val sub = Subscriber(dp) // Create a Topic val topic = Topic[VehicleDynamics](dp, “TVehicleDynamics”) Reader/Writers for User Defined for Types DataWriter DataReader // Write data val data = new VehicleDynamics(101, 131, 107, 7, 5) writer write data Reader/Writer for // But you can also write like this... application defined writer ! data Topic Types // Read new data and print it on the screen (reader read) foreach (prinln)
  • 18. Anatomy of a DDS Application [DDS C++ API 2010] Domain Domain Copyright  2011,  PrismTech  –    All  Rights  Reserved. auto dp = DomainParticipant(domainId); Participant Session // Create a Publisher / Subscriber Publisher Topic SubscriberOpenSplice DDS auto pub = Publisher(dp); auto sub = Subscriber(dp); // Create a Topic auto topic = Topic<VehicleDynamics>(dp, “TVehicleDynamics”); Reader/Writers for User Defined for Types DataWriter DataReader // Create a DataWriter/DataWriter auto writer = DataWriter<TempSensor>(pub, topic); Reader/Writer for auto reader = DataReader<TempSensor>(sub, topic); application defined Topic Types
  • 19. Anatomy of a DDS Application [DDS C++ API 2010] Domain Domain Copyright  2011,  PrismTech  –    All  Rights  Reserved. auto dp = DomainParticipant(domainId); Participant Session // Create a Publisher / Subscriber Publisher Topic SubscriberOpenSplice DDS auto pub = Publisher(dp); auto sub = Subscriber(dp); // Create a Topic auto topic = Topic<VehicleDynamics>(dp, “TVehicleDynamics”); Reader/Writers for User Defined for Types DataWriter DataReader // Write data writer.write(VehicleDynamics(101, 131, 107, 7, 5)); Reader/Writer for // But you can also write like this... writer << VehicleDynamics(101, 131, 107, 7, 5); application defined Topic Types // Read new data auto data = reader.read();
  • 20. OpenSplice DDS QoS
  • 21. QoS Model ☐ QoS-Policies control local and end-to-end properties of DDS Copyright  2011,  PrismTech  –    All  Rights  Reserved. entities Type Matching QoS matching QoS QoS QoS QoS QoS QoS QoS ☐ Local properties controlled by Topic QoS are related resource usage Publisher Name Subscriber ... DataWriter writes Type reads DataReader ... ☐ End-to-end propertiesOpenSplice DDS ... controlled by QoS are related DomainParticipant DataWriter writes Type reads DataReader DomainParticipant Name to temporal and spatial aspects Topic of data distribution QoS QoS QoS ☐ Some QoS-Policies are matched based on a Request vs. Offered Model thus QoS- enforcement
  • 22. QoS Policies [T: Topic] [DR: DataReader] [DW: DataWriter] [P: Publisher] [S: Subscriber] [DP: Domain Participant] QoS Policy Applicability RxO Modifiable Copyright  2011,  PrismTech  –    All  Rights  Reserved. USER_DATA DP, DR, DW N Y TOPIC_DATA T N Y Configuration GROUP_DATA P, S N Y DURABILITY T, DR, DW Y NOpenSplice DDS DURABILITY T, DW N N SERVICE Data Availability HISTORY T, DR, DW N N PRESENTATION P, S Y N RELIABILITY T, DR, DW Y N PARTITION P, S N Y Data Delivery DESTINATION T, DR, DW Y N ORDER LIFESPAN T, DW N Y
  • 23. QoS Policies [T: Topic] [DR: DataReader] [DW: DataWriter] [P: Publisher] [S: Subscriber] [DP: Domain Participant] QoS Policy Applicability RxO Modifiable Copyright  2011,  PrismTech  –    All  Rights  Reserved. DEADLINE T, DR, DW Y Y LATENCY T, DR, DW Y Y BUDGET Temporal/ TRANSPORT T, DW N Y Importance PRIORITY CharacteristicsOpenSplice DDS TIME BASED DR N Y FILTER OWNERSHIP T, DR, DW Y N OWNERSHIP DW N Y Replication STRENGTH LIVELINESS T, DR, DW Y N Fault-Detection
  • 24. Temporal Properties Copyright  2011,  PrismTech  –    All  Rights  Reserved. TimeBasedFilter Deadline [Inbound]OpenSplice DDS Throughput LatencyBudget Latency [Outbound] TransportPriority
  • 25. Data Availability History Copyright  2011,  PrismTech  –    All  Rights  Reserved. DataOpenSplice DDS Lifespan Durability Availability Ownership Ownership Strength
  • 26. OpenSplice DDS Partition Data Delivery Reliability Presentation Data Delivery Order Destination Copyright  2011,  PrismTech  –    All  Rights  Reserved.
  • 27. Setting QoS Policies Copyright  2011,  PrismTech  –    All  Rights  Reserved. C++ // Setting Partition QoS-Policy on Publisher qos::PublisherQos pubQos; pubQos << policy::Partition("Partition"); Publisher pub(dp, pubQoS);OpenSplice DDS // Setting various QoS-Policy on a Topic qos::TopicQos tqos; tqos << policy::Reliability::Reliable() << policy::Durability::Transient() << policy::History::KeepLast(5); Topic<VehicleDynamics> topic(dp,"Partition", tqos);
  • 28. Data Reader/Writer CachesOpenSplice DDS
  • 29. DDS Model Topic Instances DataWriter DataReader DataReader Copyright  2011,  PrismTech  –    All  Rights  Reserved. ... ... ... ... ... ... ... ... ... ... ... ... ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache Writer History Reader History network
  • 30. DDS Model DataWriter DataReader DataReader Copyright  2011,  PrismTech  –    All  Rights  Reserved. ... ... ... ... ... ... ... ... ... ... ... ... ...OpenSplice DDS DataWriter Cache DataReader Cache DataReader Cache QoS Policies ‣ History ‣ Destination Order ‣ Presentation network ‣ Partition ‣ Time Based Filter QoS Policies ‣ Ownership ‣ Reliability ‣ Durability ‣ History ‣ Transport Priority ‣ Latency Budget ‣ Time Based Filter
  • 31. Dynamic View of a Stream Copyright  2011,  PrismTech  –    All  Rights  Reserved.OpenSplice DDS ... Stream: Set of samples written over time for a given topic instance.
  • 32. Dynamic View of a Stream Assumptions: Reader History = KeepLast (n) WriterHistory = KeepLast (m) Copyright  2011,  PrismTech  –    All  Rights  Reserved. Writer HistoryOpenSplice DDS ... Stream: Set of samples written over time for a given topic instance.
  • 33. Dynamic View of a Stream Assumptions: Reader History = KeepLast (n) WriterHistory = KeepLast (m) Copyright  2011,  PrismTech  –    All  Rights  Reserved. Samples ‘on Writer History the wire’OpenSplice DDS ... Stream: Set of samples written over time for a given topic instance.
  • 34. Dynamic View of a Stream Assumptions: Reader History = KeepLast (n) WriterHistory = KeepLast (m) Copyright  2011,  PrismTech  –    All  Rights  Reserved. Samples ‘on Writer History the wire’ Reader HistoryOpenSplice DDS ... Stream: Set of samples written over time for a given topic instance.
  • 35. Dynamic View of a Stream Assumptions: Reader History = KeepLast (n) WriterHistory = KeepLast (m) Copyright  2011,  PrismTech  –    All  Rights  Reserved. Samples ‘on Writer History the wire’ Reader History ‘Past’ SamplesOpenSplice DDS ... Stream: Set of samples written over time for a given topic instance.
  • 36. Eventual View of a Stream Assumptions (Default Settings): Reader History = KeepLast (1) WriterHistory = KeepLast (1) Copyright  2011,  PrismTech  –    All  Rights  Reserved. Writer History Reader History ‘Past’ SamplesOpenSplice DDS Stream: Set of samples written over time for a given topic instance.
  • 37. Eventual View of a Stream Assumptions: Reader History = KeepLast (n) with n > 1 WriterHistory = KeepLast (1) Copyright  2011,  PrismTech  –    All  Rights  Reserved. Writer History Reader History ‘Past’ SamplesOpenSplice DDS Stream: Set of samples written over time for a given topic instance.
  • 38. Eventual View of a Stream Assumptions: Reader History = KeepLast (n) with n > 1 WriterHistory = KeepLast (m) with n > m > 1 Copyright  2011,  PrismTech  –    All  Rights  Reserved. Writer History Reader History ‘Past’ SamplesOpenSplice DDS Stream: Set of samples written over time for a given topic instance.
  • 39. Reading Data Samples ☐ Samples can be read from the Data Reader History Cache Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The action of reading a sample is non-destructive. Samples are not removed from the cacheOpenSplice DDS DataReader DataReader ... read ... DataReader Cache DataReader Cache
  • 40. Taking Data Samples ☐ Samples can be taken from the Data Reader History Cache Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The action of taking a sample is destructive. Samples are removed from the cacheOpenSplice DDS DataReader DataReader ... ... take DataReader Cache DataReader Cache
  • 41. Read vs. Take Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The read operation should always be access the latest know value for topics that represent distributed stateOpenSplice DDS ☐ The take operation should be used to get the last notification from a topic that represent an event
  • 42. Data SelectorsOpenSplice DDS
  • 43. Cherry Picking in DDS Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ DDS provides some very flexible mechanisms for selecting the data to be read:OpenSplice DDS ☐ Data Content ☐ Data Status ☐ These mechanisms are composable
  • 44. Content-Based Data SelectionOpenSplice DDS
  • 45. Filters and Queries Application ☐ DDS Filters allow to control what gets into a DataReader cache Copyright  2011,  PrismTech  –    All  Rights  Reserved. Query ☐ DDS Queries allow to control what gets out of a DataReader cache DataReaderOpenSplice DDS ☐ Filters are defined by means of ... ContentFilteredTopics ... ... ... ☐ Queries operate in conjunction with DataReader Cache read operations Filter ☐ Filters and Queries are expressed as SQL where clauses
  • 46. Filters [Scala API] struct VehicleDynamics { long vid; long x; long y; long dx; long dy;}; #pragma keylist VehicleStatus vid Copyright  2011,  PrismTech  –    All  Rights  Reserved. /** ** NOTE: The Scala API if not provided with DP/Sub/Pub assumes default domains and ** default partition. **/ // Create a Topic val topic = Topic[VehicleDynamics](“TVehicleDynamics”)OpenSplice DDS // Define filter expression and parameters val filter = “x < %0 AND y < %1” val params = List(“200”, “300”) // Define content filtered topic val cftopic = ContentFilteredTopic[VehicleDynamics](“CFTVehicleDynamics”, topic, filter, params) // Create a DataReader for the content-filtered Topic val reader = DataReader[VehicleDynamics](cftopic)
  • 47. QueryAPI 2010] struct VehicleDynamics { [DDS C++ long vid; long x; long y; long dx; long dy;}; #pragma keylist VehicleStatus vid Copyright  2011,  PrismTech  –    All  Rights  Reserved. // Define the query and the parametersOpenSplice DDS std::vector<std::string> p; p.push_back("100"); p.push_back("100"); dds::core::Query q("x < %0 AND y < %1", p.begin(), p.end()); auto data = reader .selector() .filter_content(q) .read();
  • 48. Instances ☐ DDS provides a very efficient way of reading data belonging to a Copyright  2011,  PrismTech  –    All  Rights  Reserved. specific Topic Instance ☐ Obviously, one could use queries to match the key’s value, but this is not as efficient as the special purpose instance selectorOpenSplice DDS // C++ auto data = reader .selector() .instance(handle) .read(); // Scala val data = reader read(handle)
  • 49. State-Based SelectionOpenSplice DDS
  • 50. Sample, Instance, and View State ☐ The samples included in the DataReader cache have associated Copyright  2011,  PrismTech  –    All  Rights  Reserved. some meta-information which, among other things, describes the status of the sample and its associated stream/instance ☐ The Sample State (READ, NOT_READ) allows to distinguish betweenOpenSplice DDS new samples and samples that have already been read ☐ The View State (NEW, NOT_NEW) allows to distinguish a new instance from an existing one ☐ The Intance State (ALIVE, NOT_ALIVE_DISPOSED, NOT_ALIVE_NO_WRITERS) allows to track the life-cycle transitions of the instance to which a sample belongs
  • 51. State Selector in Action Copyright  2011,  PrismTech  –    All  Rights  Reserved. // Read only new samples auto data = reader C++ Scala .selector() // Read only new samples .filter_state(status::DataState::new_data()) val data = reader read .read()OpenSplice DDS // Read any samples from live instances // Read any samples from live instances val data = reader read(SampleSelector.AnyData) auto data = reader .selector() .filter_state(status::DataState::any_data()) .read()
  • 52. Putting all Together Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Selectors can be composed in a flexible and expressive mannerOpenSplice DDS C++ auto data = reader ! .selector() .instance(handle) ! ! .filter_state(status::DataState::new_data()) ! ! .filter_content(q) .read();
  • 53. OpenSplice DDS S D D tandard
  • 54. DDS Standard Ecosystem Application Application Copyright  2011,  PrismTech  –    All  Rights  Reserved. 2012 API 2012 DDS RMI DDS RMI 2012 2010 2010 2012 ANSI C ISO C++ Java-5 ScalaOpenSplice DDS 2004 2010 2010 201x Security Security X-Types X-Types DDS 2004 Wire Protocol DDSI-RTPS network DDSI-RTPS 2006 2006
  • 55. Concluding Remarks Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The Data Distribution Service for Real-Time Systems provides a powerful and feature-rich topic-based publish/subscribe abstractionOpenSplice DDS ☐ This technology is widely used in mission and business critical systems and it being swiftly adopted in data- centric/big-data systems
  • 56. References Copyright  2011,  PrismTech  –    All  Rights  Reserved. OpenSplice | DDS Escalier ¥ #1 OMG DDS Implementation ¥ Fastest growing JVM Language ¥ Scala API for OpenSplice DDS ¥ Open Source ¥ Open Source ¥ Open Source ¥ www.opensplice.org ¥ www.scala-lang.org ¥ github.com/kydos/escalierOpenSplice DDS [C++] [Java] DDS-PSM-Cxx 2010 ¥ Simple C++ API for DDS ¥ DDS-PSM-Java for OpenSplice DDS ¥ DDS-PSM-Cxx API Standard ¥ Open Source ¥ Open Source ¥ Open Source ¥ code.google.com/p/simd-cxx ¥ github.com/kydos/simd-java ¥ github.com/kydos/dds-psm-cxx
  • 57. :: Connect with Us :: ¥opensplice.com ¥forums.opensplice.org ¥@acorsaro ¥opensplice.org ¥opensplicedds@prismtech.com ¥@prismtechOpenSplice DDS ¥ crc@prismtech.com ¥sales@prismtech.com ¥youtube.com/opensplicetube ¥slideshare.net/angelo.corsaro