OMG DDS: The Data Distribution Service for Real-Time Systems

OpenSplice DDS Delivering Performance, Openness, and Freedom Angelo Corsaro, Ph.D. OMG DDS: The Data Distribution Product Strategy Manager OMG RTESS and DDS SIG Co-Chair angelo.corsaro@prismtech.com Service for Real-Time Systems

Motivating Forces The Standard Agenda Architectural Principles Programming Model Future Directions

The Need for Data Distribution Time Scale Geographical Scale Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Parallelism Determinism Throughput, Availability Scalability, Persistence, Security Systemic Real-Time Near Real-Time Fault- Data Complex Information Signal Information Tolerant Information Processing Processing Management Processing Processing Data Distribution Parallel Systems Distributed Systems © 2009, PrismTech. All Rights Reserved

Data Exchange / Distribution ‣ Network Centric Architectures are emerging as a key trend for next generation military and civil system of systems ‣ Efficient, scalable and QoS-enabled data dissemination is an enabling technology for Network Joint Forces Joint Forces Centric Systems Global Info Grid Global Info Grid Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Adapted from “The Future of AWACS”, by LtCol Joe Chapa The Right Information => To the Right People => At the Right Time © 2009, PrismTech. All Rights Reserved

Addressing Data Distribution Challenges DDS is standard designed to address the data-distribution challenges across The OMG DDS Standard a wide class of Defense and Aerospace Applications ‣ Introduced in 2004 to address the Data Distribution challenges faced by a wide class of Defense and Aerospace Applications Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Key requirement for the standard were its ability to deliver very high performance while seamlessly scaling from embedded to ultra- large-scale deployments ‣ Today recommended by key administration worldwide and widely adopted across several different application domains, such as, Automated Trading, Simulations, SCADA, Telemetry, etc. © 2009, PrismTech. All Rights Reserved

The OMG Data Distribution Service (DDS) DDS v1.2 API Standard ‣ Language Independent, OS and HW architecture Application independent Object/Relational Mapping ‣ DCPS. Standard API for Data-Centric, Topic- Data Local Reconstruction Layer (DLRL) Based, Real-Time Publish/Subscribe Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Content ‣ Ownership Durability DLRL. Standard API for creating Object Views out Subscription of collection of Topics Minimum Proﬁle Data Centric Publish/Subscribe (DCPS) DDSI/RTPS v2.1 Wire Protocol Standard ‣ Standard wire protocol allowing interoperability Real-Time Publish/Subscribe Protocol DDS Interoperability Wire Protocol between different implementations of the DDS standard UDP/IP ‣ Interoperability demonstrated among key DDS vendors in March 2009 © 2009, PrismTech. All Rights Reserved

OMG DDS Recommendations is churning… ! The infrastructure evolution cycle – New -> Emerging -> Standard -> Commodity – Middleware is emerging as Mandated/Recommended by key Administrations OS declines ! …DDS is maturing… ‣ US Navy: Open Architecture focus – OMG – Wire spec Proprietary Information - Distribution without Expressed Written Permission is Prohibited. – Tools ‣ DISR/DISA: Net-centric Systems – Enterprise integration – Multiple products fielded ‣ EuroControl: Air Traffic Control Center Operational – Deployed applications! Interoperability ! …and adoption is on the rise – Navy for VSI ‣ QinetiQ: Recommending DDS – DISR – FCS/SoSCOE – Many significant applications © 2009, PrismTech. All Rights Reserved

OMG DDS Adoptions Adopted by Most Challenging International Programs ‣ US FCS/SOSCOE Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ European Next Generation Flight Data Processor ‣ US ZUMWALT [DDG(1000)] ‣ VTID adopting DDS for the architectural demonstrator © 2009, PrismTech. All Rights Reserved

OMG DDS Applicability Adapted from NSWC-DD Open Architecture Documentation Communication Technologies Standards Web SVCs The DDS is the only technology that spans across Java RTSJ the board. Proprietary Information - Distribution without Expressed Written Permission is Prohibited. JMS It guarantees exceptional real- time behavior, CORBA RT-CORBA while providing unparalleled level OMG Data Distribution Service (DDS) of throughput ! MPI Message Passing Interface Non Real-Time Soft Real-Time Hard Real-Time Extreme Real-Time © 2009, PrismTech. All Rights Reserved

Some DDS Use Cases Defense ‣ Combat Management Systems ‣ Flycatcher system ‣ Unmanned Vehicles (Air + Land) ‣ Tactical Links Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Radar Processing ‣ Submarine Systems ‣ Future Combat Systems ‣ Simulation SCADA/Utilities ‣ Industrial Automation ‣ Power Grids ‣ Rocket Launch Systems © 2009, PrismTech. All Rights Reserved

Some DDS Use Cases Transportation ‣ Air Traffic Management/Control ‣ Metropolitan Traffic Management ‣ Underground Metropolitan Transportation Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Financial Services ‣ Automated Trading Firms ‣ Compliance Systems ‣ Market Data Platforms © 2009, PrismTech. All Rights Reserved

Leading DDS Implementations ‣ Full Implementation of the OMG DDS Standard (DCPS+DLRL + DDSI) ‣ DCPS + DDSI available as Open Source at http://www.opensplice.org ‣ Developed and Commercially supported by PrismTech Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Implementation of the OMG DDS Standard (DCPS + DDSI) ‣ Developed and Commercially supported by RTI ‣ Partial Implementation of the OMG DDS Standard (Subset of DCPS) ‣ Available as Open Source at http://www.opendds.com ‣ Developed and Commercially supported by OCI © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom DDS vs AMQP

Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Object Relational Mapping Standard API Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Data Local Reconstruction Layer (DLRL) DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Proﬁle Pub/Sub Standard API Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Wire Protocol Standard Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved

Architectural Style AMQP OMG DDS Subscriber Subscriber Publisher Publisher Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Broker Broker Publisher Broker Subscriber Publisher Broker Subscriber Broker Broker Publisher Publisher Subscriber Subscriber © 2009, PrismTech. All Rights Reserved

Key Differences in Summary AMQP DDS ‣ Brokered Communication ‣ Peer-to-Peer Communication Architecture Architecture ‣ Wire Protocol Standard ‣ Wire Protocol + API Standard Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Interoperability across AMQP ‣ Interoperability across DDS implementations implementations ‣ Application Portability across DDS Implementation ‣ Header Based Routing ‣ Content-Based Filtering/Quering ‣ Not geared for High-Performance ‣ Designed bottom-up for maximum performance and determinism © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Loosely Coupled Technology

Counting the “W’s” of Coupling Request Client Server The “4Ws” of Client/Server Reply ‣ Who+Where: Space Coupling ‣ What: Structural Coupling Client/Server ‣ When: Time Coupling CORBA, COM+, Java RMI, .Net Remoting, Web SVCs Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Subscriber Publisher Subscriber A Single “W” for DDS Publisher ‣ What: Structural Coupling Subscriber Publisher DDS Subscriber DDS features dynamic discovery and full time decoupling! © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom High Performance/Availability Pub/Sub

High Performance Pub/Sub The right data, at the right place, at the right time ‣ Fully distributed, Peer-to-Peer -- All the Time. Communication ‣ No Single Point of Failure Publisher Subscriber ‣ No Single Point of Bottleneck Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Multicast-enabled Brokers Subscriber ‣ High performance and highly Publisher scalable ‣ High availability Publisher ‣hot-swap Subscriber ‣hot-hot architecture © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Application Ownership Object/Relational Mapping Data Local Reconstruction Layer (DLRL) Durability Content Data-Centric Pub/Sub Subscription Minimum Proﬁle Data Centric Publish/Subscribe (DCPS) Real-Time Publish/Subscribe Protocol DDS Interoperability Wire Protocol UDP/IP

Data-Centric Pub/Sub ‣ Data-Centric Features are built-in and don’t rely on an external DBMS ‣ Providing thus performance, scalability, and availability ‣ Distributed Relational Data Model Subscriber Publisher DBMS ‣ Local Queries B ‣ Continuous Queries / Content Based Proprietary Information - Distribution without Expressed Written Permission is Prohibited. m Subscriptions A F Subscriber ‣ Windows Publisher J D C ‣ Object/Relational Mapping K E ‣ Support for a subset of SQL-92 Publisher Subscriber Perfect Blend of Data-Centric and Real-Time Publish/Subscribe Technologies © 2009, PrismTech. All Rights Reserved

“Visualizing” Data-Centric Pub/Sub Subscriber Publisher B D J D C E K A B A B Proprietary Information - Distribution without Expressed Written Permission is Prohibited. m Subscriber D C Publisher A F J J A F D C K A F K E Subscriber Publisher D J D K E E © 2009, PrismTech. All Rights Reserved

Organizing Information ‣ All DDS communication is Domain Partition happens within a Domain ‣ Domain can divided into Publisher Subscriber Partitions Proprietary Information - Distribution without Expressed Written Permission is Prohibited. B ‣ Topics are published and m subscribed across on or A F Subscriber more Partitions Publisher J D C K E Publisher Subscriber © 2009, PrismTech. All Rights Reserved

Topics and Data-Centric Pub/Sub ‣ Topics. Unit of information exchanged between Publisher and Subscribers. Topic ‣ Data Types. Type associated to a Topic must be a structured type expressed in IDL struct TempSensor { long tID; Topic Type float temp; ‣ Topic Instances. Key values in a datatype Proprietary Information - Distribution without Expressed Written Permission is Prohibited. float humidity; uniquely identify a Topic Instance (like rows in }; table) #pragma keylist TempSensor tID TempSensor ‣ Content Awareness. SQL Expressions can be tID temp humidity used to do content-aware subscriptions, Instances 1 21 62 2 27 78 queries, joins, and correlate topic instances 3 25.5 72.3 y idit idit y ity SELECT * FROM TempSensor t um Samples mid h um hu 62 h 65 63 WHERE t.temp > 25 p p tem tem p tem 21 22 tID 23 tID tID temp humidity tID 1 1 1 2 27 78 3 25.5 72.3 t1 t2 t3 Time © 2009, PrismTech. All Rights Reserved

Distributed Relational Information Modeling ‣ Topic Keys can be used to identify instances as well as relationships ‣ Relationships can be navigated by relying on a subset of SQL 92 ‣ One-to-many relationships can be captured using foreign keys ‣ Many-to-many relationships need to be modeled using a topics Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Keys can be represented by an arbitrary number of Topic fields TempSensor Room Floor tID: int rID: int fID: int temp: float width: float level: int humidity: float height: float open: int rID: int length: float fID: int © 2009, PrismTech. All Rights Reserved

Data Centric Publish/Subscribe Publishers Subscribers TempSensor TempSensor tID temp humidity tID temp humidity 1 18 60 1 18 60 2 22 75 3 21 71 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. struct TempSensor { TempSensor int tID; float temp; TempSensor tID temp humidity float humidity; tID temp humidity 2 22 75 }; 2 22 75 #pragma keylist TempSensor tID 3 21 71 SELECT * FROM TempSensor t WHERE s.temp > 20 s.t TempSensor ID == tID temp humidity Fully Distributed Global 1 TempSensor 3 21 71 Data Space tID temp humidity 1 18 60 © 2009, PrismTech. All Rights Reserved

Data Centric Publish/Subscribe Publishers Subscribers TempSensor TempSensor tID temp humidity tID temp humidity 1 21 62 1 21 62 2 22 75 3 21 71 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. struct TempSensor { TempSensor int tID; float temp; TempSensor tID temp humidity float humidity; 2 22 75 tID temp humidity }; #pragma keylist TempSensor tID 1 21 62 2 22 75 3 21 71 SELECT * FROM TempSensor t s.t WHERE s.temp > 20 TempSensor ID == tID temp humidity Fully Distributed Global 1 TempSensor 3 21 71 Data Space tID temp humidity 1 21 62 © 2009, PrismTech. All Rights Reserved

Processing Content & Structure Complex Event Processing... ‣ Local Queries DDS provides some of the most useful features found in Complex Event Processing platforms! ‣ Continuous Queries (Content-based Subscriptions) ‣ Topics Joins & Projections ‣ Events windows (via History QoS) Proprietary Information - Distribution without Expressed Written Permission is Prohibited. tID temp humidity S1 3 21.5 72.3 tID temp humidity {iID = 1} 2 20 78 tID temp humidity 1 21 62 S2 P {temp > 21 | humidity > 75} S3 { temp <16 } © 2009, PrismTech. All Rights Reserved

Processing Content & Structure Complex Event Processing... ‣ Local Queries DDS provides some of the most useful features found in Complex Event Processing platforms! ‣ Continuous Queries (Content-based Subscriptions) ‣ Topics Joins & Projections ‣ Events windows (via History QoS) Proprietary Information - Distribution without Expressed Written Permission is Prohibited. tID temp humidity 1 21 62 S1 {iID = 1} tID temp humidity 2 20 78 3 21.5 72.3 S2 P {temp > 21 | humidity > 75} S3 { temp <16 } © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Application Ownership Object/Relational Mapping Data Local Reconstruction Layer (DLRL) Durability Content Object-Oriented Pub/Sub Subscription Minimum Proﬁle Data Centric Publish/Subscribe (DCPS) Real-Time Publish/Subscribe Protocol DDS Interoperability Wire Protocol UDP/IP

Object/Relational Mapping TempSensor temp: float humidity: float ‣ Automatically bridges the getTemp() getHumidity() * Room Floor 1 width: float level: int Object/Relational Impedance height: float length: float * open: int 1 climate: int Conditioner Mismatch temp: float humidity: float 1 climate: int set_climate(); set_status() set_climate() * ‣ Arbitrary object setTemp() get_level() Proprietary Information - Distribution without Expressed Written Permission is Prohibited. setHumidity() setFanLevel() reconstructions start() stop() ‣ Automatic Relationships pause(): Management TConditioner cID: int TTempSernsor ‣ Inheritance temp: float tID: int humidity: float temp: float fan_level: int TRoom humidity: float ‣ Local Operations status: int rID: int rID: int width: float height: float rID: int ‣ Local/Distributed State length: float fID: int TFloor fID: int level: int open: int © 2009, PrismTech. All Rights Reserved

Object/Relational Mapping Unleashing the power of Objects... Object Oriented Canvas OO Relational handle: Handle ‣ Middleware can automatically Flight RadarTrack manage the generation and FlightView call_sign: string x: float image: string route: list association between the refresh_rate: int * origin: string y: float * z: float * 1 Object-Oriented Model and the visible: bool dest: string radar_id: string Relational Model uid: GUID aircraft: string operationOne() display() operationX() operationTwo() Proprietary Information - Distribution without Expressed Written Permission is Prohibited. operationY() FlightViewTopic FlightViewAssociation FlightTopic Relational call_sign: string image: string call_sign: string route: list RadarTrackTopic refresh_rate: int uid: GUID origin: string x: float visible: bool handle: Handle dest: string y: float uid: GUID aircraft: string z: float radar_id: string radar_id: string Relational OO ‣ The Relational Model can be mapped to an Object Oriented model ‣ The mapping is under control of the architect © 2009, PrismTech. All Rights Reserved

Multiple Object/Relational Mappings Canvas FastFlightView image: string handle: Handle DLRL DLRL refresh_rate: int visible: bool Flight uid: GUID RadarTrack FlightView call_sign: string x: float x: float image: string route: list y: float y: float refresh_rate: int * * origin: string 1 * z: float z: float visible: bool dest: string call_sign: string radar_id: string uid: GUID aircraft: string display() operationOne() Proprietary Information - Distribution without Expressed Written Permission is Prohibited. display() operationX() operationTwo() operationY() FlightViewTopic FlightViewAssociation FlightTopic DCPS call_sign: string image: string call_sign: string route: list RadarTrackTopic refresh_rate: int uid: GUID origin: string x: float visible: bool handle: Handle dest: string y: float uid: GUID aircraft: string z: float radar_id: string radar_id: string Multiple Mappings ‣ Different Object/Relatioal Mappings (ORM) can be used for different subsystems ‣ The state is shared across all the local reconstruction (ORMs) © 2009, PrismTech. All Rights Reserved

An Example High-End 3D Visualization Primary Radar RadarTrack x: float y: float z: float radar_id: string operationOne() operationTwo() Proprietary Information - Distribution without Expressed Written Permission is Prohibited. FastFlightView RadarTrackTopic x: float image: string y: float refresh_rate: int FlightViewTopic image: string z: float radar_id: string visible: bool refresh_rate: int visible: bool uid: GUID uid: GUID x: float y: float RadarTrackTopic x: float z: float y: float z: float call_sign: string radar_id: string display() RadarTrackTopic x: float y: float FlightTopic call_sign: string z: float route: list radar_id: string origin: string dest: string aircraft: string radar_id: string Flight call_sign: string route: list RadarTrack origin: string RadarTrackList dest: string aircraft: string operationX() operationY() Flight Data Processor © 2009, PrismTech. All Rights Reserved

An Example High-End 3D Visualization Primary Radar RadarTrack x: float y: float z: float radar_id: string operationOne() operationTwo() Proprietary Information - Distribution without Expressed Written Permission is Prohibited. FastFlightView RadarTrackTopic x: float image: string y: float z: float refresh_rate: int FlightViewTopic image: string radar_id: string visible: bool refresh_rate: int visible: bool uid: GUID uid: GUID x: float y: float RadarTrackTopic x: float z: float y: float z: float call_sign: string radar_id: string display() RadarTrackTopic x: float y: float FlightTopic call_sign: string z: float route: list radar_id: string origin: string dest: string aircraft: string radar_id: string Flight call_sign: string route: list RadarTrack origin: string RadarTrackList dest: string aircraft: string operationX() operationY() Flight Data Processor © 2009, PrismTech. All Rights Reserved

An Example High-End 3D Visualization Primary Radar Secondary Radar RadarTrack x: float y: float z: float radar_id: string RadarTrack operationOne() x: float operationTwo() y: float Proprietary Information - Distribution without Expressed Written Permission is Prohibited. z: float radar_id: string operationOne() operationTwo() FastFlightView RadarTrackTopic RadarTrackTopic x: float x: float image: string y: float y: float refresh_rate: int FlightViewTopic image: string z: float radar_id: string z: float radar_id: string visible: bool refresh_rate: int RadarTrackTopic visible: bool x: float uid: GUID uid: GUID y: float x: float z: float radar_id: string y: float RadarTrackTopic z: float x: float call_sign: string RadarTrackTopic y: float x: float z: float display() y: float radar_id: string z: float FlightTopic call_sign: string radar_id: string route: list origin: string RadarTrack dest: string Flight aircraft: string call_sign: string radar_id: string route: list origin: string RadarTrackList dest: string aircraft: string RadarTrack operationX() operationY() Automatically Manage Flight Data Processor Relationships © 2009, PrismTech. All Rights Reserved

Custom Filter Example High-End 3D Visualization Primary Radar Secondary Radar RadarTrack x: float y: float z: float radar_id: string RadarTrack operationOne() x: float operationTwo() y: float Proprietary Information - Distribution without Expressed Written Permission is Prohibited. z: float radar_id: string operationOne() operationTwo() FastFlightView RadarTrackTopic RadarTrackTopic x: float x: float image: string y: float y: float refresh_rate: int FlightViewTopic z: float z: float SQL Filter image: string radar_id: string radar_id: string visible: bool refresh_rate: int RadarTrackTopic uid: GUID visible: bool uid: GUID x: float y: float Custom Filter x: float z: float radar_id: string y: float z: float RadarTrackTopic x: float call_sign: string y: float RadarTrackTopic z: float display() x: float radar_id: string y: float FlightTopic call_sign: string z: float route: list radar_id: string origin: string RadarTrack Flight dest: string aircraft: string call_sign: string Filter based on Region-of-interest radar_id: string route: list origin: string RadarTrackList dest: string SQL Filter aircraft: string RadarTrack Custom Filter operationX() operationY() Automatically Manage Flight Data Processor Relationships © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom ...Putting it all Together

DCPS Application Application ‣ The application works directly at the DCPS level TConditioner cID: int temp: float TTempSensor tID: int temp: float Proprietary Information - Distribution without Expressed Written Permission is Prohibited. humidity: float TRoom humidity: float ‣ When using OO Programming fan_level: int rID: int rID: int status: int width: float rID: int height: float Languages the Object/Relational length: float fID: int TFloor fID: int level: int Impedance Mismatch has to be open: int manually dealt Content DDS v1.2 Ownership Durability Subscription Minimum Profile Data Centric Publish/Subscribe (DCPS) © 2009, PrismTech. All Rights Reserved

DLRL Application Application TempSensor temp: float ‣ The DLRL Layer is used to humidity: float getTemp() getHumidity() * Room Floor width: float level: int 1 height: float open: int 1 climate: int provide a Language Integrated Conditioner length: float * temp: float climate: int set_status() humidity: float 1 set_climate(); set_climate() fan_level: int * set_temp() get_level() set_humidity() access to DDS data status: int setTemp() setHumidity() setFanLevel() Proprietary Information - Distribution without Expressed Written Permission is Prohibited. start() stop() ‣ The Designer has great freedom pause(): DDS v1.2 Object/Relational Mapping in deciding how Objects have to Data Local Reconstruction Layer (DLRL) map to Topics TConditioner cID: int TTempSensor tID: int temp: float temp: float humidity: float TRoom humidity: float ‣ Different Object Reconstruction fan_level: int rID: int rID: int status: int width: float rID: int height: float length: float TFloor can be created for different fID: int fID: int level: int open: int applications Content DDS v1.2 Ownership Durability Subscription Minimum Profile Data Centric Publish/Subscribe (DCPS) © 2009, PrismTech. All Rights Reserved

Generic DDS Application Application TempSensor temp: float humidity: float getTemp() getHumidity() * Room Floor width: float level: int 1 ‣ In the most general case, different height: float open: int Conditioner length: float 1 climate: int * temp: float climate: int set_status() humidity: float 1 set_climate(); set_climate() fan_level: int * set_temp() get_level() set_humidity() portion of the application might rely on status: int setTemp() setHumidity() setFanLevel() start() Proprietary Information - Distribution without Expressed Written Permission is Prohibited. DLRL or DCPS depending on their stop() pause(): specific needs Application Object/Relational Mapping Data Local Reconstruction Layer (DLRL) ‣ DCPS access might be required for TConditioner TTempSensor cID: int tID: int temp: float temp: float humidity: float TRoom humidity: float accessing and tuning some specific fan_level: int status: int rID: int rID: int width: float height: float rID: int length: float TFloor QoS fID: int fID: int level: int open: int Content DDS v1.2 Ownership Durability Subscription Minimum Profile Data Centric Publish/Subscribe (DCPS) © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom QoS-Enabled Pub/Sub

Communicating with Topics, Partitions and Domains Topic Topic Topic Samples Samples Instances Samples Instances Instances 121 62 1 21 62 1 22 62 1 22 62 1 23 63 1 23 63 DataReader DataReader 1 21 62 2 20 61 1 22 62 2 19 60 1 23 63 DataReader DataWriter Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 2 20 61 2 19 60 DataReader DataReader 2 20 61 2 19 60 DataWriter DataWriter DataReader 3 25 70 3 25 70 3 25 71 25 3 25 71 25 3 25 74 3 25 74 3 26 77 3 26 77 3 26 77 DataWriter struct TempSensor { 3 25 70 3 25 71 25 3 25 74 int tID; float temp; float humidity; }; #pragma keylist TempSensor tID © 2009, PrismTech. All Rights Reserved

Communicating with Topics, Partitions and Domains Topic Topic Topic Samples Samples Instances Samples Instances Instances 121 62 1 21 62 1 22 62 1 22 62 1 23 63 1 23 63 DataReader DataReader 1 21 62 2 20 61 1 22 62 2 19 60 1 23 63 DataReader DataWriter Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 2 20 61 2 19 60 DataReader DataReader 2 20 61 2 19 60 DataWriter DataWriter DataReader 3 25 70 3 25 70 3 25 71 25 3 25 71 25 3 25 74 3 25 74 3 26 77 3 26 77 3 26 77 DataWriter struct TempSensor { 3 25 70 3 25 71 25 3 25 74 int tID; float temp; Arrows }; float humidity; show #pragma keylist TempSensor tID structural relationship s, not data- Subscriber Publisher ﬂows Partition © 2009, PrismTech. All Rights Reserved

Communicating with Topics, Partitions and Domains Topic Topic Topic Samples Samples Instances Samples Instances Instances 121 62 1 21 62 1 22 62 1 22 62 1 23 63 1 23 63 DataReader DataReader 1 21 62 2 20 61 1 22 62 2 19 60 1 23 63 DataReader DataWriter Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 2 20 61 2 19 60 DataReader DataReader 2 20 61 2 19 60 DataWriter DataWriter DataReader 3 25 70 3 25 70 3 25 71 25 3 25 71 25 3 25 74 3 25 74 3 26 77 3 26 77 3 26 77 DataWriter struct TempSensor { 3 25 70 3 25 71 25 3 25 74 int tID; float temp; Arrows }; float humidity; show #pragma keylist TempSensor tID structural relationship s, not data- Subscriber Publisher ﬂows Partition Domain Participant Domain © 2009, PrismTech. All Rights Reserved

QoS Model ‣ QoS-Policies are used to control relevant properties of OpenSplice DDS entities, Type Matching QoS matching such as: QoS QoS QoS QoS QoS QoS QoS ‣ Temporal Properties ‣ Priority Topic Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Name Publisher Subscriber ‣ Durability ... DataWriter writes Type reads DataReader ... ... ‣ Availability DomainParticipant writes Type reads DataReader DomainParticipant DataWriter ‣ ... Name ‣ Some QoS-Policies are matched based on Topic QoS QoS QoS a Request vs. Offered Model thus QoS- enforcement ‣ Publications and Subscriptions match only if the declared vs. requested QoS are compatible ‣ e.g., it is not possible to match a publisher which delivers data unreliably with a subscriber which requires reliability © 2009, PrismTech. All Rights Reserved

Sample QoS Policies QoS Policy Applicability RxO Modiﬁable Type Matching QoS matching DURABILITY T, DR, DW Y N Data Availability QoS QoS QoS QoS QoS QoS QoS DURABILITY SERVICE T, DW N N Topic Name Publisher Subscriber LIFESPAN T, DW - Y ... DataWriter writes Type reads DataReader ... T, DR, DW N N ... HISTORY DomainParticipant DataWriter writes Type reads DataReader DomainParticipant PRESENTATION P, S Y N Data Delivery Name Topic RELIABILITY T, DR, DW Y N PARTITION P, S N Y QoS QoS QoS Proprietary Information - Distribution without Expressed Written Permission is Prohibited. DESTINATION ORDER T, DR, DW Y N OWNERSHIP T, DR, DW Y N ‣ Rich set of QoS allow to configure OWNERSHIP DW - Y STRENGTH several different aspects of data DEADLINE T, DR, DW Y Y Data Timeliness availability, delivery and timeliness LATENCY BUDGET T, DR, DW Y Y TRANSPORT PRIORITY T, DW - Y ‣ QoS can be used to control and optimize network as well as TIME BASED FILTER DR - Y Resources computing resource RESOURCE LIMITS T, DR, DW N N USER_DATA DP, DR, DW N Y Conﬁguration TOPIC_DATA T N Y GROUP_DATA P, S N Y © 2009, PrismTech. All Rights Reserved

Mapping QoS Which properties does QoS controls? TimeBasedFilter Deadline History Data Data Throughput Lifespan Durability Latency Availability Ownership LatencyBudget TransportPriority Ownership Strength Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Control over Latency/Throughput tradeoff Control over data queueing Control over data latency Control over data persistency Control over data priority Control over data sources hot-swap Reliability OpenSplice DDS provides programmatic Destination QoS-driven support for configuring the most Presentation Data Delivery Order important properties of data distribution! Control over data distribution reliability Control over data ordering Control over presentation © 2009, PrismTech. All Rights Reserved

TimeBasedFilter Deadline Data Throughput Latency LatencyBudget TransportPriority Data Timeliness QoS Policy Applicability RxO Modifiable DEADLINE T, DR, DW Y Y LATENCY T, DR, DW Y Y Data BUDGET Timeliness TRANSPORT T, DW - Y PRIORITY © 2008, PrismTech. All Rights Reserved

Deadline You can’t be later than... The DEADLINE QoS policy allows to define the maximum inter-arrival time between data samples QoS Policy Applicability RxO Modifiable QoS matching DEADLINE T, DR, DW Y Y QoS QoS QoS ‣ DataWriter indicates that the application QoS Topic Name QoS commits to write a new value at least once Proprietary Information - Distribution without Expressed Written Permission is Prohibited. DataWriter writes Type reads DataReader Publisher ... Subscriber every deadline period ... ... ‣ writes Type reads DataReader DataReaders are notified by the DDS when DataWriter Name the DEADLINE QoS contract is violated Topic QoS QoS QoS QoS matching Publisher Subscriber Deadline Deadline Deadline Deadline Deadline Deadline Violation © 2009, PrismTech. All Rights Reserved

Latency Budget I need to get there in at most... The LATENCY_BUDGET QoS policy specifies the maximum acceptable delay from the time the data is written until the data is inserted in the receiver's application-cache QoS Policy Applicability RxO Modifiable LATENCY T, DR, DW Y Y BUDGET ‣ The default value of the duration is zero indicating that the delay should be minimized QoS matching ‣ This policy is a hint to the DDS, not something that must be QoS QoS QoS Proprietary Information - Distribution without Expressed Written Permission is Prohibited. monitored or enforced. QoS Topic QoS Name DataWriter writes Type reads DataReader Publisher ... Subscriber ... ... DataWriter writes Type reads DataReader Name TBuff Latency Budget = Latency = TBuff +T1+T2+T3 Topic QoS QoS QoS QoS matching T3 T1 T2 © 2009, PrismTech. All Rights Reserved

Transport Priority VIP Data, stay clear! QoS Policy Applicability RxO Modifiable TRANSPORT T, DW - Y QoS matching PRIORITY QoS QoS QoS QoS Topic QoS The TRANSPORT_PRIORITY QoS policy Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Name is a hint to the infrastructure as to how Publisher DataWriter writes Type reads DataReader Subscriber ... ... ... to set the priority of the underlying Type reads DataReader transport used to send the data. DataWriter writes Name Topic QoS QoS QoS QoS matching © 2009, PrismTech. All Rights Reserved

History Data Lifespan Durability Availability Ownership Ownership Strength Data Availability QoS Policy Applicability RxO Modifiable DURABILITY T, DR, DW Y N DURABILITY T, DW N N SERVICE Data Availability LIFESPAN T, DW - Y HISTORY T, DR, DW N N © 2008, PrismTech. All Rights Reserved

Durability The DURABILITY QoS controls the data availability w.r.t. late joiners, specifically the DDS provides the following variants: QoS Policy Applicability RxO Modifiable DURABILITY T, DR, DW Y N ‣ Volatile. No need to keep data instances DURABILITY SERVICE T, DW N N for late joining data readers ‣ Transient Local. Data instance availability Proprietary Information - Distribution without Expressed Written Permission is Prohibited. QoS matching for late joining data reader is tied to the QoS QoS QoS data writer availability QoS Topic QoS ‣ Transient. Data instance availability Name DataWriter writes Type reads DataReader outlives the data writer Publisher ... ... ... Subscriber ‣ Persistent. Data instance availability DataWriter writes Type reads DataReader outlives system restarts Topic Name QoS QoS QoS QoS matching The DURABILITY_SERVICE QoS provide control over configuration of the service that implements the transient and persistent durability features © 2009, PrismTech. All Rights Reserved

Lifespan For how long will this data be available? ‣ The LIFESPAN QoS policy QoS Policy Applicability RxO Modifiable LIFESPAN T, DW - Y allows to control what happens to stale data QoS matching ‣ It specifies the validity interval QoS QoS QoS Proprietary Information - Distribution without Expressed Written Permission is Prohibited. for data written by the QoS Topic Name QoS DataWriter DataWriter writes Type reads DataReader ‣ Publisher ... Subscriber ... The default validity interval is ... DataWriter writes Type reads DataReader infinite Name Topic QoS QoS QoS QoS matching © 2009, PrismTech. All Rights Reserved

History How many data samples should I keep? The HISTORY QoS policy controls whether the DDS should deliver only the most recent value, attempt to deliver all intermediate values, or do something in between. The policy can be configured to provide the following semantics: ‣ Keep Last. The DDS will only attempt to QoS Policy HISTORY Applicability T, DR, DW RxO N Modifiable N keep the most recent “depth” samples of Proprietary Information - Distribution without Expressed Written Permission is Prohibited. each instance of data identified by its key QoS matching ‣ Keep All. The DDS will attempt to keep all QoS QoS QoS the samples of each instance of data identified by its key. QoS Topic Name QoS ‣ On the DataWriter Samples are kept until DataWriter writes Type reads DataReader delivered to all known subscribers Publisher ... ... ... Subscriber ‣ On the DataReader side samples are kept DataWriter writes Type reads DataReader until the application “takes” them Topic Name QoS QoS QoS QoS matching © 2009, PrismTech. All Rights Reserved

Reliability Destination Presentation Data Delivery Order Data Delivery QoS Policy Applicability RxO Modifiable PRESENTATION P, S Y N RELIABILITY T, DR, DW Y N PARTITION P, S N Y DESTINATION T, DR, DW Y N Data Delivery ORDER OWNERSHIP T, DR, DW Y N OWNERSHIP DW - Y STRENGTH © 2008, PrismTech. All Rights Reserved

Reliability The RELIABILITY QoS indicate the level of guarantee offered by the DDS in delivering data to subscribers. Possible variants are: QoS Policy Applicability RxO Modifiable ‣ Reliable. In steady-state the RELIABILITY T, DR, DW Y N middleware guarantees that all Proprietary Information - Distribution without Expressed Written Permission is Prohibited. samples in the DataWriter history will QoS matching eventually be delivered to all the QoS QoS QoS DataReader QoS Topic QoS ‣ Best Effort. Indicates that it is DataWriter writes Type Name reads DataReader acceptable to not retry propagation of Publisher ... Subscriber ... ... any samples DataWriter writes Type reads DataReader Name Topic QoS QoS QoS QoS matching © 2009, PrismTech. All Rights Reserved

Ownership Who owns the data? The OWNERSHIP QoS specifies whether it is allowed for multiple DataWriters to write the same instance of the data and if so, how these modifications should be arbitrated. Possible choices are: ‣ Shared. Multiple writers are allowed to QoS Policy Applicability RxO Modifiable update the same instance and all the OWNERSHIP T, DR, DW Y N Proprietary Information - Distribution without Expressed Written Permission is Prohibited. updates are made available to the reader QoS matching ‣ Exclusive. Indicates that each instance can QoS QoS QoS only be owned by one DataWriter, but the Topic owner of an instance can change QoS Name QoS dynamically -- due to liveliness changes DataWriter writes Type reads DataReader Publisher ... Subscriber ... ‣ The selection of the owner is controlled by DataWriter writes ... Type reads DataReader the setting of the OWNERSHIP_STRENGTH Name QoS policy Topic QoS QoS QoS QoS matching © 2009, PrismTech. All Rights Reserved

Ownership Strength How strong are you? The OWNERSHIP_STRENGTH Specifies the value of the “strength” used to arbitrate among DataWriters that attempt to modify the same data instance ‣ Data instance are identified QoS matching by the couple (Topic, Key) QoS QoS QoS ‣ The policy applies only if the Proprietary Information - Distribution without Expressed Written Permission is Prohibited. OWNERSHIP is EXCLUSIVE QoS Topic Name QoS DataWriter writes Type reads DataReader Publisher ... Subscriber ... ... QoS Policy Applicability RxO Modifiable OWNERSHIP DW - Y DataWriter writes Type reads DataReader STRENGTH Name Topic QoS QoS QoS QoS matching © 2009, PrismTech. All Rights Reserved

Ownership @ Work ‣ A Topic can have Shared or Exclusive Ownership ‣ Exclusively owned Topics can be modified by a single writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote symbol: "MSFT" StockQuote StockQuote name: "Microsoft Corp." StockQuote symbol: "GOOG" symbol: "GOOG" exchange: "NASD""GOOG" symbol: name: "Google Inc." quote: 33.73 name: "Google Inc." name: "Google Inc." exchange: "NASD" exchange: "NASD" exchange: "NASD" StockQuote StockQuote StockQuote quote: 663.97 W1 quote: 663.97"AAPL" symbol: "AAPL" quote: 663.97 symbol: "AAPL" symbol: name: "Apple Inc." name: "Apple Inc." name: "Apple Inc." exchange: "NASD" exchange: "NASD" exchange: "NASD" R1 quote: 165.37 quote: 165.37 quote: 165.37 STRENGTH=3 StockQuote symbol: "AAPL" StockQuote Inc." name: "Apple symbol: "GOOG""NASD" exchange: name: quote: 165.37 "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W1’ R2 STRENGTH=2 StockQuote symbol: "AAPL" StockQuote Inc." name: "Apple symbol: "GOOG""NASD" exchange: name: quote: 165.37 "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W2’’ R3 STRENGTH=1 © 2009, PrismTech. All Rights Reserved

Managing Hot-Swap ‣ A Topic can have Shared or Exclusive Ownership ‣ Exclusively owned Topics can be modified by a single writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote R1 symbol: "AAPL" name: "Apple Inc." exchange: "NASD" quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote symbol: "AAPL" name: "Apple Inc." exchange: "NASD" R2 quote: 165.37 StockQuote StockQuote StockQuote StockQuote symbol: "MSFT" symbol: "MSFT" symbol: "MSFT" symbol: "GOOG" name: "Microsoft Corp." name: "Microsoft Corp." name: "Microsoft Corp." exchange: "NASD" exchange: "NASD" name: "Google Inc." exchange: "NASD" quote: 33.73 quote: 33.73 quote: 33.73 exchange: "NASD" quote: 663.97 StockQuote R3 symbol: "AAPL" W2’’ name: "Apple Inc." exchange: "NASD" quote: 165.37 STRENGTH=1 © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Shared Global Data Space

Shared Global Data Space Coordination Model ‣ DDS applications are asynchronous and communicate by reading/writing from/to a Global Data Space ‣ DDS applications communicate by simply addressing items in the Global Data Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Space and without any direct knowledge of the parties involved in the production/ consumption of data Consistency Model ‣ The Shared Global Data Space implemented by DDS, can be configured to supports at most the “Eventual Consistency Model” ‣ Under an Eventual Consistency Model we are guaranteed that eventually all application in the system will have a consistent view of the “world”s © 2009, PrismTech. All Rights Reserved

Eventual Consistency & R/W Caches DataReader DataReader 1 1 2 1 DataWriter 1 1 3 1 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 2 1 1 2 Topic DataReader 1 2 2 3 3 1 DataReader Cache 2 2 2 3 3 1 Topic 1 1 DataReader Cache 2 Topic 1 DDS DataWriter Cache 3 1 Topic DataReader Cache Under an Eventual Consistency Model, DDS guarantees that all matched Reader Caches will eventually be identical of the respective Writer Cache © 2009, PrismTech. All Rights Reserved

QoS Impacting the Consistency Model The DDS Consistency Model is a property that can be associated to Topics or further refined by Reader/Writers. The property is controlled by the following QoS Policies: ‣ DURABILITY ‣ VOLATILE | TRANSIENT_LOCAL | TRANSIENT | PERSISTENT ‣ LIFESPAN Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ RELIABILITY ‣ RELIABLE | BEST_EFFORT ‣ DESTINATION ORDER ‣ SOURCE_TIMESTAMP | DESTINATION_TIMESTAMP QoS Policy Applicability RxO Modiﬁable DURABILITY T, DR, DW Y N LIFESPAN T, DW - Y RELIABILITY T, DR, DW Y N DESTINATION ORDER T, DR, DW Y N © 2009, PrismTech. All Rights Reserved

QoS Impacting the Consistency Model DURABILITY RELIABILITY DESTINATION_ORDER LIFESPAN Eventual Consistency VOLATILE RELIABLE SOURCE_TIMESTAMP INF. (No Crash / Recovery) Eventual Consistency TRANSIENT_LOCAL RELIABLE SOURCE_TIMESTAMP INF. Proprietary Information - Distribution without Expressed Written Permission is Prohibited. (Reader Crash / Recovery) Eventual Consistency TRANSIENT RELIABLE SOURCE_TIMESTAMP INF. (Crash/Recovery) Eventual Consistency PERSISTENT RELIABLE SOURCE_TIMESTAMP INF. (Crash/Recovery) Weak Consistency ANY ANY DESTINATION_TIMESTAMP ANY Weak Consistency ANY BEST_EFFORT ANY ANY Weak Consistency ANY ANY ANY N © 2009, PrismTech. All Rights Reserved

Eventual Consistency @ Work DURABILITY RELIABILITY DESTINATION_ORDER LIFESPAN Eventual Consistency (Reader TRANSIENT_LOCAL RELIABLE SOURCE_TIMESTAMP INF. Crash / Recovery) {A} Eventual Consistency TRANSIENT RELIABLE SOURCE_TIMESTAMP INF. {B} (Crash/Recovery) Weak Consistency ANY ANY ANY N {J} Proprietary Information - Distribution without Expressed Written Permission is Prohibited. S = {A, D} S1 P = {A, B} B B P1 m S= {A, B, J} BA A A F S2 J D C P = {D, C, J} P2 K E S = {A} S4 © 2009, PrismTech. All Rights Reserved

Eventual Consistency @ Work DURABILITY RELIABILITY DESTINATION_ORDER LIFESPAN Eventual Consistency (Reader TRANSIENT_LOCAL RELIABLE SOURCE_TIMESTAMP INF. Crash / Recovery) {A} Eventual Consistency TRANSIENT RELIABLE SOURCE_TIMESTAMP INF. {B} (Crash/Recovery) Weak Consistency ANY ANY ANY N {J} Proprietary Information - Distribution without Expressed Written Permission is Prohibited. S = {A, D} S1 B B A m S= {A, B, J} A F S2 J J D D C BA P = {D, C, J} S= {A, B, D, J} P2 K E S3 J S = {A} S4 © 2009, PrismTech. All Rights Reserved A

Design Guidelines ‣ For all (non-periodic) Topics for which an eventually consistent model is required use the following QoS settings: DURABILITY RELIABILITY DESTINATION_ORDER LIFESPAN Eventual Consistency TRANSIENT RELIABLE SOURCE_TIMESTAMP INF. Proprietary Information - Distribution without Expressed Written Permission is Prohibited. (Crash / Recovery) ‣ For information produced periodically, with a period P, where P is small enough to be acceptable as a consistency convergence delay, the following QoS settings will provide an approximation of the eventual consistency: DURABILITY RELIABILITY DESTINATION_ORDER LIFESPAN Eventual Consistency VOLATILE BEST_EFFORT SOURCE_TIMESTAMP INF. (Crash / Recovery) © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Topic Used in next section. Getting the Data struct Counter { int cID; int count; Read Semantics }; #pragma keylist Counter cID

Reading Samples ‣ Read iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Samples are not removed 2 1 2 2 2 3 from the local cache as DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 4 3 5 result of a read 3 1 3 2 3 3 ‣ Read samples can be read Topic again, by accessing the Samples Read Samples not Read cache with the proper options (more later) DataReader Cache © 2009, PrismTech. All Rights Reserved

Reading Samples ‣ Read iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Samples are not removed 2 1 2 2 2 3 from the local cache as DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 3 2 3 3 3 4 3 5 result of a read ‣ Read samples can be read Topic again, by accessing the Samples Read Samples not Read cache with the proper options (more later) DataReader Cache © 2009, PrismTech. All Rights Reserved

Sample, Instance and View States History Depth = 2 ‣ DataReaders are provided with data samples ane state information allowing to detect relevant transitions in the life-cycle of data as well as data writers DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. SampleInfo ‣ Sample State (READ | NOT_READ): Determines wether a sample has already been read by this 1 1 1 2 DataWriter or not. 2 2 2 3 ‣ Instance State (ALIVE, NOT_ALIVE, DISPOSED). 3 1 Determines wether (1) writer exist for the specific Samples instance, or (2) no matched writers are currently Topic available, or (3) the instance has been disposed DataReader Cache ‣ View State (NEW, NOT_NEW). Determines wether this is the first sample of a new (or re-born) instance © 2009, PrismTech. All Rights Reserved

Anatomy of a Read DDS::ReturnCode_t read( TSeq& samples, Proprietary Information - Distribution without Expressed Written Permission is Prohibited. DDS::SampleInfoSeq& info, long max_samples, DDS::SampleStateMask samples_state, DDS::ViewStateMask views_state, DDS::InstanceStateMask instances_state) © 2009, PrismTech. All Rights Reserved

Reading Data Example 1 Example 2 TSeq samples; TSeq samples(N); SampleInfoSeq info; SampleInfoSeq info(N); reader->read(samples, reader->read(samples, info, info, DDS::LENGTH_UNLIMITED, N, Proprietary Information - Distribution without Expressed Written Permission is Prohibited. DDS::NOT_READ_SAMPLE_STATE, DDS::NOT_READ_SAMPLE_STATE, DDS::ANY_VIEW_STATE, DDS::ANY_VIEW_STATE, DDS::ANY_INSTANCE_STATE); DDS::ANY_INSTANCE_STATE); // Do something // Do something reader->return_loan(samples, info); Example 3 reader->read(samples, info, N, DDS::NOT_READ_SAMPLE_STATE, DDS::NEW_VIEW_STATE, DDS::ALIVE_INSTANCE_STATE); © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Topic Used in next section. Getting the Data struct Counter { int cID; int count; Take Semantics }; #pragma keylist Counter cID

Taking Samples ‣ Take iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Taken Samples are 2 1 2 2 2 3 removed from the local DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 4 3 5 cache as result of a take 3 1 3 2 3 3 ‣ Topic Samples not Taken DataReader Cache © 2009, PrismTech. All Rights Reserved

Taking Samples ‣ Take iterates over the available sample instances 1 2 1 3 1 4 ‣ Taken Samples are 2 2 2 3 removed from the local DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 2 3 3 3 4 3 5 cache as result of a take Topic Samples not Taken DataReader Cache © 2009, PrismTech. All Rights Reserved

Taking Samples ‣ Take iterates over the available sample instances 1 3 1 4 ‣ Taken Samples are 2 3 removed from the local DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 3 3 4 3 5 cache as result of a take Topic Samples not Taken DataReader Cache © 2009, PrismTech. All Rights Reserved

Anatomy of a take DDS::ReturnCode_t take( TSeq& samples, Proprietary Information - Distribution without Expressed Written Permission is Prohibited. DDS::SampleInfoSeq& info, long max_samples, DDS::SampleStateMask samples_state, DDS::ViewStateMask views_state, DDS::InstanceStateMask instances_state) © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Topic Used in next section. Writing Data struct Counter { int cID; int count; }; #pragma keylist Counter cID

Writing Samples ‣ Samples are written in the local cache ‣ Writer control the creation 1 1 DataWriter of instances Topic Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ The DDS ensures that the local caches for the matched DataReader will be Last Sample Written eventually consistent with that of the Data Writer DataWriter Cache © 2009, PrismTech. All Rights Reserved

Writing Samples ‣ Samples are written in the local cache ‣ Writer control the creation 1 1 1 2 DataWriter of instances Topic Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ The DDS ensures that the local caches for the matched DataReader will be Last Sample Written eventually consistent with that of the Data Writer DataWriter Cache © 2009, PrismTech. All Rights Reserved

Writing Samples ‣ Samples are written in the local cache ‣ Writer control the creation 1 2 1 3 DataWriter of instances Topic Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ The DDS ensures that the local caches for the matched DataReader will be Last Sample Written eventually consistent with that of the Data Writer DataWriter Cache © 2009, PrismTech. All Rights Reserved

Anatomy of a write DDS::ReturnCode_t write(const T& sample, Proprietary Information - Distribution without Expressed Written Permission is Prohibited. InstanceHandle_t handle); © 2009, PrismTech. All Rights Reserved

Writing Data writer->write(sample, DDS::HANDLE_NIL); Proprietary Information - Distribution without Expressed Written Permission is Prohibited. InstanceHandle_t handle = writer->register_instance(sample); // ... writer->write(sample, handle); © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Topic Used in next section. Managing Data History struct Counter { int cID; int count; }; #pragma keylist Counter cID

How many samples? History Depth = 1 (DDS Default) ‣ The History QoS Controls the 1 1 number of samples-per- DataReader 2 1 instance that will be stored by 3 1 the middleware on behalf of a Topic Reader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Keep Last K. The History QoS can be set so to always have History Depth = 5 the latest K samples 1 1 1 2 1 3 1 4 1 5 ‣ Keep All. The History QoS can 2 1 2 2 2 3 2 4 2 5 be set so keep all samples DataReader 3 3 3 4 produced by the writer and not 3 1 3 2 3 5 yet taken, until resource limits Topic are not reached © 2009, PrismTech. All Rights Reserved

History in Action History Depth = 1 History Depth = 1 (DDS Default) (DDS Default) Network 1 1 1 2 DataReader 2 1 1 2 2 3 DataWriter Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 2 2 2 3 3 1 Topic Topic DataReader Cache DataWriter Cache Note: The Reliability QoS controls wether data is sent reliably, or best-effort, from the DataWriter to matched DataReaders © 2009, PrismTech. All Rights Reserved

History in Action History Depth = 1 History Depth = 1 (DDS Default) (DDS Default) Network 1 2 1 2 DataReader 2 2 2 3 DataWriter Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 2 3 3 1 Topic Topic DataReader Cache DataWriter Cache Note: The Reliability QoS controls wether data is sent reliably, or best-effort, from the DataWriter to matched DataReaders © 2009, PrismTech. All Rights Reserved

History in Action History Depth = 1 History Depth = 1 (DDS Default) (DDS Default) Network 1 2 1 2 DataReader 2 3 2 3 DataWriter Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 3 1 Topic Topic DataReader Cache DataWriter Cache Note: The Reliability QoS controls wether data is sent reliably, or best-effort, from the DataWriter to matched DataReaders © 2009, PrismTech. All Rights Reserved

History in Action History Depth = 2 History Depth = 1 (DDS Default) Network 1 1 1 2 DataReader 2 1 1 2 2 3 DataWriter Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 2 2 2 3 3 1 Topic Topic DataReader Cache DataWriter Cache Note: The Reliability QoS controls wether data is sent reliably, or best-effort, from the DataWriter to matched DataReaders © 2009, PrismTech. All Rights Reserved

History in Action History Depth = 2 History Depth = 1 (DDS Default) Network 1 1 1 2 1 2 DataReader 2 1 2 2 2 3 DataWriter Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 2 3 3 1 Topic Topic DataReader Cache DataWriter Cache Note: The Reliability QoS controls wether data is sent reliably, or best-effort, from the DataWriter to matched DataReaders © 2009, PrismTech. All Rights Reserved

History in Action History Depth = 2 History Depth = 1 (DDS Default) Network 1 1 1 2 1 2 DataReader 2 2 2 3 2 3 DataWriter Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 3 1 Topic Topic DataReader Cache DataWriter Cache Note: The Reliability QoS controls wether data is sent reliably, or best-effort, from the DataWriter to matched DataReaders © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Topic Used in next section. Application / DDS Coordination struct Counter { int cID; int count; }; #pragma keylist Counter cID

Application / DDS Coordination DDS provides three main mechanism for exchanging information with the application ‣ Polling. The application polls from time to time for new data or status changes. The interval might depend on the kind of applications as well as data Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ WaitSets. The application registers a WaitSet with DDS and waits (i.e. is suspended) until one of the specified events has happened. ‣ Listeners. The application registers a listener with a specific DDS entity to be notified when relevant events occur, such as state changes or © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Putting it All Together Writing a DDS Application

Steps for Writing a DDS Application Writing a DDS application can be decomposed in the following few simple steps: ‣ Step#1: Define Topics ‣ Step #2: Identify QoS representing key non-functional invariants for your system ‣ Transport Priority ‣ Deadline ‣ Durability Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Step #3: Define Topics / Partition Mapping ‣ Step #4: Identify Topic Readers/Writers ‣ Step #5: Define QoS requirements for Readers/Writers ‣ History ‣ Latency Budget ‣ Auto-Dispose ‣ Transport Priority ‣ Deadline ‣ Step #6: Code-it in your favorite programming language © 2009, PrismTech. All Rights Reserved

Making the Hello DDS World Step #1: Topic definition ‣ We are going to define a simple key-less topic that will carry the name to greet. module swatch { Proprietary Information - Distribution without Expressed Written Permission is Prohibited. struct hello { string<256> name; }; #pragma keylist hello }; © 2009, PrismTech. All Rights Reserved

Making the Hello DDS World Step #2: Topic QoS ‣ Reliability QoS: RELIABLE ‣ Durability QoS: TRANSIENT Proprietary Information - Distribution without Expressed Written Permission is Prohibited. module swatch { struct hello { string<256> name; }; #pragma keylist hello }; © 2009, PrismTech. All Rights Reserved

Making the Hello DDS World Step #3: Topics/Partitions Mapping ‣ swatch::hello will be mapped into the default-partition (thus no action to take) Proprietary Information - Distribution without Expressed Written Permission is Prohibited. © 2009, PrismTech. All Rights Reserved

Making the Hello DDS World Step #4: Readers/Writers ‣ A Generic DataReader that will read the topic swatch:: hello ‣ We’ll be able to run as many of this as we want ‣ A Generic DataWriter that will read the topic swatch:: hello Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ We’ll be able to run as many of this as we want © 2009, PrismTech. All Rights Reserved

Making the Hello DDS World ‣ Step #5: Define QoS requirements for Readers/Writers ‣ Writer: ‣ Inherit TopicQoS, and ‣ No-Auto Dispose Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ History QoS: Keep Last N ‣ Reader ‣ Inherit TopicQoS, and ‣ History QoS: Keep Last N © 2009, PrismTech. All Rights Reserved

Hello World in a Conceptual Picture History Depth = 2 History Depth = 2 1 1 1 1 2 2 2 2 3 3 DataReader DataWriter Proprietary Information - Distribution without Expressed Written Permission is Prohibited. SampleInfo SampleInfo Network 1 1 1 2 1 1 1 2 2 2 2 2 2 3 2 3 3 1 3 1 Samples Samples swatch::hello swatch::hello DataReader Cache DataWriter Cache © 2009, PrismTech. All Rights Reserved

Visualizing the Strucutre... Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Subscriber Default Partition Publisher Domain Participant Domain © 2009, PrismTech. All Rights Reserved

SIMple Dds == SIMD! ‣ Today we’ll write our DDS application using SIMD ‣ SIMD is a C++ library that takes advantage of C++ Template Meta- Programming to: SIMD ‣ Vastly Improve Productivity Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Simplify Usage ‣ Automate Resource Management (All DDS Entities are Garbage Collected via Ref-Counting) ‣ Zero Overhead © 2009, PrismTech. All Rights Reserved

Hello-pub.cpp (Default QoS) int main(int argc, char* argv[]) { swatch::hello sample; if (!parse_args(argc, argv)) std::stringstream ss; return 1; for (int i = 0; i < N; ++i) { // -- init the SIMD runtime ss << i; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. simd::Runtime::init(); std::string tmp = ss.str() + "." + message; ss.str(""); // -- create the DDS Topic sample.name = DDS::string_dup(tmp.c_str()); simd::Topic<swatch::hello> helloTopic("helloTopic"); std::cout << "<<= " << sample.name << std::endl; writer.write(sample); // -- create the DDS DataWriter usleep(period*1000); simd::DataWriter<swatch::hello> writer(helloTopic, } dwqos); std::cout << "[done]" << std::endl; return 0; } Only 3-lines of DDS-Speciﬁc Code Business Logic © 2009, PrismTech. All Rights Reserved

Hello-pub.cpp int main(int argc, char* argv[]) { if (!parse_args(argc, argv)) return 1; swatch::hello sample; // -- init the SIMD runtime simd::Runtime::init(); std::stringstream ss; simd::TopicQos tqos; for (int i = 0; i < N; ++i) { ss << i; tqos.set_reliable(); Proprietary Information - Distribution without Expressed Written Permission is Prohibited. tqos.set_transient(); std::string tmp = ss.str() + "." + message; ss.str(""); // -- create the DDS Topic sample.name = DDS::string_dup(tmp.c_str()); std::cout << "<<= " << sample.name << std::endl; simd::Topic<swatch::hello> helloTopic("helloTopic", tqos); writer.write(sample); usleep(period*1000); simd::DataWriterQos dwqos(tqos); dwqos.set_keep_last(history_depth); } std::cout << "[done]" << std::endl; dwqos.set_auto_dispose(false); return 0; } // -- create the DDS DataWriter Business Logic simd::DataWriter<swatch::hello> writer(helloTopic, dwqos); © 2009, PrismTech. All Rights Reserved

Hello-sub.cpp (Default QoS) int main(int argc, char* argv[]) { swatch::helloSeq samples; if (!parse_args(argc, argv)) DDS::SampleInfoSeq infos; return 1; while (true) { // -- init the SIMD runtime reader.read(samples, infos); Proprietary Information - Distribution without Expressed Written Permission is Prohibited. simd::Runtime::init(); for (int i = 0; i < samples.length(); ++i) { std::cout << "=>> " << samples[i].name // -- create the DDS Topic << std::endl; simd::Topic<swatch::hello> helloTopic("helloTopic"); } if (samples.length() > 0) // -- create the DDS DataReader std::cout << "--" << std::endl; simd::DataReader<swatch::hello> reader(helloTopic); reader.return_loan(samples, infos); usleep(period*1000); } return 0;} Only 3-lines of DDS-Speciﬁc Code Business Logic © 2009, PrismTech. All Rights Reserved

Hello-sub.cpp int main(int argc, char* argv[]) { if (!parse_args(argc, argv)) return 1; swatch::helloSeq samples; // -- init the SIMD runtime DDS::SampleInfoSeq infos; simd::Runtime::init(); while (true) { simd::TopicQos tqos; reader.read(samples, infos); Proprietary Information - Distribution without Expressed Written Permission is Prohibited. tqos.set_reliable(); for (int i = 0; i < samples.length(); ++i) { tqos.set_transient(); std::cout << "=>> " << samples[i].name << std::endl; // -- create the DDS Topic } simd::Topic<swatch::hello> helloTopic("helloTopic", if (samples.length() > 0) tqos); std::cout << "--" << std::endl; simd::DataReaderQos drqos(tqos); reader.return_loan(samples, infos); drqos.set_keep_last(history_depth); usleep(period*1000); } // -- create the DDS DataReader return 0;} simd::DataReader<swatch::hello> reader(helloTopic, drqos); Business Logic © 2009, PrismTech. All Rights Reserved

DDS Today Development ‣ UML Profile for DDS Application ‣ IDL Derived API Object/Relational Mapping Integration Data Local Reconstruction Layer (DLRL) Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Complete knowledge of the information model Ownership Durability Content Subscription ‣ Consistent information model across systems Minimum Proﬁle Deployment Data Centric Publish/Subscribe (DCPS) ‣ Optimized for LAN, multi-LAN deployment Real-Time Publish/Subscribe Protocol ‣ WAN deployments supported via: DDS Interoperability Wire Protocol ‣ Multicast-enabled VPN UDP/IP ‣ DDSI Unicast © 2009, PrismTech. All Rights Reserved

DSS Standard Directions The DDS Standard will be extended in the following directions: ‣ Simplicity. New, API for ISO STD C++, and Java 6 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Extensibility and Integration. Extensible and Dynamic Topic Types ‣ Scale. Support for large-scale WAN deployments ‣ Data Access. Support for accessing the DDS Global Data Space via SOA and Web Technologies (e.g. REST, WS-*, JMS, etc.) © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Simplicity

ISO STD C++ PSM for DDS Context ‣ The DDS API is currently derived from the existing IDL2C++ standard ‣ The existing C++ API, although being very powerful, does not always hide accidental complexity ‣ The existing C++ API does not leverage the latest ISO STD C++ features, such as standard containers, iterators, templates, etc. Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ The existing C++ API leaves the burden of DDS entities manual memory management to the programmer Upcoming Standard ‣ Fully aligned with the ISO STD C++ Standard and also looking forward to the upcoming C++0x ‣ Simpler, neater, safer API ‣ Automatic Memory Management (still debated) Status ‣ Final joint revised submission due for the Dec 2009 OMG Meeting © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Extensibility

Extensible and Dynamic Topic Types Context ‣ The OMG DDS standard has been designed to effectively support statically defined data models. ‣ This assumption ensures that topics data types are known at compile time, that every member of the DDS global data space agrees precisely on the same Proprietary Information - Distribution without Expressed Written Permission is Prohibited. topic-type association ‣ Extensions are captured by either DLRL classes inheritance, or by IS-A relations at the DCPS layer ‣ Under this model: ‣ it is hard to cope with data models evolving over time unless all the elements of the system are upgraded consistently. ‣ It is hard to deal with dynamically defined topics, although in some case this is achievable by means of proprietary APIs. © 2009, PrismTech. All Rights Reserved

Extensible and Dynamic Topic Types Upcoming Standard: Four dimension considered ‣ Topic’s Type Definition ‣ Available syntax for defining the types associated with a topic ‣ Currently IDL only is supported ‣ Support for sparse types as well as for some pre-defined topic types Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Encapsulation Definition ‣ Controls how a topic is encapsulated on the wire ‣ Currently CDR is specified ‣ Extensibility Mechanism ‣ Controls how Topics Types can be extended without breaking interoperability w.r.t. portion of the system relying on older version of the same topic ‣ Dynamic API ‣ An API which allows to publish and subscribe topics for a Topic that is not known a priory © 2009, PrismTech. All Rights Reserved

Type Extensibility & Evolution ‣ Type evolution captures the changes that a type experience in a system throughout its life-time ‣ Type extension capture, via sub-typing, refinement and specialization. ‣ NOTE: Often Type Extension is (ab)used in order to capture type evolution, but this is an “trick” more than a good practice struct RadarTrack { Proprietary Information - Distribution without Expressed Written Permission is Prohibited. long trackID; t ion long x; eE volu long y; Typ float estErr; }; struct RadarTrack { #pragma keylist RadarTrack trackID long trackID; long x; long y; }; #pragma keylist RadarTrack trackID struct RadarTrack3D : RadarTrack { long z; }; Type E #pragma keylist RadarTrack3D xtensi RadarTrack3D::RadarTrack::trackID o n © 2009, PrismTech. All Rights Reserved

Extensible and Dynamic Topic Types Status ‣ Joint Revised Submission available on OMG WWW ‣ Joint Revision Submission due on Sep 2009 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Scale

DDS for Ultra-Large-Scale Systems Context ‣ The DDS Standard covers very well the “System” Use Case ‣ Natural extension point exist in the DDS Standard properly support Ultra-Large-Scale Systems, over WAN ‣ Further standardization is required to ensure that the nice DDS properties enjoyed on a Proprietary Information - Distribution without Expressed Written Permission is Prohibited. “System” are carried on in a “Ultra-Large-Scale System” w/o limiting interoperability or trading it for scalability Upcoming Standard ‣ Ultra-Large-Scale extensions for DDS Status ‣ 2 Concept paper submitted by leading vendors to the OMG RTWS ’09 ‣ RFP process to be kick-started at the Sept ’09 OMG meeting © 2009, PrismTech. All Rights Reserved

DDS for Ultra-Large-Scale Systems ‣ DDS-Level Routers will be introduced that H 1,1,h N2,5 establish a Peer-to-Peer overlay that relays R R N1,1 H 1,1,1 DDS data over the WAN SN H 1,1,0 H 2,1,m ‣ A Network of Super-Nodes is used for H 1,0,k SN Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ensuring: R SN N2,0 H 1,0,1 ‣ Scalable (Global) Discovery N1,0 H 2,0,i H 1,0,0 R N2,4 ‣ Subscription Management (when crossing the R boundaries of a System) R SN N2,1 ‣ Support for establishing communication between SN H 2,1,n Pub and Sub (perhaps via STUNT to circumvent NAT/FIREWALL issues) N2,2 R R N2,3 H 2,1,j H 2,1,k H 2,1,h © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Data Access

RESTful DDS Context ‣ REST is becoming the dominant way of accessing resources via the internet and a new style of implementing Web Services, supported by the latest Web2.0 applications, e.g. Twitter, Blogspot, etc. ‣ REST provides basic CRUD functionalities that nicely map to DDS ‣ REST’s resource-based approach nicely maps to DDS Data-based approach, since a Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Topic nicely maps into a resource ‣ A RESTful access to the DDS global data space is natural and enables to naturally address Topics, e.g.: ‣ http://buket.mywww.com/system/domain/partition/topic/ Upcoming Standard ‣ RESTful DDS Status ‣ Initial RFP Drafting © 2009, PrismTech. All Rights Reserved

Concluding Remarks State of the Art ‣ The OMG DDS Standard uniquely addresses some of the key requirements for next generation Mission Application Critical Systems ‣ The OMG DDS is one of the few middleware standard Object/Relational Mapping Data Local Reconstruction Layer (DLRL) defining both API as well as Wire-Protocol, thus Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ensuring application portability and interworking Content Ownership Durability ‣ The DDS Standard has been recommended/ Subscription mandated by key administration throughout the world Minimum Proﬁle as well as adopted by the most challenging Defense Data Centric Publish/Subscribe (DCPS) programs Real-Time Publish/Subscribe Protocol Looking Ahead DDS Interoperability Wire Protocol ‣ The DDS Standard is swiftly moving forward to expand UDP/IP its applicability, simplify its use, and facilitate integration © 2009, PrismTech. All Rights Reserved

Online Resources http://www.opensplice.com/ http://twitter.com/acorsaro/ emailto:opensplicedds@prismtech.com Proprietary Information - Distribution without Expressed Written Permission is Prohibited. http://bit.ly/1Sreg http://opensplice.blogspot.com http://www.dds-forum.org http://www.youtube.com/OpenSpliceTube http://portals.omg.org/dds © 2009, PrismTech. All Rights Reserved

OMG DDS: The Data Distribution Service for Real-Time Systems

0 comments

Notes on slide 1

1 Favorite