• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
DDS Made Simple
 

DDS Made Simple

on

  • 1,208 views

 

Statistics

Views

Total Views
1,208
Views on SlideShare
1,163
Embed Views
45

Actions

Likes
3
Downloads
40
Comments
0

3 Embeds 45

https://twitter.com 41
http://www.icorsaro.net 2
http://icorsaro.net 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    DDS Made Simple DDS Made Simple Presentation Transcript

    • OpenSpliceDDS Angelo CORSARO, Ph.D. Chief Technology Officer OMG DDS Sig Co-Chair PrismTech angelo.corsaro@prismtech.com DDS Made Simple
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS What I’ll Cover ☐ Understand DDS and its Data Sharing abstraction ☐ Learn about the benefits that Data Sharing provide over plain Messaging ☐ Learn how much simpler is to build distributed systems with DDS than with messaging technologies
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Distributed System Definition A Distributed System is a collection autonomous software components distributed across a network that coordinate their activities and share system resources to run as a “single system” N.B. It is useful to remark how this definition does not mention message passing or shared memory, those are implementation details
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Distributed Systems Challenges A number of challenges are inherent to the nature of distributed systems: ☐ Transparency ☐ Scalability ☐ Dependability ☐ Performance ☐ Flexibility
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Building Distributed Systems Several abstractions have been proposed to build Distributed Systems: ☐ Distributed Shared Memory ☐ Message Passing/Queueing ☐ Distributed Objects ☐ Tuple Spaces
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Building Distributed Systems Several abstractions have been proposed to build Distributed Systems: ☐ Distributed Shared Memory ☐ Message Passing/Queueing ☐ Distributed Objects ☐ Tuple Spaces
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Message Queueing ☐ Message Queueing, as suggested by its name, provide a coordination abstraction based on the exchange of messages between distributed processes ☐ Message delivery is often mediated by brokers ☐ A Message is composed by a header and a body, where the body is opaque (often binary) data Broker BodyHeader Message
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Tuple Space ☐ Tuple Space provide the abstraction of a Global Tuple Space that can be used by autonomous applications to coordinate by inserting (out), reading (rd) and removing (in) tuples ☐ The Tuple Space model was initially introduced by Gelernter and Carrero’s Linda and extended over the years <a,b,c> <x,y, <j, k>> <x,y, <j, k>> out(<a,b,c>) rd(<a,_,_>) in(<15,_>)eval(...)
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Tuple Spaces vs Messaging ☐ Tuple Spaces provides spatial and temporal decoupling along with content awareness ☐ Tuples remain in the tuple space up to when they are not removed ☐ Tuple Spaces allow application to share information ☐ Overall Tuple Spaces provide a high-level abstraction for building distributed systems ☐ Messaging doesn’t provide content awareness and usually provides limited temporal decoupling ☐ Messages usually disappear once they are delivered to consumers ☐ Messaging allows application to exchange messages. Information has to be reconstructed from messages ☐ Messaging provides a relatively low-level abstraction for building distributed systems
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Why is Messaging so Popular? At this point you may ask yourself: If Tuple Spaces are so cool, why are messaging technologies so popular? The answer is actually pretty simple: Tuple Spaces where originally quite inefficient and this lead people to adopt different approaches, such as messaging and distributed objects. Nowadays, however, there are plenty of technologies deriving from the tuple space concept that have very high performance (often superior than messaging).
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Why is Messaging so Popular? At this point you may ask: OK, that makes sense. Yet there are still plenty of people using messaging technologies. Why? The answer is again pretty simple: The adoption of Tuple Space inspired technologies is growing steadily, but many people are still using messaging since this is what they are familiar with. But enlightenment will eventually hit them ;-)
    • OpenSpliceDDS DDS
    • OpenSpliceDDS Ba D D Sics
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Data Distribution Service ☐ Topics: Unit of data sharing ☐ DataWriters: data producers ☐ DataReaders: data consumers DDS provides a relaxed Tuple Space Abstraction abstraction based on: DDS Global Data Space ... TopicA TopicB TopicC TopicD Data Writer Data Writer Data Writer Data Writer Data Reader Data Reader Data Reader Data Reader For Real-Time Systems
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Data Distribution Service ☐ DataWriters and DataReaders are automatically and dynamically matched by the DDS Dynamic Discovery ☐ A rich set of QoS allows to control existential, temporal, and spatial properties of data DDS Global Data Space ... TopicA TopicB TopicC TopicD Data Writer Data Writer Data Writer Data Writer Data Reader Data Reader Data Reader Data Reader For Real-Time Systems
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Fully Distributed Tuple Space ☐ DDS Implements a fully distributed Tuple Space with relaxed consistency. ☐ DDS provides eventual consistency as opposed to strong consistency ☐ In addition, DDS provides a local “take” and a global “dispose” in place of the “in” operation <a,b,c> <x,y, <j, k>> <x,y, <j, k>> out(<a,b,c>) rd(<a,_,_>) in(<15,_>)eval(...) Traditional Tuple Space DDS Data Space Logical Global Data Space Physical Local Data Space The  local  Data  Space  is  a   projection  of  the  Logical   Global  Data  Space.  This   projection  reflects  the   interest  of  the  given   application. The  Global  Data  Space  is  an   abstraction  in  DDS  built  by   the  composition  of  Local   Data  Spaces.
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS What DDS Provides ☐ Location Transparency => Dynamic Discovery ☐ Anonymity and Temporal Decoupling => Tuple Space ☐ Data Centricity => Tuple Space ☐ Dependability => Fault-Tolerant and Secure ☐ Scalability => Fully distributed architecture easy to scale-out ☐ Portability => OS and Programming Language Independence ☐ Interoperability => Standardized Wire Protocol
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS IDL DDS Topics ☐ A Topic defines a class of streams ☐ A Topic has associated a user defined extensible type and a set of QoS policies ☐ The Topic name, type and QoS defines the key functional and non-functional invariants ☐ Topics can be discovered or locally defined DURABILITY, DEADLINE, PRIORITY, … “TVehicleDynamics” Topic Type Name QoS struct VehicleDynamics { long vid; long x; long y; long dx; long dy;}; #pragma keylist VehicleDynamics vid VehicleDynamics
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Instances ☐ Each unique key value identifies a unique topic instance ☐ DDS not only demultiplexes “streams” but provides also lifecycle information ☐ A DDS DataWriter can write multiple instances struct Person { long ssn; String name; String surname; }; #pragma keylist Person ssn <101,Leslie, Lamport > <010,Edsger, Dijkstra> <110,Haskell, Curry>
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Anatomy of a DDS Application
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Running App ☐ During this webcast we’ll be looking at a few DDS aspects through the iShapes applications ☐ The advantage of targeting the iShapes application is that we can actually “see” what our application will do
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS OpenSplice Mobile
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS In Action...
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS iShapes Information Model “Circle” Topic Type Name QoS ShapeType IDL struct ShapeType { string color; long x; long y; long shapesize;}; #pragma keylist ShapeType color “Square” Topic Type Name QoS ShapeType “Triangle” Topic Type Name QoS ShapeType
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Anatomy of a DDS Application Domain Participant [ISO C++ API] auto dp = DomainParticipant(domainId);
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Anatomy of a DDS Application Domain Participant Topics auto dp = DomainParticipant(domainId); // Create a Topic auto topic = Topic<ShapeType>(dp, “Circle”); [ISO C++ API]
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Anatomy of a DDS Application Domain Participant Topics auto dp = DomainParticipant(domainId); // Create a Publisher auto pub = Publisher(dp); // Create a Topic auto topic = Topic<ShapeType>(dp, “Circle”); Publisher // Create a Subscriber auto sub = Subscriber(dp); Subscriber [ISO C++ API]
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Anatomy of a DDS Application Domain Participant Data Writer Topics // Create a DataWriter auto writer = DataWriter<ShapeType>(pub, topic); auto dp = DomainParticipant(domainId); // Create a Publisher auto pub = Publisher(dp); // Create a Topic auto topic = Topic<ShapeType>(dp, “Circle”); Publisher // Create a Subscriber auto sub = Subscriber(dp); Subscriber Data Reader // Create a DataWriter auto reader = DataReader<ShapeType>(sub, topic); [ISO C++ API]
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Putting it all together 1 int main(int argc, char* argv[]) { 2 try { 3 DomainParticipant dp(0); 4 Topic<ShapeType> topic(dp, "Circle"); 5 Publisher pub(dp); 6 DataWriter<ShapeType> dw(pub, topic); 7 10 for (int i = 0; i < N; ++i) { 11 ShapeType bc = {"RED", i, i, 60}; 12 ShapeType rc = {"BLUE", N-i, N-i, 60}; 13 dw.write(bc); 14 // You can also write with streaming operators! 15 dw << rc; 16 std::cout << "." << std::flush; 17 usleep(sleep_time); 18 } 19 20 } catch (const dds::core::Exception& e) { 21 std::cout << e.what() << std::endl; 22 } 23 return 0; 24 }
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Anatomy of a DDS Application Domain Participant Data Writer Topics // Create a DataWriter DataWriter<ShapeType> writer = pub.createDataWriter<ShapeType>(topic); DomainParticipant dp = dpf.createParticipant(domainId); // Create a Publisher Publisher pub = dp.createPublisher(); // Create a Topic Topic<ShapeType> topic = dp.createTopic(“Circle”,ShapeType.class); Publisher // Create a Subscriber Subscriber sub = dp.createSubscriber(); Subscriber Data Reader [Java 5 API] Bootstrap ServiceEnvironment env = ServiceEnvironment.createInstance(this.getClass.getClassLoader); DomainParticipantFactory dpf = DomainParticipantFactory.getInstance(env) // Create a DataWriter DataReader<ShapeType> reader = sub.createDataReader<ShapeType>(topic);
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Anatomy of a DDS Application Dependency Injection through Implicits Data Writer Topics // Create a DataWriter val writer = DataWriter[ShapeType](topic) [Scala API] import dds.config.DefaultEntities._ // Create a Topic val topic = Topic[ShapeType](“Circle”) Data Reader // Create a DataWriter val reader = DataReader[ShapeType](topic)
    • OpenSpliceDDS Writing Data
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Writing Data in DDS writer.write(SpapeType(“RED”, 10, 20, 30)); // -- or -- writer << SpapeType(“RED”, 10, 20, 30); [ISO C++ API]
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Writing Data in DDS [Java 5 API] writer.write(new SpapeType(“RED”, 10, 20, 30));
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Writing Data in DDS [Scala API] writer.write(new ShapeType(“RED”, 10, 20, 30)) // -- Or equivalently: writer write (new ShapeType(“RED”, 10, 20, 30))
    • OpenSpliceDDS Reading Data
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Reading Data // Read Data auto data = reader.read(); // -- you can also read with streaming operators auto data = reader << read(); // Print the received data on the console std::for_each(samples.data().begin(), samples.data.end(), demo::printShape) [ISO C++ API]
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Reading Data // You can read data on your favorite container using iterators auto samples = std::vector<Samples<ShapeType>>(MAX_SHAPES); // Read data into the provided container reader.read(samples.begin(), MAX_SHAPES); // Print the received data on the console std::for_each(samples.data().begin(), samples.data.end(), demo::printShape) [ISO C++ API]
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Reading Data      //  Read  Data      Sample.Iterator<ShapeType>  iterator  =  reader.read();                                                        while  (iterator.hasNext())  {        //  Assuming  ShapeType  ovrrides  toString    out.println(iterator.next().getData());    } [Java API]
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Reading Data      //  Read  Data      reader.read  foreach  {  s  =>  println(s.data())  }                                                   [Scala API]
    • OpenSpliceDDS Selecting Samples
    • OpenSpliceDDS Data Selectors
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Cherry Picking in DDS ☐ DDS provides some very flexible mechanisms for selecting the data to be read: ☐ Data Content ☐ Data Status ☐ These mechanisms are composable
    • OpenSpliceDDS Content-Based Data Selection
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Filters and Queries ☐ DDS Filters allow to control what gets into a DataReader cache ☐ DDS Queries allow to control what gets out of a DataReader cache ☐ Filters are defined by means of ContentFilteredTopics ☐ Queries operate in conjunction with read operations ☐ Filters and Queries are expressed as SQL where clauses DataReader Cache DataReader ... ... ... ... Filter Query Application
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Query struct VehicleDynamics { long vid; long x; long y; long dx; long dy;}; #pragma keylist VehicleStatus vid // Define the query and the parameters std::vector<std::string> p; p.push_back("100"); p.push_back("100"); dds::core::Query q("x < %0 AND y < %1", p.begin(), p.end()); auto data = reader .select() .content(q) .read(); [DDS C++ API 2010]
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Instances ☐ DDS provides a very efficient way of reading data belonging to a specific Topic Instance ☐ Obviously, one could use queries to match the key’s value, but this is not as efficient as the special purpose instance selector // C++ auto data = reader .select() .instance(handle) .read(); // Scala val data = reader read(handle)
    • OpenSpliceDDS State-Based Selection
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Sample, Instance, and View State ☐ The samples included in the DataReader cache have associated some meta-information which, among other things, describes the status of the sample and its associated stream/instance ☐ The Sample State (READ, NOT_READ) allows to distinguish between new samples and samples that have already been read ☐ The View State (NEW, NOT_NEW) allows to distinguish a new instance from an existing one ☐ The Intance State (ALIVE, NOT_ALIVE_DISPOSED, NOT_ALIVE_NO_WRITERS) allows to track the life-cycle transitions of the instance to which a sample belongs
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS State Selector in Action // Read only new samples auto data = reader .select() .state(status::DataState::new_data()) .read() // Read any samples from live instances auto data = reader .select() .state(status::DataState::any_data()) .read() C++ // Read only new samples val data = reader read // Read any samples from live instances val data = reader read(SampleSelector.AnyData) Scala
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Putting all Together ☐ Selectors can be composed in a flexible and expressive manner C++ auto data = reader ! .select() .instance(handle) ! ! .state(status::DataState::new_data()) ! ! .content(q) .read();
    • OpenSpliceDDS QoS
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS QoS Model ☐ QoS-Policies control local and end-to-end properties of DDS entities ☐ Local properties controlled by QoS are related resource usage ☐ End-to-end properties controlled by QoS are related to temporal and spatial aspects of data distribution ☐ Some QoS-Policies are matched based on a Request vs. Offered Model thus QoS-enforcement Publisher DataWriter Topic Type QoS Name writes QoS DataWriter Topic Typewrites Subscriber DataReaderreads DataReaderreads ... QoS Name QoS QoS QoS QoS matching ...... QoS QoS Type Matching DomainParticipant DomainParticipant QoS QoS
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS QoS Policies QoS Policy Applicability RxO Modifiable USER_DATA TOPIC_DATA GROUP_DATA DURABILITY DURABILITY SERVICE HISTORY PRESENTATION RELIABILITY PARTITION DESTINATION ORDER LIFESPAN DP, DR, DW N Y ConfigurationT N Y Configuration P, S N Y Configuration T, DR, DW Y N Data Availability T, DW N N Data Availability T, DR, DW N N Data Availability P, S Y N Data Delivery T, DR, DW Y N Data Delivery P, S N Y Data Delivery T, DR, DW Y N Data Delivery T, DW N Y Data Delivery [T: Topic] [DR: DataReader] [DW: DataWriter] [P: Publisher] [S: Subscriber] [DP: Domain Participant]
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS QoS Policies QoS Policy Applicability RxO Modifiable DEADLINE LATENCY BUDGET TRANSPORT PRIORITY TIME BASED FILTER OWNERSHIP OWNERSHIP STRENGTH LIVELINESS T, DR, DW Y Y Temporal/ Importance Characteristics T, DR, DW Y Y Temporal/ Importance Characteristics T, DW N Y Temporal/ Importance Characteristics DR N Y Temporal/ Importance Characteristics T, DR, DW Y N ReplicationDW N Y Replication T, DR, DW Y N Fault-Detection [T: Topic] [DR: DataReader] [DW: DataWriter] [P: Publisher] [S: Subscriber] [DP: Domain Participant]
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Setting QoS Policies C++ // Setting Partition QoS-Policy on Publisher qos::PublisherQos pubQos; pubQos << policy::Partition("Partition"); Publisher pub(dp, pubQoS); // Setting various QoS-Policy on a Topic qos::TopicQos tqos; tqos << policy::Reliability::Reliable() << policy::Durability::Transient() << policy::History::KeepLast(5); Topic<VehicleDynamics> topic(dp,"Partition", tqos);
    • OpenSpliceDDS Ipse Dixit For non latin speakers: "He, himself, said it"
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS The “Ipse Dixit” Dejavu ☐ As an historical reference“Ipse Dixit” was used by the Holy Inquisition to push back any argument that would contrast with established knowledge ☐ A famous example was the argument around the Geo- Centric model of the Solar System ☐ What has this to do with Middleware... Let me show you!
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS What’s Wrong with this Paper? ☐ When looking at DDS the Authors did not refer to the state of the art ☐ DDS is discarded as being too complicated (really?!?!) -- they did not check out the new APIs available since 2010 ☐ Beyond that, I think the authors where comparing a bit apples and oranges since DDS provides far more than ZeroMQ!
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS DDS vs. 0MQ -- What’s Simpler? import  dds.config.DefaultEntities._ //  Create  a  Topic val  topic  =  Topic[ShapeType](“Circle”) //  Create  a  DataWriter val  writer  =  DataWriter[ShapeType](topic) //  Create  a  DataWriter writer  write  (new  ShapeType(“RED”,  1,2,3)) import zmq import time //  Create  a  Context context = zmq.Context() //  Create  a  Socket publisher = context.socket (zmq.PUB) //  Bind  the  socket  to  a  name publisher.bind ("ipc://nasdaq-feed") //  Send  Data publisher.send (message) Sending Data
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS DDS vs. 0MQ -- What’s Simpler? import  zmq import  time //  Create  a  Context context  =  zmq.Context() //  Create  a  Socket subscriber  =  context.socket  (zmq.SUB) //  !!!  MANUALLY  CONNECT  IT  !!!  What  if  your   //  topology  changes?!? subscriber.connect  ("tcp://192.168.55.112:5556") subscriber.connect  ("tcp://192.168.55.201:7721") //  Define  subscrption subscriber.setsockopt  (zmq.SUBSCRIBE,  "NASDAQ") //  Receive  message message  =  subscriber.recv() //  ...  Now  decode  the  msg  and  do  something   //  with  it Receiving Data import  dds.config.DefaultEntities._ //  Create  a  Topic val  topic  =  Topic[ShapeType](“Circle”) //  Create  a  DataReader val  reader  =  DataReader[ShapeType](topic) //  Create  a  DataWriter reader.read()  foreach(s  =>  println(s.data))
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS Concluding Remarks ☐ DDS provides a powerful and feature-rich abstraction for building distributed systems ☐ This technology is widely used in mission and business critical systems and it being swiftly adopted in data-centric/big-data systems
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS
    • Copyright  2013,  PrismTech  –    All  Rights  Reserved. OpenSpliceDDS ¥@prismtech ¥@acorsaro ¥youtube.com/opensplicetube ¥slideshare.net/angelo.corsaro ¥ crc@prismtech.com ¥sales@prismtech.com ¥opensplice.com ¥forums.opensplice.org ¥opensplice.org ¥opensplicedds@prismtech.com :: Connect with Us ::