Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Beyond messaging

on

  • 1,783 views

The OMG Data Distribution Service (DDS) has brought a disruptive wave of innovation into what is commonly referred to as “Messaging” technology. DDS raises the abstraction level from unstructured, ...

The OMG Data Distribution Service (DDS) has brought a disruptive wave of innovation into what is commonly referred to as “Messaging” technology. DDS raises the abstraction level from unstructured, or loosely structured, messages to sound data types that allow architects and application developers to deal with concepts close to their problem space as opposed to having to worry about “plumbing” -- as it happens as with many messaging technologies. As it is common for disruptive innovations to take some time to be digested, many people still see DDS as an ultra-fast real-time messaging -- yet this is only the tip of the iceberg.

This presentation will explain how Data Distribution is different from Messaging and as a result how DDS differs from messaging technologies such as JMS. The webcast will cover the architectural benefits brought by data centricity as well as how the features introduced by the recently adopted DDS specification on Dynamic and Extensible Topic Types will support the design of efficient, extensible and evolvable systems.

Statistics

Views

Total Views
1,783
Views on SlideShare
1,662
Embed Views
121

Actions

Likes
1
Downloads
74
Comments
0

3 Embeds 121

http://www.prismtech.com 118
http://icorsaro.net 2
http://dds4u.blogspot.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Beyond messaging Beyond messaging Presentation Transcript

    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Beyond Messaging.... Angelo Corsaro, Ph.D. Chief Technology Officer PrismTech OMG DDS SIG Co-Chair angelo.corsaro@prismtech.com
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD ./messaging
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD What is Messaging about? P2P Messaging ‣ Messaging is about distributing Queue messages generated by a producer to its consumers Pub/Sub Messaging ‣ Messaging technologies often supports P2P and Pub/Sub interactions ‣ The producer/consumer exchange are Topic often mediated by brokers
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD What is a Message? ‣ A message is the mechanism provided to messaging applications for exchanging information Header Body ‣ A Message is usually composed by a header and an opaque body ‣ In other terms, a message is an untyped (or weakly typed) information holder
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD An Example: AMQP Messages ‣ AMQP define a message as made by a header and a body ‣ The body is an opaque sequence of bytes ‣ Thus, data serialization is delegated at the application
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD An Example: JMS Messages ‣ JMS define a message as made by a header and a body ‣ The body can be of five different types ‣ Text ‣ Map ‣ Array of Bytes ‣ Stream of native types ‣ Serialized Object
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD ISO/OSI view of Messaging ‣ Messaging operates at the Session layer but Messaging App does deal with the Messaging presentation layer (e.g. data representation). ‣ Messaging applications have to deal directly with data presentation
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Messaging in JMS Sending Message ‣ The user data has to be represented in a format // Create a TextMessage supported by the JMS message TextMessage message = queueSession.createTextMessage(); body // Create and Set the Message Body ‣ Object serialization has some String msgText = "body"; message.setText(msgText); shortcomings: // Send the Message ‣ not interoperable with other queueSender.send(message); programming languages ‣ Not the most time/space efficient way of encoding your data
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Messaging in JMS Receiving Message ‣ Application have to figure-out // Receive the Message the actual type of the Message m = queueReceiver.receive(); “Message” // Figure-out what it is if (m instanceof TextMessage) { ‣ Data has to be extracted TextMessage message = (TextMessage) m; String body = message.getText(); from the message System.out.println("Received: " + body); } ‣ Lot’s of potential for else if (m instanceof MapMessage){ // do something else introducing errors that a } sounder type-system could else { // Handle Error prevent }
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD In Summary ‣ Messaging provides a mechanism for shipping messages from producers to consumers ‣ The Message is a first class citizen, application data is not. ‣ The conversion of user data-types into the types supported by the messaging has to be addressed at the application level ‣ Byte sequences for AMQP ‣ Text/Map/Bytes/Stream/Object for JMS ‣ etc..
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD ./data-distribution
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD What is Data Distribution? [1/2] TrakCo.VehiclePosition TrakCo.VehicleSpeed ‣ Data distribution is about struct VehiclePosition { struct VehicleSpeed { making application defined string plate; string plate; long x, y; } long dx, dy; } data available where needed and when needed VehiclePosition plate x y ‣ Data is a first class concept, “A123” “B456” 101 202 303 404 it can be created, updated, “C789” 505 606 read and eventually disposed VehicleSpeed plate dx dy “A123” 50 0 ‣ The last value (or last N- “B456” “C789” 0 10 45 30 values) of a Data is available
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD What is Data Distribution? [2/2] TrakCo.VehiclePosition TrakCo.VehicleSpeed struct VehiclePosition { struct VehicleSpeed { string plate; string plate; long x, y; } long dx, dy; } ‣ The details of how application VehiclePosition data is encapsulated and plate “A123” x 101 202 y propagated are dealt by the “B456” “C789” 303 404 505 606 data distribution technology VehicleSpeed plate dx dy “A123” 50 0 “B456” 0 45 “C789” 10 30
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Topics Data in DDS [1/4] TrackCo.VehiclePosition ‣ The definition of Data in DDS is tied Name to the definition of a Topic ‣ A Topic is the association between: ‣ Unique Name Type Topic QoS ‣ Type VehiclePosition { Reliability ‣ QoS Deadline, Priority ‣ A Topic can have associated a key Transient, ... in order to express data instances } identities
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Topic Instances Data in DDS [2/4] ‣ A Topic key can be made of an arbitrary number of Topic Type Topic TrakCo.VehiclePosition attributes Topic Type VehiclePosition ‣ Each unique key value identify a Topic Key plate x y “A123” 101 202 Topic Instance Topic Instances “B456” 303 202 ‣ Topic Instance, can be Created, “C789” 101 606 Read, Updated, and Disposed (CRUD)
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Samples Data in DDS [3/4] ‣ An update for a Topic Instance is called Sample ‣ DDS can maintain a configurable history of samples per Topic Instance Samples Topic TrakCo.VehiclePosition Topic Type VehiclePosition VehiclePosition VehiclePosition Topic Key plate x y plate x y plate x y “A123” 101 202 “A123” 110 202 “A123” 150 222 Topic Instances “B456” 303 202 “B456” 333 202 “B456” 366 202 “C789” 101 606 “C789” 80 606 “C789” 50 606 Time
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Instances Lifecycle Data in DDS [4/4] Creation ‣ A Topic Instance is automatically created the first time it is written Topic TrakCo.VehiclePosition Topic Type VehiclePosition Topic Key plate x y Topic Instances “A123” 101 202 Time
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Instances Lifecycle Data in DDS [4/4] Update ‣ Subsequent writes will simply update the value of the instance Topic TrakCo.VehiclePosition Topic Type VehiclePosition VehiclePosition Topic Key plate x y plate x y Topic Instances “A123” 101 202 “A123” 170 202 Time
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Instances Lifecycle Data in DDS [4/4] Dispose ‣ An instance can be explicitly disposed Topic TrakCo.VehiclePosition Topic Type VehiclePosition VehiclePosition VehiclePosition ! d Topic Key plate x y plate x y plate x sey Topic Instances “A123” 101 202 “A123” 170 202 “A123” is po 170 202 D Time
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD ISO/OSI View of Data Distribution ‣ Data Distribution operates at the Application Session+Presentation Data layers Distribution ‣ Data Distribution Applications transparently read/ write application defined data
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Data Distribution in DDS [1/2] Writing Data ‣ User defined types can be // Create data written with a simple write VehiclePosition pos(“A101”, 101, 202); operation. Encapsulation and // Write Data distribution are taken care by dataWriter.write(pos); DDS [Syntax compliant with the DDS-PSM-C++ upcoming standard as currently defined at http://code.google.com/p/dds-psm-cxx ]
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Data Distribution in DDS [2/2] Reading Data ‣ User defined types can be // Create data+info containers written with a simple write std::vector<VehiclePosition> vpos; std::vector<SampleInfo> info; operation. Encapsulation and // Read Data distribution are taken care by dataReader.read(vpos, info); DDS [Syntax compliant with the DDS-PSM-C++ upcoming standard as currently defined at http://code.google.com/p/dds-psm-cxx ]
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD In Summary ‣ Data Distribution provides a mechanism for sharing user-defined distributed data types and transparently reading and writing instances of these types ‣ Data types and data instances are first class citizens. This implies that data-life-cycle transitions are also dealt-with ‣ User defined Data-Types are preserved end-to-end. Their encoding is transparently dealt by the data distribution service
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD ./vehicles-tracking
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD A Vehicle Tracking System Imagine to develop a s system that has to deal with the following: ‣ Identify when a car enters and leaves a city ‣ Track the position of each identified car and draw a path of the last N positions ‣ Track the instantaneous as well as the window average speed. Fine if the window average exceeds the speed-limit
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Vehicle Tracking with Messaging ‣ How are you going to manage the presence/absence of cars? ‣ This concern has to be implemented in your application logic. ‣ How are you going to easily distinguish between cars? ‣ Your application will have to deal with it. ‣ How are you going to maintain a window of vehicle speed so to enforce speed limits? ‣ Again, your application will have to explicitly maintain a last K-values cache of the speed for each care and make sure that this is updated each time you receive an update. Is there a better way to do this?
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Vehicle Tracking with DDS ‣ How are you going to manage the presence/absence of cars? ‣ DDS provide that information through entity life-cycle management! ‣ How are you going to easily distinguish between cars? ‣ DDS allow to access samples on a per-instance basis. ‣ How are you going to maintain a window of vehicle speed so to enforce speed limits? ‣ DDS provide support for data history.
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Vehicle Tracking with DDS Much More with DDS ‣ Dynamic Discovery ‣ Cars will be discovered as soon as there is network connectivity ‣ QoS ‣ Local as well as End-to-End resource usage and temporal characteristic can be controlled via DDS QoSes ‣ Filtering ‣ Filter can be performed on the whole content and is not limited by the attributes that are attached to the header by data publishers ‣ DDS is wire-interoperable, language, and OS independent, thus it is easy to deploy it anywhere
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Tracking System Topics Topic TrakCo.Vehicle Topic Type ‣ Three different topics are used Topic Key struct Vehicle { string plate; to represent the Vehicle, its string model; } position and the Speed Topic TrakCo.VehiclePosition ‣ The three topics define the Topic Type struct VehiclePosition { same key so that information Topic Key string plate; long x, y; } associated to the same car can be easily correlated Topic TrakCo.VehicleSpeed Topic Type struct VehicleSpeed { Topic Key string plate; long dx, dy; }
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Tracking System Topics QoS TrakCo.Vehicle Reliability History Durability Destination Order struct Vehicle { RELIABLE KEEP_LAST TRANSIENT SOURCE_TIMESTAMP string plate; string model; } TrakCo.VehiclePosition struct VehiclePosition { Reliability History Durability Destination Order string plate; BEST_EFFORT KEEP_LAST(3) VOLATILE SOURCE_TIMESTAMP long x, y; } TrakCo.VehicleSpeed struct VehicleSpeed { Reliability History Durability Destination Order string plate; BEST_EFFORT KEEP_LAST(5) VOLATILE SOURCE_TIMESTAMP long dx, dy; }
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD ./extensible-topics
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Dealing with Type Evolutions ‣ A DDS-based System might need to evolve its information model without inducing any global impact on required upgrades nor breaking interoperability between new and existing part for the system ‣ As an example let’s assume that we have now some cars that can provide more information on the vehicle, how do we capture this evolution?
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD DDS ExtensibleType System ‣ DDS is now (see Extensible and Dynamic Topic Types Specification) equipped with a structural type system ‣ This type system support type extension (evolution in general) relying on a structural sub-type operator and a set of annotations and QoS that allow to control and constraint the type system
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD ./xtype-in-action
    • DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS Type Extensibility © 2010 PrismTech. All Rights Reserved. Problem ‣ What if we need to extend the VehiclePosition type to add the error estimate on the x and y direction? ‣ How can we do this with no impact on running applications?
    • DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS Type Extensibility Solution ‣ Simply extend the existing VehiclePosition and properly annotate new © 2010 PrismTech. All Rights Reserved. attributes ‣ Existing application will detect that the new type is an extension and will “project/extend” it to the known type ‣ Type “projection/extensions” can be controlled via QoS struct VehiclePosition { struct VehiclePosition { string plate; //@key string plate; //@key long x, y; } long x, y; float xe, ye;}
    • DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS Type Extensibility Problem © 2010 PrismTech. All Rights Reserved. ‣ OK, that was good. But now I need to add an optional image to my Vehicle ‣ How can I add an optional field? Furthermore, I don’t want the figure to be copied into the topic, I’d like it to be a reference. Can I do this to limit the number of copies and reduce he memory utilization?
    • DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS Type Extensibility Solution © 2010 PrismTech. All Rights Reserved. ‣ Simply extend the existing Vehicle and properly annotate new attributes struct Vehicle { struct Vehicle { string plate; //@key string plate; //@key string model; string model; } sequence<octet> photo; //@optional @shared }
    • DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS ./efficiency © 2010 PrismTech. All Rights Reserved.
    • DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS Encoding Efficiency Size for Serialized CoFlight Flight Data Plan © 2010 PrismTech. All Rights Reserved. ‣ DDS serialization is several times more efficient than JSON, XML and YAML ‣ Differences can be quite extreme when dealing with see Esposito et al. OMG RTWS 2008 complex types DDS encoding is 10x more space efficient than XML!
    • DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS Encoding Impact on RTT DDS RTT with different Serialization Format see Esposito et al. OMG RTWS 2008 © 2010 PrismTech. All Rights Reserved. ‣ Inefficient encoding can easily dominate RTT ‣ DDS native encoding is the most efficient among extensible encodings!
    • DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS DDS is Very Efficient! © 2010 PrismTech. All Rights Reserved. ‣ DDS implementations are capable of delivering very low and predictable Latency (usec) latencies ‣ No other Pub/Sub technology can approach similar level of performance! Size (bytes)
    • DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS OMG DDS In Summary ‣ DDS provides a strongly typed extensible type system © 2010 PrismTech. All Rights Reserved. ‣ Types can be evolved and extended without breaking backward compatibility nor loosing type-safety ‣ Type “Projection/Promotions” can be fully controlled via QoS ‣ This flexibility does not come at the cost of introducing the time +space overhead typical of XML-based approaches DDS provides extensibility without compromising efficiency!
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD ./summing-up
    • OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DDS OpenSplice DD Concluding Remarks ‣ Data Distribution give you far more than plain Messaging: ‣ higher level of abstraction and more built-in functionalities ‣ end-to-end type-safety ‣ extensibility ‣ performance ‣ Data Distribution is well beyond messaging, it is the most effective way of designing loosely coupled distributed applications.
    • OpenSplice DDS Delivering Performance, Openness, and Freedom http://www.opensplice.com/ References http://www.opensplice.org/ http://www.slideshare.net/angelo.corsaro emailto:opensplicedds@prismtech.com http://bit.ly/1Sreg http://twitter.com/acorsaro/ http://www.youtube.com/OpenSpliceTube http://opensplice.blogspot.com