Beyond messaging

Angelo Corsaro, Ph.D.
Chief Technology Ofﬁcer
PrismTech
OMG DDS SIG Co-Chair
angelo.corsaro@prismtech.com
OpenSpliceDDSOpenSpliceDDOpenSpliceDDSOpenSpliceDDSOpenSpliceDDS
Beyond Messaging....

./messaging

What is Messaging about?
‣ Messaging is about distributing
messages generated by a producer to
its consumers
‣ Messaging technologies often supports
P2P and Pub/Sub interactions
‣ The producer/consumer exchange are
often mediated by brokers
Topic
Queue
P2P Messaging
Pub/Sub Messaging

‣ A message is the mechanism provided
to messaging applications for
exchanging information
‣ A Message is usually composed by a
header and an opaque body
‣ In other terms, a message is an
untyped (or weakly typed) information
holder
What is a Message?
Header Body

An Example:AMQP Messages
‣ AMQP define a message as made
by a header and a body
‣ The body is an opaque sequence
of bytes
‣ Thus, data serialization is
delegated at the application

An Example: JMS Messages
‣ JMS define a message as made by a
header and a body
‣ The body can be of five different types
‣ Text
‣ Map
‣ Array of Bytes
‣ Stream of native types
‣ Serialized Object

ISO/OSI view of Messaging
‣ Messaging operates at
the Session layer but
does deal with the
presentation layer (e.g.
data representation).
‣ Messaging applications
have to deal directly
with data presentation
Messaging
Messaging App

Messaging in JMS
‣ The user data has to be
represented in a format
supported by the JMS message
body
‣ Object serialization has some
shortcomings:
‣ not interoperable with other
programming languages
‣ Not the most time/space efficient way
of encoding your data
// Create a TextMessage
TextMessage message =
queueSession.createTextMessage();
// Create and Set the Message Body
String msgText = "body";
message.setText(msgText);
// Send the Message
queueSender.send(message);
Sending Message

Messaging in JMS
‣ Application have to figure-out
the actual type of the
“Message”
‣ Data has to be extracted
from the message
‣ Lot’s of potential for
introducing errors that a
sounder type-system could
prevent
// Receive the Message
Message m = queueReceiver.receive();
// Figure-out what it is
if (m instanceof TextMessage) {
TextMessage message = (TextMessage) m;
String body = message.getText();
System.out.println("Received: " + body);
}
else if (m instanceof MapMessage){
// do something else
}
else {
// Handle Error
}
Receiving Message

In Summary
‣ Messaging provides a mechanism for shipping messages from
producers to consumers
‣ The Message is a first class citizen, application data is not.
‣ The conversion of user data-types into the types supported by
the messaging has to be addressed at the application level
‣ Byte sequences for AMQP
‣ Text/Map/Bytes/Stream/Object for JMS
‣ etc..

./data-distribution

What is Data Distribution?
‣ Data distribution is about
making application defined
data available where needed
and when needed
‣ Data is a first class concept,
it can be created, updated,
read and eventually disposed
‣ The last value (or last N-
values) of a Data is available
VehicleSpeedVehicleSpeedVehicleSpeed
plate
“A123”
“B456”
“C789”
dx dy
50 0
0 45
10 30
VehiclePositionVehiclePositionVehiclePosition
plate
“A123”
“B456”
“C789”
x y
101 202
303 404
505 606
[1/2]
TrakCo.VehiclePosition
struct VehiclePosition {
string plate;
long x, y; }
TrakCo.VehicleSpeed
struct VehicleSpeed {
string plate;
long dx, dy; }

What is Data Distribution?
‣ The details of how application
data is encapsulated and
propagated are dealt by the
data distribution technology VehicleSpeedVehicleSpeedVehicleSpeed
plate
“A123”
“B456”
“C789”
dx dy
50 0
0 45
10 30
plate
“A123”
“B456”
“C789”
x y
101 202
303 404
505 606
[2/2]
string plate;
long x, y; }
TrakCo.VehicleSpeed
string plate;
long dx, dy; }

Data in DDS
‣ The definition of Data in DDS is tied
to the definition of a Topic
‣ A Topic is the association between:
‣ Unique Name
‣ Type
‣ QoS
‣ A Topic can have associated a key
in order to express data instances
identities
VehiclePosition
TrackCo.VehiclePosition
{
Reliability
Deadline,
Priority
Transient,
...
}
TopicType QoS
Name
[1/4]
Topics

Data in DDS
‣ A Topic key can be made of an
arbitrary number of Topic Type
attributes
‣ Each unique key value identify a
Topic Instance
‣ Topic Instance, can be Created,
Read, Updated, and Disposed
(CRUD)
[2/4]
plate
“A123”
“B456”
“C789”
x y
101 202
303 202
101 606
TrakCo.VehiclePositionTopic
TopicType
Topic Instances
Topic Key
Topic Instances

Data in DDS
‣ An update for a Topic Instance is called Sample
‣ DDS can maintain a configurable history of samples per
Topic Instance
[3/4]
plate
“A123”
“B456”
“C789”
x y
101 202
303 202
101 606
TopicType
Topic Instances
Topic Key
plate
“A123”
“B456”
“C789”
x y
110 202
333 202
80 606
plate
“A123”
“B456”
“C789”
x y
150 222
366 202
50 606
Samples
Time
Samples

Data in DDS
Creation
‣ A Topic Instance is automatically created the first time it is
written
[4/4]
Instances Lifecycle
plate
“A123”
x y
101 202
TopicType
Topic Instances
Topic Key
Time
NEW

Data in DDS
Update
‣ Subsequent writes will simply update the value of the instance
[4/4]
Instances Lifecycle
plate
“A123”
x y
101 202
TopicType
Topic Instances
Topic Key
Time
NEW
plate
“A123”
x y
170 202

Data in DDS
Dispose
‣ An instance can be explicitly disposed
[4/4]
Instances Lifecycle
plate
“A123”
x y
101 202
TopicType
Topic Instances
Topic Key
Time
NEW
plate
“A123”
x y
170 202
plate
“A123”
x y
170 202
Disposed!

ISO/OSIView of Data Distribution
‣ Data Distribution
operates at the
Session+Presentation
layers
‣ Data Distribution
Applications
transparently read/
write application
defined data
Data
Distribution
Application

Data Distribution in DDS
‣ User defined types can be
written with a simple write
operation. Encapsulation and
distribution are taken care by
DDS
[1/2]
// Create data
VehiclePosition pos(“A101”, 101, 202);
// Write Data
dataWriter.write(pos);
Writing Data
[Syntax compliant with the DDS-PSM-C++ upcoming standard as
currently deﬁned at http://code.google.com/p/dds-psm-cxx ]

Data Distribution in DDS
‣ User defined types can be
written with a simple write
operation. Encapsulation and
distribution are taken care by
DDS
[2/2]
// Create data+info containers
std::vector<VehiclePosition> vpos;
std::vector<SampleInfo> info;
// Read Data
dataReader.read(vpos, info);
Reading Data
[Syntax compliant with the DDS-PSM-C++ upcoming standard as
currently deﬁned at http://code.google.com/p/dds-psm-cxx ]

In Summary
‣ Data Distribution provides a mechanism for sharing user-defined
distributed data types and transparently reading and writing
instances of these types
‣ Data types and data instances are first class citizens. This
implies that data-life-cycle transitions are also dealt-with
‣ User defined Data-Types are preserved end-to-end. Their
encoding is transparently dealt by the data distribution service

./vehicles-tracking

AVehicleTracking System
Imagine to develop a s system that
has to deal with the following:
‣ Identify when a car enters and
leaves a city
‣ Track the position of each
identified car and draw a path of
the last N positions
‣ Track the instantaneous as well
as the window average speed.
Fine if the window average
exceeds the speed-limit
NEW
BYE
NEW

VehicleTracking with Messaging
‣ How are you going to manage the presence/absence of cars?
‣ This concern has to be implemented in your application logic.
‣ How are you going to easily distinguish between cars?
‣ Your application will have to deal with it.
‣ How are you going to maintain a window of vehicle speed so to
enforce speed limits?
‣ Again, your application will have to explicitly maintain a last K-values cache
of the speed for each care and make sure that this is updated each time
you receive an update.
Is there a better way to do this?

VehicleTracking with DDS
‣ How are you going to manage the presence/absence of cars?
‣ DDS provide that information through entity life-cycle management!
‣ How are you going to easily distinguish between cars?
‣ DDS allow to access samples on a per-instance basis.
‣ How are you going to maintain a window of vehicle speed so to
enforce speed limits?
‣ DDS provide support for data history.

VehicleTracking with DDS
‣ Dynamic Discovery
‣ Cars will be discovered as soon as there is network connectivity
‣ QoS
‣ Local as well as End-to-End resource usage and temporal characteristic can
be controlled via DDS QoSes
‣ Filtering
‣ Filter can be performed on the whole content and is not limited by the
attributes that are attached to the header by data publishers
‣ DDS is wire-interoperable, language, and OS independent, thus it
is easy to deploy it anywhere
Much More with DDS

Tracking SystemTopics
‣ Three different topics are used
to represent the Vehicle, its
position and the Speed
‣ The three topics define the
same key so that information
associated to the same car
can be easily correlated
TopicType
Topic Key
string plate;
long x, y; }
TrakCo.VehicleSpeedTopic
TopicType
Topic Key
string plate;
long dx, dy; }
TrakCo.VehicleTopic
TopicType
Topic Key
struct Vehicle {
string plate;
string model; }

Tracking SystemTopics QoS
string plate;
long x, y; }
TrakCo.VehicleSpeed
string plate;
long dx, dy; }
TrakCo.Vehicle
struct Vehicle {
string plate;
string model; }
Reliability History Durability Destination Order
RELIABLE KEEP_LAST TRANSIENT SOURCE_TIMESTAMP
BEST_EFFORT KEEP_LAST(3) VOLATILE SOURCE_TIMESTAMP
BEST_EFFORT KEEP_LAST(5) VOLATILE SOURCE_TIMESTAMP

./extensible-topics

Dealing withType Evolutions
‣ A DDS-based System might need to evolve its information model
without inducing any global impact on required upgrades nor
breaking interoperability between new and existing part for the
system
‣ As an example let’s assume that we have now some cars that
can provide more information on the vehicle, how do we capture
this evolution?

DDS ExtensibleType System
‣ DDS is now (see Extensible and Dynamic Topic Types
Specification) equipped with a structural type system
‣ This type system support type extension (evolution in general)
relying on a structural sub-type operator and a set of
annotations and QoS that allow to control and constraint the
type system

./xtype-in-action

©2010PrismTech.AllRightsReserved.
DDSOMGDDSOMGDDSOMGDDSOMGDDSOMGDDSOMGDDSOMGDDS
Type Extensibility
Problem
‣ What if we need to extend the VehiclePosition type to add the
error estimate on the x and y direction?
‣ How can we do this with no impact on running applications?

Type Extensibility
Solution
‣ Simply extend the existing VehiclePosition and properly annotate new
attributes
‣ Existing application will detect that the new type is an extension and
will “project/extend” it to the known type
‣ Type “projection/extensions” can be controlled via QoS
string plate; //@key
long x, y; }
long x, y;
float xe, ye;}

Type Extensibility
Problem
‣ OK, that was good. But now I need to add an optional image to
my Vehicle
‣ How can I add an optional field? Furthermore, I don’t want the
figure to be copied into the topic, I’d like it to be a reference.
Can I do this to limit the number of copies and reduce he
memory utilization?

Type Extensibility
Solution
‣ Simply extend the existing Vehicle and properly annotate
new attributes
struct Vehicle {
string model; }
struct Vehicle {
string model;
sequence<octet> photo;
//@optional @shared
}

./efﬁciency

Encoding Efﬁciency
‣ DDS serialization is several
times more efficient than
JSON, XML and YAML
‣ Differences can be quite
extreme when dealing with
complex types
Size for Serialized CoFlight Flight Data Plan
see Esposito et al. OMG RTWS 2008
DDS encoding is 10x more space efﬁcient than XML!

Encoding Impact on RTT
see Esposito et al. OMG RTWS 2008
DDS RTT with different Serialization Format
‣ Inefficient encoding can
easily dominate RTT
‣ DDS native encoding is
the most efficient among
extensible encodings!

Size (bytes)
Latency(usec)
‣ DDS implementations are
capable of delivering very
low and predictable
latencies
‣ No other Pub/Sub
technology can approach
similar level of
performance!
DDS isVery Efﬁcient!

In Summary
‣ DDS provides a strongly typed extensible type system
‣ Types can be evolved and extended without breaking backward
compatibility nor loosing type-safety
‣ Type “Projection/Promotions” can be fully controlled via QoS
‣ This flexibility does not come at the cost of introducing the time
+space overhead typical of XML-based approaches
DDS provides extensibility without compromising efﬁciency!

./summing-up

Concluding Remarks
‣ Data Distribution give you far more than plain Messaging:
‣ higher level of abstraction and more built-in functionalities
‣ end-to-end type-safety
‣ extensibility
‣ performance
‣ Data Distribution is well beyond messaging, it is the most effective
way of designing loosely coupled distributed applications.

http://www.opensplice.com/
http://www.opensplice.org/
emailto:opensplicedds@prismtech.com
http://www.youtube.com/OpenSpliceTube http://opensplice.blogspot.com
http://bit.ly/1Sreg
http://www.slideshare.net/angelo.corsaro
http://twitter.com/acorsaro/
D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m
OpenSplice DDS
References

Beyond messaging

More Related Content

Viewers also liked

Similar to Beyond messaging

More from Angelo Corsaro

Recently uploaded

Beyond messaging