High Performance Distributed Computing with DDS and Scala

High Performance Distributed
Computing with DDS and Scala
OpenSplice DDS

Angelo CORSARO, Ph.D.
Chief Technology Ofﬁcer
OMG DDS Sig Co-Chair
PrismTech
angelo.corsaro@prismtech.com

OpenSplice DDS

Background

Data Distribution Service
For Real-Time Systems

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ Introduced in 2004 to address
the Data Distribution Real-Timeliness Scale

challenges faced by a wide
class of Defense and
Aerospace Applications
OpenSplice DDS

Key requirement for the
Parallelism Determinism Throughput, Availability Scalability, Persistence, Security
☐
standard were to deliver very Systemic
Signal
Data Real-Time Information
Near Real-Time Fault-
Tolerant Information
Complex Information

high and predictable
Processing Processing Management
Processing Processing

performance while scaling
from embedded to ultra- Parallel Systems Distributed Systems

large-scale deployments


Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ Recommended by key Real-Timeliness Scale

administration worldwide,
e.g. DoD, MoD, EUROCAE,
etc.
OpenSplice DDS

Widely adopted across
Parallelism Determinism Throughput, Availability Scalability, Persistence, Security
☐
several different domains, Systemic
Signal
Data Real-Time Information
Near Real-Time Fault-
Tolerant Information
Complex Information

e.g., Automated Trading,
Processing Processing Management
Processing Processing

Simulations, SCADA,
Telemetry, etc. Parallel Systems Distributed Systems

Core Standard

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Application
☐ DDS is nowadays a family
Data Centric Publish Subscriber (DCPS)
of standards Content
Ownership Durability
Subscription
OpenSplice DDS

☐ The two standards at the Minimum Proﬁle
core are:
☐ DDS v1.2 -- API DDS Interoperability Wire Protocol - DDSI-RTPS

☐ DDSI v2.1 -- Wire Protocol
UDP/IP
Application

Standards: What For?

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Common “Language”
Data Model +
OpenSplice DDS

QoS Requirements

Interoperability
DDSI Wire Interoperability

OpenSplice DDS

DDS
Application

API Standard
Standards: What For?

Portability

Copyright
2011,
PrismTech
–

All
Rights
Reserved.

Defense and Aerospace

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
OpenSplice DDS

Integrated Modular Vetronics Training & Simulation Systems Naval Combat Systems

Air Trafﬁc Control & Management Unmanned Air Vehicles Aerospace Applications

Commercial Applications

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
OpenSplice DDS

Agricultural Vehicle Systems Large Scale SCADA Systems Smart Cities

Train Control Systems Complex Medical Devices High Frequency Auto-Trading

OpenSplice DDS

D
D
BaS ics


DDS provides a Topic-Based Publish/

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Subscribe abstraction based on: Data
Reader
Data
Writer
☐ Topics: data distribution subject’s
OpenSplice DDS

Data
Reader
Data TopicD
Writer
DataWriters: data producers
TopicA
☐
Data
TopicB Reader
Data
Writer
☐ DataReaders: data consumers TopicC
...

Data Data
Writer Reader

DDS Global Data Space


Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ DataWriters and DataReaders
are automatically and Data
Reader
dynamically matched by the Data
Writer

DDS Dynamic Discovery
OpenSplice DDS

Data
Reader
Data TopicD
Writer
TopicA
☐ A rich set of QoS allows to Data
Reader
TopicB
control existential, temporal,
Data
Writer
TopicC
...
and spatial properties of data Data Data
Writer Reader

DDS Global Data Space

OpenSplice DDS

DDS Entities

Domain Participant
Domain (e.g. Domain 123)

Domain
Participant

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Topic
Partition (e.g. “Telemetry”, “Shapes”, )

T1 Publisher
T1 T3 Subscriber
Tx
OpenSplice DDS

Topic Instances/Samples
Ty
Tb
Ta DataWrter DataReader
Tc

☐ A DomainParticipant provides access to a DDS Domain
☐ DDS Domains are identified by natural numbers, such as 0, 1, 2, ...
☐ Each DDS Domain has associated some unique communication and
discovery endpoints, automatically deduced by the Domain ID

OpenSplice DDS

Topics

DDS Topics
“Circle”, “Square”, “Triangle”, ...
☐ A Topic defines a class of streams,

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
also called topic instances Name
☐ A Topic has associated a unique Topic
name, a user defined extensible

Typ

S
OpenSplice DDS

type and a set of QoS policies DURABILITY,

Qo
e
DEADLINE,
ShapeType
☐ QoS Policies capture the Topic PRIORITY,
…
non-functional invariants
struct ShapeType {
☐ Topics can be discovered or @Key
string color;
locally defined long x;
long y;
long shapesize;
};

Topic Instances
☐ Each unique key value

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Instances Instances
identifies a unique stream
of data color =”Green”

DDS not only
struct ShapeType {
☐ color =”red” @Key string color;
Topic
OpenSplice DDS

long x; long y;

demultiplexes “streams” color = “Blue”
long shapesize;};

but provides also lifecycle
information
☐ A DDS DataWriter can
write multiple instances

OpenSplice DDS

Topic Types

Topic Types Type System
☐ DDS features a structural type system and the sub-type

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
relationship ( <: ) defined by this type system is used to guide
subscription matching

☐ In general, a DataReader[X] will match a DataWriter[Y], where
OpenSplice DDS

X and Y are DDS Topic types, iff Y <: X

☐ Annotations available to:
☐ Constraint the subtype relationship
☐ Control the set of attributes considered when performing the
subtype test

Type-Level Annotations
☐ The nature of a DDS Topic Type can be controlled through

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
annotations applicable at a type or attribute level

☐ Specifically the @Extensibility annotation controls the allowed
“extensibility”, in the sense of sub-typing for the given type
OpenSplice DDS

☐ Possible values are:
☐ Final: the type is sealed and can’t be changed
☐ Extensible: Only monotonic extensions are allowed, e.g. “adding
attributes to the end”
☐ Mutable: Arbitrary extensions are allowed such as attribute re-ordering,
removal, etc.

Attribute-Level Annotations
☐ Attribute semantics can also be controlled via an extensible

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
annotation system

☐ Some of the most commonly used annotations are:
@Optional: This attribute is allowed to be “nil”
OpenSplice DDS

☐
☐ @Shared: This attribute behaves like a pointer
☐ @MustUnderstand: This attribute must be present in order to match
subscriptions
☐ @ID: Ordinal value associated to the type
☐ @Key: This attribute is part of the topic key

Publishers and Subscribers
OpenSplice DDS

Publishers / Subscribers

Domain
Participant

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Topic

T1 Publisher
T1 T3 Subscriber
Tx
OpenSplice DDS

Ty
Tb
Tc

Publishers and Subscribers in DDS play two roles:
☐ Coordinate actions across DataReaders/DataWriters
☐ e.g. Atomic writes across data-writers
☐ Control access to DDS partitions, a mechanism provided by DDS to further

DDS Partitions
Domain

Partitions DDS Publishers and

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐
Subscribers can specify a
M:0:0 M:0:1 M:0:2 list of regular expressions
representing the partitions
OpenSplice DDS

to join
M:1:0 M:1:1 M:1:2
☐ Actual partition can be
any string, but some
M:2:0 M:2:1 M:2:2 structure can be given in
order mimic hierarchy

DDS Partitions Matching
Domain

Partitions

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
M:0:0 M:0:1 M:0:2 M:0:*
OpenSplice DDS

M:1:0 M:1:1 M:1:2

M:2:0 M:2:1 M:2:2

Domain

Partitions

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
M:0:0 M:0:1 M:0:2
OpenSplice DDS

M:1:0 M:1:1 M:1:2

M:2:0 M:2:1 M:2:2
M:*:2

Domain

Partitions

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
M:0:0 M:0:1 M:0:2
OpenSplice DDS

M:1:0 M:1:1 M:1:2

M:2:0 M:2:1 M:2:2 M:0:0, M:1:1, M:2:2

DataReaders & DataWriters
OpenSplice DDS

DataReader / DataWriter

Domain
Participant

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Topic

T1 Publisher
T1 T3 Subscriber
Tx
OpenSplice DDS

Ty
Tb
Tc

☐ DataWriters allow to write samples for a specific Topic with a given QoS
☐ DataReaders allow to read samples for a specific Topic with a given QoS
☐ The Samples written/read are those belonging to the partitions associated with the
Publisher/Subscriber

Subscription Matching
OpenSplice DDS

Subscription Matching
For a DataWriter and a DataReader to match the following

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
conditions have to hold:

☐ The Topic Name has to be the identical
OpenSplice DDS

☐ Said X the Topic Type associated with the DataWriter and Y the
Topic Type associated with the Data reader, then X <: Y (recall that
annotations control the result subtype test...)

☐ The QoS with which the DataWriter produces data matches or
exceeds the DataReader QoS

Type Matching
struct ShapeType3D { struct ShapeType3D {
@Key @Key
string color; string color;
long x; long x;
long y; long y;

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
long shapesize; DW long shapesize;
DR1
long z; long z;
}; };

Circle
OpenSplice DDS

Circle

☐ As ShapeType3D <: ShapeType
DW and DR2 match Circle

struct ShapeType {

DDS will take care of projecting
@Key
☐ DR2
string color;

ShapeType3D into ShapeType
long x;
long y;
long shapesize;
};

Type Matching
@Extensibility(MUTABLE_EXTENSIBILITY)
@Extensibility(MUTABLE_EXTENSIBILITY) struct ShapeTypeC {
struct ShapeTypeA { @Key @ID(0)
@Key @ID(0) string color;
string color; @ID(10) @MustUnderstand
@ID(10) @MustUnderstand DW1 long x;
DR1

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
long x; @ID(11) @MustUnderstand
@ID(11) @MustUnderstand long y; };
long y;
@ID(12) @Optional
long z;
@ID(20) @Optional
☐ In this example, as a long shapesize; }; Circle
Circle

consequence of
OpenSplice DDS

annotations, DR1 Circle
Circle

and DR2 match @Extensibility(MUTABLE_EXTENSIBILITY)
struct ShapeTypeB {

DW1 and DW2
@Key @ID(0)
@Extensibility(MUTABLE_EXTENSIBILITY)
string color;
struct ShapeTypeA {
@ID(10) @MustUnderstand
@Key @ID(0)
long x; DR2
string color;
@ID(11) @MustUnderstand DW2 @ID(10) @MustUnderstand
long y; };
long x;
@ID(11) @MustUnderstand
long y;
@ID(12) @Optional
long z;
@ID(20) @Optional
long shapesize; };

Anatomy of a DDS Application
OpenSplice DDS

Stepping into Scala
OpenSplice DDS

Angelo CORSARO, Ph.D.
Chief Technology Ofﬁcer
OMG DDS Sig Co-Chair
PrismTech
angelo.corsaro@prismtech.com

What is Scala

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ Scala (pronounced Skah-lah) stands for “Scalable language”

☐ It is a language that carefully and creatively blends Object
Oriented and Functional constructs into a statically typed
OpenSplice DDS

language with sophisticated type inference

☐ Scala targets the JVM and .NET runtime and is 100% compatible
with Java

Why Should You Care?
Scala is simple to write, extremely compact and easy to

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐
understand

☐ Scala is strongly typed with a structural type system
OpenSplice DDS

☐ Scala is an extensible language (many construct are build in the
standard library)

☐ Scala makes it easy to design Domain Specific Language

Case Study: Complex Numbers
OpenSplice DDS

Complex Numbers

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ To explore some of the nice features of Scala, let’s see how we
might design a Complex number class

☐ What we expect to be able to do is all mathematical operations
OpenSplice DDS

between complex numbers as well as scalar multiplications and
division
☐ [(1+i2)+2*(3-i5)*(i4)]/(1+i3)
☐ ~(1+i2) [conjugate]
☐ !(3+i4) [Modulo]

Constructor
☐ Scala allows to implicitly create constructors and attributes starting from

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
a simple argument list associated with the class declaration
OpenSplice DDS

class Complex(val re: Float, val im: Float)

In Java
public class Complex {

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
! private final float re;
! private final float im;

! public Complex(float re, float im) {
! ! this.re = re;
OpenSplice DDS

! ! this.im = im;
! }
! public Complex(float f) {
! ! this.re = f;
! ! this.im = 0;
! }

! public float re() { return re;}

! public float im() { return im;}

Methods
☐ Everything in Scala is a method even operators

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ Methods name can be symbols such as *, !, +, etc.
OpenSplice DDS

def + (c: Complex) : Complex = Complex(re + c.re, im + c.im)

or, taking advantage of type inference....
def + (c: Complex) = Complex(re + c.re, im + c.im)

In Java...

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
public Complex add(Complex that) {
return new Complex(this.re() + that.re(),
OpenSplice DDS

this.im() + that.im());
}

As a result...
val result = Complex(1,2) + Complex(2,3)

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
OpenSplice DDS

Complex c1 = new Complex(1, 2);
Complex c2 = new Complex (3,4);
Complex c3 = c1.add(c2);
or...
Complex c3 = (new Complex(1, 2).add(new Complex (3,4));

Companion Object
☐ Scala does not have the concept of static methods/attributes
☐ On the other hand it provides built-in support for Singletons,

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
which are specified with the “object” keyword as opposed to
the “class” keyword
OpenSplice DDS

object Complex {
The companion object, is def apply(real: Float, img: Float) = new Complex(real, img)

the object associated with def apply(real: Float) = new Complex(real, 0)
a class, which shares the
same name and provides implicit def floatToReComplex (f: Float) = new ReComplex(f)

typically helper methods implicit def intToReComplex(i : Int) = new ReComplex(i)
}

“apply” Magic
☐ When an instance of a class is followed by parentheses with a list of zero

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
or more parameters, the compiler invokes the apply method for that
instance
☐ This is true for an object with a defined apply method (such as a
companion object), as well as an instance of a class that defines an
OpenSplice DDS

apply method

val result = Complex(1,2)
is the same as....
val result = Complex.apply(1,2)

Negation and Scalar Multiplication

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ In order to design a Complex class that is well integrated in our type
system we should be able to support the following cases:
☐ -(a+ib)
☐ c*(a+ib)
OpenSplice DDS

☐ (a+ib)*c
☐ How can we supporting something like -(a+ib) and c*(a+ib)?

Scala Unary Operators
☐ Scala allows to define unary operators for the following method

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
identifiers +, -, !, ~

def unary_-() = Complex(-re, -im)
OpenSplice DDS

!
def unary_!() = Math.sqrt(re*re + im*im)
!
def unary_~() = Complex(re, -im)
as a result we can write:
val result = -Complex(1,2) + ~Complex(2,3)

Scala Implicit Conversions

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ The expression: val c3 = 3*Complex(5, 7)

☐ Is equivalent to:
OpenSplice DDS

val c3 = 3.*(Complex(5, 7))
☐ Yet, the method to multiply a Integer to a Complex is not present in the
Scala Int class
☐ What can we do to make the trick?

Scala Implicit Conversions

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ Scala does not support Open Classes, thus allowing to add new
methods to existing classes
☐ Yet Scala supports implicit conversions that can be used to achieve the
same result
OpenSplice DDS

☐ Lets see how...

Scala Implicit Conversion
object Complex {

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
implicit def floatToReComplex (f: Float) = new ReComplex(f)

implicit def intToReComplex(i : Int) = new ReComplex(i)
}
OpenSplice DDS

class ReComplex(re: Float) {

def * (that: Complex) = Complex(re*that.re,re*that.im)

}

The Result is...
val c3 = 3*Complex(5, 7)

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
is converted automatically into:

val c3 = ReComplex(3).*(Complex(5, 7))
OpenSplice DDS

Putting it all together
case class Complex(val re: Float, val im: Float) { // Unary Operators

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
def unary_-() = Complex(-re, -im)
// Binary Operators def unary_!() = Math.sqrt(re*re + im*im)
def + (c: Complex) = Complex(re + c.re, im + c.im) def unary_~() = Complex(re, -im)
def - (c: Complex) = Complex(re - c.re, im - c.im)
// Formatting
def * (f: Float) = Complex(f*re, f*im) override def toString() : String = {
OpenSplice DDS

def * (c: Complex) = Complex((re*c.re) - (im*c.im), if (im > 0) re + "+i" + im
((re*c.im) + (im*c.re))) else if (im < 0) re + "-i" + (-im)
else re.toString
def / (c: Complex) = {

val d = c.re*c.re + c.im*c.im
}
Complex(((re*c.re) + (im + c.im))/d,
}
((im*c.re) - (re*c.im))/d )
}

Functions, Closures and Currying
OpenSplice DDS

Functions

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ Scala has first-class functions
☐ Functions can be defined and called, but equally functions can
be defined as unnamed literals and passed as values
OpenSplice DDS

def inc(x: Int) = x + 1 val vinc = (x: Int) => x+1
inc(5) vinc(5)

Notice once again the uniform access principle

Playing with Functions
val list = List(1,2,3,4,5,6,7,8,9)

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
val g5 = list.filter((x: Int) => x > 5)
g5: List[Int] = List(6, 7, 8, 9)
OpenSplice DDS

Or with placeholder syntax

val list = List(1,2,3,4,5,6,7,8,9)
val g5 = list.filter(_ > 5)
g5: List[Int] = List(6, 7, 8, 9)

Closures

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ Scala allows you to define functions which include free variables
meaning variables whose value is not bound to the parameter list
☐ Free variable are resolved at runtime considering the closure of visible
variable
OpenSplice DDS

☐ Example:

def mean(e : Array[Float]) : Float = { def mean(e : Array[Float]) : Float = {
var sum = 0.0F var sum = 0.0F
e.foreach((x: Int) => sum += x) e.foreach(sum += _)
return sum/e.length return sum/e.length
} }

Currying
☐ Scala provides support for curried functions which are applied to

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
multiple argument lists, instead of just one
☐ Currying is the mechanism Scala provides for introducing new control
abstraction
OpenSplice DDS

def curriedSum(x: Int)(y: Int) = x + y

curriedSum(1) {
3 +5
}

OpenSplice DDS

Traits

Traits
☐ Scala supports single inheritance from classes but can mix-in multiple

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
traits
☐ A trait is the unit of code reuse for Scala. It encapsulate methods and
field definitions
Traits usually expect a class to implement an abstract method, which
OpenSplice DDS

☐
constitutes the “narrow” interface that allows to implement a rich
behaviour
☐ Traits are also very useful for dependency injection

Ordered Complex Numbers

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ Our complex numbers are not comparable
☐ Let’s assume that we wanted to make them comparable, and lets
supposed that we define the total order as based on the module of the
complex number
OpenSplice DDS

☐ How can we implement this behavior?

Ordered Trait

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ The Ordered[T] traits encapsulates the set of methods that allow to
define a total ordering over a type
☐ All the behaviour is defined in terms of an abstract method, namely
“compare”
OpenSplice DDS

☐ Classes that mix-in this trait have to implement the “compare” method

class Complex(val re: Float, val im: Float) extends
Ordering[Complex] {
def compare(x: Complex, y: Complex) = {
if (x == y) 0
else if (!x > !y) 1
else -1
}

Case Classes
&
OpenSplice DDS

Pattern Matching

Case Classes and Pattern Matching

☐ Case Classes and Pattern Matching are twin constructs that are pretty

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
useful when dealing with tree-like recursive data structures
☐ These constructs allow to match patterns in an expression and
reconstruct the object graph that makes it up
OpenSplice DDS

☐ Lets see an example...

Case Classes and Pattern
abstract class Expr
case class Var(name: String) extends Expr

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
case class Number(num: Float) extends Expr
case class UnOp(operator: String, arg: Expr) extends Expr
case class BinOp(operator: String, left: Expr, right: Expr)
OpenSplice DDS

def simplifyExpr(expr: Expr) : Expr = expr match {
case UnOp("-", UnOp("-", e)) => e
case BinOp("+", e, Number("0")) => e
case BinOp("*", e, Number("1")) => e
case _ => expr
}

Type Parametrization
OpenSplice DDS

Type Parametrization

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ Scala provides support for type parametrization and makes it
available for both classes as well as traits
trait Queue[T] {
def head: T
OpenSplice DDS

def tail: Queue[T]
def append(x: T) : Queue[T]
}
☐ Scala allows to annotate the parametrized type to control the
resulting type variance

Type Variance
☐ If S <: T is Queue[S] <: Queue[T]?

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ By default Scala makes generic types nonvariant. This behaviour can be
changed using the following annotations:
☐ Queue[+T] indicates that the the sub-typing is covariant in the
parameter T
OpenSplice DDS

☐ Queue[-T] indicates that the the sub-typing is contravariant in the
parameter T

[Scala API]

Domain
Domain

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
val dp = DomainParticipant(domainId)
Participant

Publisher
Topic
Subscriber
OpenSplice DDS

DataWriter
DataReader

[Scala API]

Domain
Domain

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Participant

Session
// Create a Topic
Publisher
Topic
Subscriber
OpenSplice DDS

val topic = Topic[ShapeType](dp, “Circle”)
// Create a Publisher / Subscriber
val pub = Publisher(dp)
val sub = Subscriber(dp)

DataWriter
DataReader

[Scala API]

Domain
Domain

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Participant

Session
// Create a Topic
Publisher
Topic
Subscriber
OpenSplice DDS


Reader/Writers for User Deﬁned for Types DataWriter
DataReader
// Create a DataWriter/DataWriter
val writer = DataWriter[ShapeType](pub, topic) Reader/Writer for
val reader = DataReader[ShapeType](sub, topic) application deﬁned
Topic Types

[Scala API]

Domain
Domain

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Participant

Session
// Create a Topic
Publisher
Topic
Subscriber
OpenSplice DDS


Reader/Writers for User Deﬁned for Types DataWriter
DataReader
// Write data
val data = new ShapeType(“RED”, 131, 107, 75)
writer write data
Reader/Writer for
// But you can also write like this... application deﬁned
writer ! data Topic Types
// Read new data and print it on the screen
(reader read) foreach (prinln)

Data Reader/Writer Caches
OpenSplice DDS

Reader/Writer Caches
Application Application Application

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
DataWriter DataReader DataReader

...
OpenSplice DDS

DataWriter Cache DataReader Cache DataReader Cache

Network

write( ) Application Application

Copyright
2011,
PrismTech
–

All
Rights
Reserved.

...
OpenSplice DDS


Network


Copyright
2011,
PrismTech
–

All
Rights
Reserved.

...
OpenSplice DDS


Topic Instance
Network


Copyright
2011,
PrismTech
–

All
Rights
Reserved.

...
OpenSplice DDS


Topic Instance Topic Instance

Network


Copyright
2011,
PrismTech
–

All
Rights
Reserved.

...
OpenSplice DDS


Writer History

Network


Copyright
2011,
PrismTech
–

All
Rights
Reserved.

...
OpenSplice DDS


Reader History Reader History

Network


Copyright
2011,
PrismTech
–

All
Rights
Reserved.

...
OpenSplice DDS


QoS Policies
‣ History
‣ Destination Order Network
‣ Presentation
‣ Partition QoS Policies
‣ Time Based Filter ‣ Reliability ‣ Durability
‣ Ownership ‣ History ‣ Transport Priority
‣ Latency Budget ‣ Time Based Filter

Writer Cache Properties
☐ An entry for each topic instance

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ Each cache entry might contain one
or more samples, the number of
DataWriter
samples is controlled by the History
QoS Policy
OpenSplice DDS

...
☐ Depending on the setting of the
Durability QoS Policy, the content of
the cache may be made available DataWriter Cache
to late joiners. The depth of the
cache controls how many samples
per instance will be delivered to late
joiners

Reader Cache Properties
☐ An entry for each topic instance

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ Each cache entry might contain one or
more samples, the number of samples is
controlled by the History QoS Policy DataReader
☐ History.KeepLast(n) keeps the last n samples
OpenSplice DDS

per instance
☐ History.KeepAll maintains all the samples,
...
modulo resource constraints

☐ For a given instance: DataReader Cache
☐ Samples from the same writer are inserted in
write-order
☐ Samples from multiple-writers are ordered
using time-stamps (source or reception
depending on configurable a QoS Policy)

Eventual Consistency
DDS caches provide eventual consistency semantics

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐

☐ This means that a read will see the effect of a preceding write
eventually
OpenSplice DDS

☐ Furthermore, given a data-writer that is currently matching N
readers, we can think of DDS as providing eventual consistency
with W=0 and R=1
☐ W: the number of Acks expected in order to return from a write
☐ R: the number of sources from which a read access data

Understanding History
OpenSplice DDS

Caches or Streams?
OpenSplice DDS

Eventual View of a Stream
Assumptions (Default Settings):
Reader History = KeepLast (1)
WriterHistory = KeepLast (1)

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Writer History
Reader History ‘Past’ Samples
OpenSplice DDS


Assumptions:
Reader History = KeepLast (n) with n > 1
WriterHistory = KeepLast (m) with n > m > 1

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Writer History Reader History
‘Past’ Samples
OpenSplice DDS


Interacting with the
DataReader Cache
OpenSplice DDS

Reading Data Samples
☐ Samples can be read from the Data Reader History Cache

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ The action of reading a sample is non-destructive. Samples are
not removed from the cache
OpenSplice DDS

DataReader DataReader

... read ...

DataReader Cache DataReader Cache

Taking Data Samples
☐ Samples can be taken from the Data Reader History Cache

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ The action of taking a sample is destructive. Samples are
removed from the cache
OpenSplice DDS

DataReader DataReader

... take ...

DataReader Cache DataReader Cache

Read vs. Take

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ The read operation should always be access the latest know value
for topics that represent distributed state
OpenSplice DDS

☐ The take operation should be used to get the last notification from a
topic that represent an event

Cherry Picking in DDS

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
☐ DDS provides some very flexible mechanisms for selecting the data
to be read:
OpenSplice DDS

☐ Data Content
☐ Data Status

☐ These mechanisms are composable

Content-Based Data Selection
OpenSplice DDS

Filters and Queries Application

☐ DDS Filters allow to control what gets
into a DataReader cache

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
Query
☐ DDS Queries allow to control what gets
out of a DataReader cache DataReader
OpenSplice DDS

☐ Filters are defined by means of
...
ContentFilteredTopics

...
...

...
☐ Queries operate in conjunction with DataReader Cache
read operations
Filter
☐ Filters and Queries are expressed as SQL
where clauses

struct ShapeType {

Filters
@Key
string color;
long x;
long y;
[Scala API] };
long shapesize;

/**

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
* NOTE: The Scala API if not provided with DP/Sub/Pub assumes
* default domains and default partition.
**/
// Create a Topic
val topic = Topic[ShapeType](“Circle”)

// Define filter expression and parameters
OpenSplice DDS

val query = Query(“x < %0 AND y < %1”, List(“200”, “300”))

// Define content filtered topic
val cftopic =
ContentFilteredTopic[ShapeType](“Circle”, topic, query)

// Create a DataReader for the content-filtered Topic
val reader =
DataReader[ShapeType](cftopic)

struct ShapeType {

Query
@Key
string color;
long x;
long y;
[Scala API] };
long shapesize;

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
// Define filter expression and parameters
OpenSplice DDS

val query = Query(“x < %0 AND y < %1”, List(“200”, “300”))

dr read (contentQuery = query)

Instances
☐ DDS provides a very efficient way of reading data belonging to a

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
specific Topic Instance

☐ Obviously, one could use queries to match the key’s value, but this
is not as efficient as the special purpose instance selector
OpenSplice DDS

val data = reader read(instanceHandle = handle)

State-Based Selection
OpenSplice DDS

Sample, Instance, and View State
☐ The samples included in the DataReader cache have associated

Copyright
2011,
PrismTech
–

All
Rights
Reserved.
some meta-information which, among other things, describes the
status of the sample and its associated stream/instance
☐ The Sample State (READ, NOT_READ) allows to distinguish between
OpenSplice DDS

new samples and samples that have already been read
☐ The View State (NEW, NOT_NEW) allows to distinguish a new
instance from an existing one
☐ The Intance State (ALIVE, NOT_ALIVE_DISPOSED,
NOT_ALIVE_NO_WRITERS) allows to track the life-cycle transitions of
the instance to which a sample belongs

High Performance Distributed Computing with DDS and Scala

High Performance Distributed Computing with DDS and Scala

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to High Performance Distributed Computing with DDS and Scala

Similar to High Performance Distributed Computing with DDS and Scala (20)

More from Angelo Corsaro

More from Angelo Corsaro (20)

Recently uploaded

Recently uploaded (20)

High Performance Distributed Computing with DDS and Scala