Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
The Data Distribution Service Tutorial
1. Angelo Corsaro, PhD
Chief Technology Officer
angelo.corsaro@prismtech.com
Data Distribution Service
The
Tutorial
2. DDS is a standard technology for
ubiquitous, interoperable, secure,
platform independent, and real-time data
sharing across network connected devices
DDS in131
Characters
6. CopyrightPrismTech,2015
John Deere’s machinery uses DDS for internal communication as well as for vehicle-to-vehicle
coordination
Telemetry Data is constantly sent to a Cloud for preventive maintenance
In this use case VORTEX enables fog computing as well as cloud computing
John Deere Autonomous FarmingSmart farming
12. CopyrightPrismTech,2015
DDS provides a Distributed Data
Space abstraction where
applications can autonomously
and asynchronously read and write
data enjoying spatial and temporal
decoupling
Its built-in dynamic discovery
isolates applications from network
topology and connectivity details
DDS’ Data Space is completely
decentralised
High Level Abstraction
DDS Global Data Space
...
Data
Writer
Data
Writer
Data
Writer
Data
Reader
Data
Reader
Data
Reader
Data
Reader
Data
Writer
TopicA
QoS
TopicB
QoS
TopicC
QoS
TopicD
QoS
13. Conceptual Model
DDS Global Data Space
...
Data
Writer
Data
Writer
Data
Writer
Data
Reader
Data
Reader
Data
Reader
Data
Reader
Data
Writer
TopicA
QoS
TopicB
QoS
TopicC
QoS
TopicD
QoS
14. Conceptual Model Actual Implementation
Data
Writer
Data
Writer
Data
Writer
Data
Reader
Data
Reader
Data
Reader
Data
Writer
TopicA
QoS
TopicB
QoS
TopicC
QoS
TopicD
QoS
TopicD
QoS
TopicD
QoS
TopicA
QoS
DDS Global Data Space
...
Data
Writer
Data
Writer
Data
Writer
Data
Reader
Data
Reader
Data
Reader
Data
Reader
Data
Writer
TopicA
QoS
TopicB
QoS
TopicC
QoS
TopicD
QoS
15. The
communication
between
the
DataWriter
and
matching
DataReaders
can
be
peer-‐to-‐
peer
exploiting
UDP/IP
(Unicast
and
Multicast)or
TCP/IP
Data
Writer
Data
Writer
Data
Writer
Data
Reader
Data
Reader
Data
Reader
Data
Writer
TopicA
QoS
TopicB
QoS
TopicC
QoS
TopicD
QoS
TopicD
QoS
TopicD
QoS
TopicA
QoS
The
communication
between
the
DataWriter
and
matching
DataReaders
can
be
“brokered”
but
still
exploiting
UDP/IP
(Unicast
and
Multicast)or
TCP/IP
19. Copyright2013,PrismTech–AllRightsReserved.
Domain
• DDS data lives within a domain
• A domain is identified with a non
negative integer, such as 1, 3, 31
• The number 0 identifies the default
domain
• A domain represent an impassable
communication plane DDS Domain
20. Copyright2013,PrismTech–AllRightsReserved.
Partitions
• Partitions are the mechanism provided by DDS to
organise information within a domain
• Access to partitions is controlled through QoS Policies
• Partitions are defined as strings:
• “system:telemetry”
• “system:log”
• “data:row-‐2:col-‐3”
• Partitions addressed by name or regular expressions:
• ”system:telemetry”
• “data:row-‐2:col-‐*”
Partitions
22. Copyright2013,PrismTech–AllRightsReserved.
Topic
• A Topic defines a domain-wide information’s class
• A Topic is defined by means of a (name, type, qos)
tuple, where
• name: identifies the topic within the domain
• type: is the programming language type associated
with the topic.Types are extensible and evolvable
• qos: is a collection of policies that express the non-
functional properties of this topic, e.g. reliability,
persistence, etc.
Topic
Type
Name
QoS
23. Copyright2013,PrismTech–AllRightsReserved.
Topic and Instances
• As explained in the previous slide a topic defines a class/type of information
• Topics can be defined as Singleton or can have multiple Instances
• Topic Instances are identified by means of the topic key
• A Topic Key is identified by a tuple of attributes -- like in databases
• Remarks:
• A Singleton topic has a single domain-wide instance
• A “regular” Topic can have as many instances as the number of different key values,
e.g., if the key is an 8-bit character then the topic can have 256 different instances
25. Copyright2013,PrismTech–AllRightsReserved.
Active Floor
• Assume we are building an active floor
• This active floor is made by a matrix of
pressure sensors used to detects position,
and indirectly movement
• This information is then used by the
application that uses the active floor for
positioning or entertainment
Cell:
(i,j)
26. Copyright2013,PrismTech–AllRightsReserved.
Active Floor
• The generic active cell can be modeled with a
topic that has an instance for each value of (i,j).
The topic type can be defined as:
• Each cell is now distinguishable and associated
with a topic instance
Cell:
(i,j)
struct
TCell
{
short
row;
short
column;
float
pressure;
//
in
kPa
};
#pragma
keylist
TCell
row
column
27. Copyright2013,PrismTech–AllRightsReserved.
Active Floor
• How can we know when something is on the
cell?
• The detection can be based on the difference
between the atmospheric pressure, say P0, and
the pressure sensed by the cell
• We can model this as a Singleton Topic
ReferencePressure defined by the type:
Cell:
(i,j)
struct
TReferencePressure
{
float
pressure;
//
in
kPa
float
precision;
};
#pragma
keylist
TReferencePressure
28. Copyright2013,PrismTech–AllRightsReserved.
Active Floor
• Each sensor has associated a topic
instance identified by the
(row,column) coordinate -- the
instance key
• Each instance produces a stream of
pressure values that in DDS terms
are called samples
0 1 2 3
0
1
2
3
4
Pressure
time
Pressure
time
Pressure
time
struct
TCell
{
short
row;
short
column;
float
pressure;
//
in
kPa
};
#pragma
keylist
Cell
row
column
30. Copyright2013,PrismTech–AllRightsReserved.
Active Floors on a Building
• Let’s assume now that we
have a building that uses
active floors to detect
presence and movement
• How can we organize the
out data model?
0 1 2 3
0
1
2
3
4
Pressure
time
Pressure
time
Pressure
time
31. Copyright2013,PrismTech–AllRightsReserved.
Active Floors on a Building
• The first thing to do is to
introduce the third
dimension to our cell:
struct
TCell
{
short
row;
short
column;
short
floor;
float
pressure;
//
in
kPa
};
#pragma
keylist
TCell
row
column
floor
0 1 2 3
0
1
2
3
4
Pressure
time
Pressure
time
Pressure
time
32. Copyright2013,PrismTech–AllRightsReserved.
Active Floors on a Building
• As we move from a single floor to a
building we need to add some more
structure to our data
• We can now use:
• A Domain for each Building
• A Partition for each Floor
• A Partition for reference value, i.e.
Partition
• A Partition for the configuration
information, e.g. how many floors,
how many rows/cols per floor
0 1 2 3
0
1
2
3
4
Pressure
time
Pressure
time
Pressure
time
33. Copyright2013,PrismTech–AllRightsReserved.
Active Floors on a Building
• Thus the resulting structure is:
• Floor Partitions:
• “building:f-‐1”
• “building:f-‐2”
• ...
• ReferenceValues Partition:
• “building:refvals”
• Configuration Partition:
• “building:config”
0 1 2 3
0
1
2
3
4
Pressure
time
Pressure
time
Pressure
time
35. Copyright2013,PrismTech–AllRightsReserved.
DataWriter
• A DataWriter (DW) is a strongly typed
entity used to produce samples for one or
more instances of a Topic, with a given QoS
• Conceptually, the DataWriter QoS should be
the same as the Topic QoS or more stringent
• However, DDS does enforce a specific
relationship between the Topic and DataWriter
QoS
DW
Type
Topic
QoS
38. Copyright2013,PrismTech–AllRightsReserved.
DataReader
• A DataReader (DR) is a strongly typed entity used
to access and/or consume samples for a Topic,
with a given QoS
• Conceptually, the DataReader QoS should be the same
as the Topic QoS or less stringent
• However, DDS does enforce a specific relationship
between the Topic and DataReader QoS
DR
Type
Topic
QoS
47. Copyright2013,PrismTech–AllRightsReserved.
DataReader
• Samples are stored in the DataReader
Cache
• Samples can be read or taken from the
cache
• Samples taken are evicted from the
cache
• Samples read remain in the cache and
are simply market as read
• The cache content can be selected based
on content or state. More on this later...
0 1 2 3
0
1
2
3
4
Pressure
time
Pressure
time
Pressure
time
50. Copyright2013,PrismTech–AllRightsReserved.
DomainParticipant
• The DomainParticipant is the programming entity that gives access
to a DDS domain
• A DomainParticipant is created as follows:
//
ISO
C++
DDS
API
int
domain_id
=
18;
auto
dp
=
DomainParticipant(domain_id);
//
Java
5
DDS
API
int
domain_id
=
18;
DomainParticipantFactory
dpf
=
DomainParticipantFactory.getInstance(env)
DomainParticipant
dp
=
dpf.createParticipant(domainId);
51. Copyright2013,PrismTech–AllRightsReserved.
Topic
• Given a DomainParticipant we can define (or discover) Topics within
a domain.This can be done as follows:
• As this declaration does not explicitly provide QoS for the Topic, the
default QoS will be used
//
ISO
C++
DDS
API
auto
topic
=
Topic<TCell>(dp,
“Cell”);
//
Java
5
DDS
API
Topic<TCell>
topic
=
dp.createTopic(“Cell”,
TCell.class);
53. Copyright2013,PrismTech–AllRightsReserved.
Publisher/Subscriber
• Definition of a Publisher/Subscriber in the default partition:
• Definition of a Publisher/Subscriber with Partition settings
//
ISO
C++
DDS
API
auto
pub
=
Publisher(dp);
//
ISO
C++
DDS
API
auto
pub_qos
=
dp.default_publisher_qos()
<<
Partition(”af:telemetry”);
auto
pub
=
Publisher(dp,
pub_qos);
//
ISO
C++
DDS
API
auto
sub
=
Subscriber(dp);
//
ISO
C++
DDS
API
auto
sub_qos
=
dp.default_subscriber_qos()
<<
Partition(”af:telemetry”);
auto
sub
=
Subscriber(dp,
sub_qos);
54. Copyright2013,PrismTech–AllRightsReserved.
Publisher/Subscriber
• Definition of a Publisher/Subscriber in the default partition:
• Definition of a Publisher/Subscriber with Partition settings
//
Java
5
DDS
API
Publisher
pub
=
dp.createPublisher();
//
Java
5
DDS
API
PublisherQos
pubQoS
=
dp.getDefaultPublisherQos()
.with(pf.Partition(”af:telemetry”));
Publisher
pub
=
dp.createPublisher(pubQoS);
//
Java
5
DDS
API
Subscriber
sub
=
dp.createSubscriber();
//
Java
5
DDS
API
SubscriberQos
subQoS
=
dp.getDefaultSubscriberQos()
.with(pf.Partition(”af:telemetry”));
Subscriber
sub
=
dp.createSubscriber(subQoS);
55. Copyright2013,PrismTech–AllRightsReserved.
DataWriter
• A DataWriter with default QoS can be declared as follows:
//
ISO
C++
DDS
API
auto
dw
=
DataWriter<TCell>(pub,
topic);
//
Write
the
cell
c(1,1)
using
`writer`
TCell
c11
=
{1,
1,
15};
dw.write(c11);
//
Write
the
cell
c(1,2)
using
the
`operator
<<`
TCell
c12
=
{1,
2,
5};
dw
<<
c12;
//
Java
5
DDS
API
DataWriter<TCell>
dw
=
pub.createDataWriter<TCell>(topic);
TCell
c11
=
new
TCell(1,
2,
15);
dw.write(c11);
56. Copyright2013,PrismTech–AllRightsReserved.
DataReader
• A DataReader with default QoS can be declared as follows:
//
ISO
C++
DDS
API
auto
dr
=
DataReader<TCell>(sub,
topic);
//
Read
Samples
auto
samples
=
dr.read();
//
Do
something
with
it
std::for_each(samples.begin(),
samples.end(),
do_something);
//
Java
5
DDS
API
DataReader<TCell>
dr
=
sub.createDataReader<TCell>(topic);
62. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Filters and Queries
• DDS Filters allow to control what gets
into a DataReader cache
• DDS Queries allow to control what gets
out of a DataReader cache
• Filters are defined by means of
ContentFilteredTopics
• Queries operate in conjunction with
read operations
• Filters and Queries are expressed as
SQL where clauses
DataReader Cache
DataReader
...
...
...
...
Filter
Query
Application
63. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Filters
// == ISO C++ DDS API
// Create a Topic
auto topic = Topic<ShapeType>(dp, “Circle”);
// Define filter expression and parameters
auto filter = Filter(“x < 100 AND y < 200”);
// Define content filtered topic
auto cftopic =
ContentFilteredTopic<ShapeType>(“CFCircle”, topic,
filter)
// Create a DataReader for the content-filtered Topic
auto dr = DataReader<ShapeType>(sub,cftopic)
struct ShapeType {
string color;
long x;
long y;
long shapesize;
};
#pragma keylist ShapeType color
64. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Filters
//
==
Java
5DDS
API
==
final
PolicyFactory
pf
=
runtime.policyFactory();
final
DataReaderQos
drqos
=
sub.getDefaultDataReaderQos()
.withPolicy
(
pf.ContentFilter()
.withFilter(
new
JavaScriptFilter<ShapeType>("data.x
>
data.y"))
);
final
DataReader<ShapeType>
dr
=
sub.createDataReader(shape,
drqos);
struct ShapeType {
string color;
long x;
long y;
long shapesize;
};
#pragma keylist ShapeType color
65. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Query
//
==
ISO
C++
DDS
API
==
//
Define
filter
expression
and
parameters
auto
dr
=
DataReader<ShapeType>(sub,
topic)
val
query
=
Query(dr,
“x
<
100
AND
y
<
200”);
dr.select()
.content(query)
.read();
struct ShapeType {
string color;
long x;
long y;
long shapesize;
};
#pragma keylist ShapeType color
67. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Instances
• DDS provides a very efficient way of reading data belonging to a specific
Topic Instance
• Obviously, one could use queries to match the key’s value, but this is not as efficient
as the special purpose instance selector
//
==
ISO
C++
DDS
API
==
auto
handle
=
dr.lookup_instance(ShapeType(“RED”,
0,
0,
0));
auto
data
=
dr.select()
.instance(handle)
.read();
69. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Sample, Instance, andView State
• The samples included in the DataReader cache have associated some meta-information
which, among other things, describes the status of the sample and its associated
stream/instance
• The Sample State (READ, NOT_READ) allows to distinguish between new samples
and samples that have already been read
• The View State (NEW, NOT_NEW) allows to distinguish a new instance from an
existing one
• The Instance State (ALIVE, NOT_ALIVE_DISPOSED,
NOT_ALIVE_NO_WRITERS) allows to track the life-cycle transitions of the instance
to which a sample belongs
74. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Interaction Models
Polling
•The application proactively polls for data availability as well as special events, such as a
deadline being missed, etc. Notice that all DDS API calls, exclusion made for wait
operations, are non-blocking
Synchronous Notification
•The application synchronously waits for some conditions to be verified, e.g., data
availability, instance lifecycle change, etc.
Asynchronous Notification
•The application registers the interest to be asynchronously notified when specific
condition are satisfied, e.g. data available, a publication matched, etc.
78. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Listener Declaration
//
==
ISO
C++
DDS
API
==
class
ShapeListener
:
public
dds::sub::NoOpDataReaderListener<ShapeType>
{
public:
ShapeListener()
{}
virtual
void
on_data_available(dds::sub::DataReader<ShapeType>&
dr)
{
auto
samples
=
dr.read();
std::for_each(samples.begin(),
samples.end(),
[](const
dds::sub::Sample<ShapeType>&
sample)
{
if
(sample.info().valid())
//
Check
if
sample
contains
valid
data
std::cout
<<
sample.data()
<<
std::endl;
});
}
virtual
void
on_liveliness_changed(dds::sub::DataReader<ShapeType>&
the_reader,
const
dds::core::status::LivelinessChangedStatus&
status)
{
std::cout
<<
">>
Liveliness
Changed!
"
<<
std::endl;
}
};
79. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Listener Registration
//
==
ISO
C++
DDS
API
==
auto
l
=
new
ShapeListener();
//
Create
a
“nothing”
status
mask
StatusMask
mask
=
StatusMask::none();
//
Add
the
statuses
we
are
interested
in.
mask
<<
StatusMask::data_available()
<<
StatusMask::liveliness_changed()
<<
StatusMask::liveliness_lost();
//
Register
the
listener
with
the
associated
mask
dr.listener(l,
mask);
81. CopyrightPrismTech,2015
For data to flow from a DataWriter (DW) to
one or many DataReader (DR) a few
conditions have to apply:
The DR and DW domain participants have
to be in the same domain
The partition expression of the DR’s
Subscriber and the DW’s Publisher
should match (in terms of regular
expression match)
The QoS Policies offered by the DW
should exceed or match those requested
by the DR
Quality of Service
Domain
Participant
DURABILITY
OWENERSHIP
DEADLINE
LATENCY BUDGET
LIVELINESS
RELIABILITY
DEST. ORDER
Publisher
DataWriter
PARTITION
DataReader
Subscriber
Domain
Participant
offered
QoS
Topic
writes reads
Domain Id
joins joins
produces-in consumes-from
RxO QoS Policies
requested
QoS
82. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
QoS DSL
• The ISO C++ and Java 5 APIs provide DSL for dealing with QoS Policies configuration
• The DSL uses language specific idioms, such as fluid interfaces, as well as specific features
of the languages
• Policies as well as Entity QoS are immutable — this allows for better safety and
object sharing
• Policies are treated as algebraic data types and the DSL provide constructors of
each of the cases
• A QoS Provider can now be used to retrieve QoS settings from external sources, e.g. a
file, an HTTP server, DDS durability
85. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Reliability QoS Policy
The Reliability Policy controls the level of guarantee offered by the DDS
in delivering data to subscribers
•Reliable. In steady-state, and with no data writer crashes, guarantees
that all samples in the DataWriter history will eventually be delivered
to all the DataReader
•Best Effort. Indicates that it is acceptable not to retry propagation
of samples
QoS Policy Applicability RxO Modifiable
RELIABILITY T, DR, DW Y N
89. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
History QoS Policy
The DataWriter HISTORY QoS Policy controls
the amount of data that can be made available to
late joining DataReaders under
TRANSIENT_LOCAL Durability
The DataReader HISTORY QoS Policy controls
how many samples will be kept on the reader
cache
•Keep Last. DDS will keep the most recent
“depth” samples of each instance of data identified
by its key
•Keep All. The DDS keep all the samples of each
instance of data identified by its key -- up to
reaching some configurable resource limits
QoS Policy Applicability RxO Modifiable
HISTORY T, DR, DW N N
0 1 2 3
0
1
2
3
4
Pressure
time
Pressure
time
Pressure
time
KeepLast(3)
KeepLast(1)
KeepAll
90. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Durability QoS Policy
The DURABILITY QoS controls the data availability w.r.t. late joiners, specifically
the DDS provides the following variants:
• Volatile. No need to keep data instances for late joining data readers
• Transient Local. Data instance availability for late joining data reader is
tied to the data writer availability
• Transient. Data instance availability outlives the data writer
• Persistent. Data instance availability outlives system restarts
QoS Policy Applicability RxO Modifiable
DURABILITY T, DR, DW Y N
103. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Soft State
• In distributed systems you often need to model soft-state -- a state that
is periodically updated
• Examples are the reading of a sensor (e.g.Temperature Sensor), the
position of a vehicle, etc.
• The QoS combination to model Soft-State is the following:
Reliability
=>
BestEffort
Durability
=>
Volatile
History
=>
KeepLast(n)
[with
n
=
1
in
most
of
the
cases]
Deadline
=>
updatePeriod
LatencyBudget
=>
updatePeriod/3
[rule
of
thumb]
DestinationOrder
=>
SourceTimestamp
[if
multiple
writers
per
instance]
104. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Hard State
• In distributed systems you often need to model hard-state -- a state that
is sporadically updated and that often has temporal persistence
requirements
• Examples are system configuration, a price estimate, etc.
• The QoS combination to model Hard-State is the following:
Reliability
=>
Reliable
Durability
=>
Transient
|
Persistent
History
=>
KeepLast(n)
[with
n
=
1
in
most
of
the
cases]
DestinationOrder
=>
SourceTimestamp
[if
multiple
writers
per
instance]
105. Copyright2013,PrismTech–AllRightsReserved.Copyright2013,PrismTech–AllRightsReserved.
Events
• In distributed systems you often need to model events -- the
occurrence of something noteworthy for our system
• Examples are a collision alert, the temperature beyond a given threshold,
etc.
• The QoS combination to model Events is the following:
Reliability
=>
Reliable
Durability
=>
any
[depends
on
system
requirements]
History
=>
KeepAll
[on
both
DataWriter
and
DataReader!]
DestinationOrder
=>
SourceTimestamp