Presentation of the joint revised submission (RTI, TwinOaks, eProsima) to the DDS-XRCE (DDS for Resource Constrained Environments) RFP.

1. DDS-XRCE
DDS for Extremely Resource Contained Networks
Dra< specifica?on presented at OMG New Orleans, September2017
mars/2017-09-22
Joint Submission: RTI, TwinOaks, eProsima
Contact: Gerardo Pardo-Castellote, Ph.D.

4. Outline
• Background
• High Level Overview
• Key Concepts and Features
• An example
• Status
• Demo
©2017 Real-Time Innova?ons, Inc.

5. Backgrond
• Two ini?al submissions in Chicago, 9/2016
– Joint Submission RTI, TwinOaks, eProsima
– Submission by PrismTech
• Presenta?ons in Reston focused on differences
– Goal was explore joining submissions
• Joining Submission stalled
– Technical details and documents a moving target
– Some ques?ons submi^ed and answered
– Hard to evaluate proposals without wri^en submissions that had not been
updated since the ini?al submission 9/2016
©2017 Real-Time Innova?ons, Inc.

6. Real-Time Innova?ons, Inc.
High Level Overview

7. Common approach: Agent-Based deployment model
XRCE
Agent
Global Data Space DDS/ RTPS
Client
Client
Client
DDS-XRCE
XRCE
Agent
Client
Client
Client
DDS App
DDS App
DDS App
DDS App
DDS App

8. Agent-Based deployment model
XRCE
Agent Client
XRCE Protocol
Global Data Space
DDS/ RTPS
Exis?ng DDS
Interoperability
Wire Protocol
New protocol defined
by the DDS-XRCE
specifica?on
Resource Constrained
Network
~ 40Kbps, 100Byte MTU
Resource Constrained
Device: ~ 100 KB RAM
Ba^ery Powered
Sleep Cycles

9. Formal specifica?on of the Agent and Protocol
XRCE
Agent Client
XRCE Protocol
Global Data Space
DDS/ RTPS
XRCE
Protocol
DDS/RTPS
Protocol

10. Formal specifica?on of the Agent and Protocol
Client
XRCE Protocol
Global Data Space
DDS/ RTPS
HEADER MSG1 MSG2
XRCE Object Model
Message Structure
+
Interac=on Model
+

11. Real-Time Innova?ons, Inc.
Demo
h^ps://youtu.be/HJ5eBQ2tZNQ

12. Real-Time Innova?ons, Inc.
XRCE Object Model

13. XRCE Agent
Object Model
class DDS-Mapping
DDSXRCE::
DomainParticipant
DDSXRCE::Publisher
DDSXRCE::Subscriber
DDSXRCE::
DataWriter
DDSXRCE::DataReader
DDSXRCE::Topic
DDS::DomainParticipantFactory
DDS::DomainParticipant
DDS::Publisher
DDS::Subscriber
DDS::TopicDescription
DDS::Topic
DDS::ContentFilteredTopic
DDS::DataReader
DDS::DataWriter
«value»
DDSXRCE::Qos
«value»
DDSXRCE::
QosProfile
Qos
+ DataReaderQos
+ DataWriterQos
+ DomainParticipantQos
+ PublisherQos
+ SubscriberQos
+ TopicQos
(from DDS)
DDSXRCE::Application
«use»
«use»
«use»
«use»
«use»
«use»
0..*
«use»
«use»
«use»
«use»
«use»

14. XRCE Object Model highlights
• Uses UML to model Agent behavior
• XRCE Objects modeled as “resources”
– Addressable by their “name”
– CRUD opera?ons:
• Create/Read/Update(Write)/Delete
©2017 Real-Time Innova?ons, Inc.

15. ©2017 Real-Time Innova?ons, Inc.
class Application
DDSXRCE::Application
+ create_participant(): ReturnStatus
+ update_participant(): ReturnStatus
+ delete_participant(): ReturnStatus
+ create_waitset(): ReturnStatus
+ update_waitset(): ReturnStatus
+ delete_waitset(): ReturnStatus
DDSXRCE::DomainParticipant
DDS::DomainParticipant
DDS::
DomainParticipantFactory
DDSXRCE::AccessController
+ check_permissions(): ReturnStatus
«value»
DDSXRCE::Type
«value»
DDSXRCE::
QosProfile
DDSXRCE::QosLibrary «use»
«use»
«use»
«use»
«use»
0..*
«use»

16. ©2017 Real-Time Innova?ons, Inc.
class Participant
DDSXRCE::Application
DDSXRCE::DomainParticipant
+ register_type(): ReturnStatus
+ unregister_type(): ReturnStatus
+ create_topic(): ReturnStatus
+ update_topic(): ReturnStatus
+ delete_topic(): ReturnStatus
+ create_publisher(): ReturnStatus
+ update_publisher(): ReturnStatus
+ delete_publisher(): ReturnStatus
+ create_subscriber(): ReturnStatus
+ update_subscriber(): ReturnStatus
+ delete_subscriber(): ReturnStatus
+ get_topics(): ReturnStatus
+ get_publishers(): ReturnStatus
+ get_subscribers(): ReturnStatus
DDS::DomainParticipant
DDS::
DomainParticipantFactory
DDSXRCE::AccessController
+ check_permissions(): ReturnStatus
DDSXRCE::Topic DDSXRCE::
Publisher
DDSXRCE::
Subscriber
«value»
DDSXRCE::Type
DDSXRCE::
RegisteredType
«interface»
DDSXRCE::Entity
+ name: string
0..*
«use»
«use»
«use»
«use»
registeredTypeName
«use»
typeName
«use»
«use»

17. ©2017 Real-Time Innova?ons, Inc.
class DataWriter
DDSXRCE::DataWriter
+ create_instance(): ReturnStatus
+ update_instance(): ReturnStatus
+ delete_instance(): ReturnStatus
+ write(): ReturnStatus
DDS::DataWriter
DDSXRCE::
Publisher
DDSXRCE::Topic
DDS::Publisher
TopicDescription
DDS::Topic
«interface»
DDSXRCE::Entity
+ name: string
«use»
«use»
«use»

18. ©2017 Real-Time Innova?ons, Inc.
class DataReader
DDSXRCE::
Subscriber
DDSXRCE::DataReader
+ get(): ReturnStatus
DDSXRCE::Topic
- registeredTypeName: string
- topicName: string
DDS::
TopicDescription
DDS::Topic
DDS::DataReader
DDS::Subscriber
«interface»
DDSXRCE::Entity
+ name: string
«use»
«use»
«use»
«use»

19. ©2017 Real-Time Innova?ons, Inc.
class AccessController
DDSXRCE::AccessController
+ check_permissions(): ReturnStatus
«singleton»
DDSXRCE::Root
DDSXRCE::Application
DDSXRCE::
Publisher
DDSXRCE::
Subscriber
DDSXRCE::
DomainParticipant
DDSXRCE::ProxyClient
- clientAPIKey: string
0..*
«use»
0..*
0..*
1
«use»
«use»
«use»

20. Resource Names, Representa?on, and ID
• Each resource has:
– A name (addresses it within the Agent) and Context
• Example the type “ShapeType”
– A representa?on. Describes the resource
• Declara?ve way to describe the Agent resource
• Not expected to go on the wire in most cases
– An ID. Provides (integer) shortcurt for Client to reference XRCE object/
resource
• Mul?ple representa?ons supported: NAME, XML, BINARY
©2017 Real-Time Innova?ons, Inc.

21. Qos Representa?on
• Typical: pre-defined profiles in Agent
– Name-Representa?on -> QosPofile name
• Allow exchanging Qos between client and Agent:
– XML Representa?on. Uses XML (or future JSON format)
• Can configure complete set of DDS Qos policies
• Re-uses DDS-XTYPES
– Binary Representa?on
• Allow sending a subset of the Qos using a BINARY representa?on
• Reuses XCDR
• Example:
<qos_profile name="MyProf" base_name="ExistingProfile " >
<datawriter_qos>
<history>
<depth>10</depth>
</history>
</datawriter_qos>
</qos_profile>
©2017 Real-Time Innova?ons, Inc.

22. Type Representa?on
• Typical: types pre-defined profiles in Agent
• Name-Representa?on -> QosPofile name
• Allow exchanging Types between client and Agent:
– XML Representa?on. Uses XML (or future JSON format)
• Can configure complete set of DDS Types policies
• Re-uses DDS-XTYPES
– Binary Representa?on (possible extension)
• Allow exchange a any type informa?on using a BINARY representa?on
• Reuses XCDR
• Example type descrip?on using XML representa?on:
<struct name="ShapeStruct">
<member name="color" type="string"/>
<member name="x" type="int32"/>
<member name="y" type="int32"/>
<member name="shapesize" type="int32"/>
</struct>
©2017 Real-Time Innova?ons, Inc.

23. Real-Time Innova?ons, Inc.
Message Structure

24. Message Header + Submessages
©2017 Real-Time Innova?ons, Inc.
XRCE Header Submessage Submessage Submessage
0 8 16 24 31
+---------------+---------------+---------------+---------------+
| sessionId | streamId | sequenceNr |
+---------------+-------+-------+---------------+---------------+
| clientKey (optional is sessionId > 128) |
+---------------+---------------+---------------+---------------+
0 4 8 16 24 31
+---------------+---------------+---------------+---------------+
| submessageId | flags | submessageLength |
+---------------+---------------+---------------+---------------+
~ <payload – depends on smId & resourceType> ~
+---------------+---------------+---------------+---------------+
XRCE Header
Submessage

25. 13 kind of Sub Messages
©2017 Real-Time Innova?ons, Inc.
class Submessages
DDSXRCE::Submessage
DDSXRCE::
Submessage::
CREATE
DDSXRCE::
Submessage::
GET_INFO
DDSXRCE::
Submessage::
DELETE
DDSXRCE::
Submessage::INFO
DDSXRCE::
Submessage::
STATUS
DDSXRCE::
Submessage::
ACKNACK
DDSXRCE::
Submessage::
HEARTBEAT
DDSXRCE::Submessage::
SubmessageHeader
- submessageId: byte
- flags: byte
- submessageLength: short
DDSXRCE::
Submessage::
DATA
DDSXRCE::
Submessage::
FRAGMENT
DDSXRCE::
Submessage::
READ_DATA
DDSXRCE::
Submessage::
WRITE_DATA
DDSXRCE::
Submessage::
RESET
DDSXRCE::
Submessage::
CREATE_CLIENT
1

26. Submessage kinds
©2017 Real-Time Innova?ons, Inc.
SubmessageId Value Note
CREATE 1 Client to Agent. Specifies resource Name or Representation
GET_INFO 2 Client to Agent.
DELETE 3 Client to Agent.
STATUS 4 Response to CREATE, UPDATE or DELETE.
INFO 5 Response to a GET_INFO. Contains information about an Xrce: Object.
WRITE_DATA 6 Client to Agent. Used to write data using a Xrce::DataWriter.
READ_DATA 7 Client to Agent. Used to read data using a Xrce::DataReader.
DATA 8 Agent to Client in response to a READ_DATA
ACKNACK 9 Bi-directional.
HEARTBEAT 10 Bi-directional.
RESET 11 Bi-directional
FRAGMENT 12 Bi-directional

27. Example: DATA Message
©2017 Real-Time Innova?ons, Inc.
0 8 16 24 31
+---------------+--------+-------+----------------+----------------+
| session_id | stream_id | sequenceNr | 4
+-------+-------+----------------+----------------+----------------+
| DATA | flags | submessageLength | 8
+---------------+----------------+----------------+----------------+
| request_id | object_id | 12
+---------------+----------------+----------------+----------------+
+ XCDR_Serialization(foo) + 16
+---------------+--------+-------+----------------+----------------+
Payload format
Depends on flags
Serializa?on uses DDS-
XTYPES format

28. Example: Crea?ng a Client
©2017 Real-Time Innova?ons, Inc.
0 8 16 24 31
+---------------+---------------+---------------+---------------+
| 0x80 | 0x00 | 0x00 | 4
+-------+-------+---------------+---------------+---------------+
| CREATE_CLIENT | flags | submessageLength | 8
+-------+-------+---------------+---------------+---------------+
| request_id | object_id | 12
+-------+-------+---------------+---------------+---------------+
| xrce_cookie | 16
+-------+-------+-------+-------+-------+-------+-------+-------+
| xrce_version | xrce_vendor_id | 20
+-------+-------+---------------+---------------+---------------+
| client_timestamp.seconds | 24
+ +
| client_timestamp.nanoseconds | 26
+-------+-------+---------------+---------------+---------------+
| client_key | 30
+-------+-------+---------------+---------------+---------------+
| session_id |
+-------+-------+
Built-in security
mechanism using
ClientKey

29. Example: DATA as a sequence
©2017 Real-Time Innova?ons, Inc.
0 8 16 24 31
+---------------+----------------+----------------+----------------+
| session_id | stream_id | sequenceNr | 4
+-------+-------+----------------+----------------+----------------+
| READ_DATA | flags | submessageLength | 8
+---------------+----------------+----------------+----------------+
| request_id | object_id | 12
+---------------+----------------+----------------+----------------+
| is_present=0 | FORMAT_DATA_SEQ| max_samples | 16
+---------------+----------------+----------------+----------------+
| max_elapsed_time | 20
+---------------+----------------+----------------+----------------+
| max_rate | 24
+---------------+----------------+----------------+----------------+

30. Writer Reliability
©2017 Real-Time Innova?ons, Inc.
Initial
AllMessagesAcknowledged MessagesNotFullyAcked
write
/HighestSentSequenceNumber++
receive(ACKNACK)
/save
HigestAcknowledgedSeqNumber
repair SequenceNumber in NACK
[HeartBeatPeriod]
/Send(HeartBeat, HighestSentSequenceNumber)
write
/HighestSentSequenceNumber++
[HiguestReceivedSeqNumber == HiguestSentSequenceNumber]

31. Reader Reliability
©2017 Real-Time Innova?ons, Inc.
Initial
ReceivedAllMessages
MissingMessageFromSender
Choice
/ HiguestReceivedSequenceNumber := 0
[HiguestAnnouncedSequenceNumber <= HiguestReceivedSequenceNumber]
receive(HeartBeat, HigestAnnouncedSequenceNumber)
[HiguestAnnouncedSequenceNumber > HiguestReceivedSequenceNumber]
receive(HeartBeat, HigestAnnouncedSequenceNumber)
[XRCEClient && NackPeriod]
/Send(ACKNACK)

32. Real-Time Innova?ons, Inc.
Interac?on Model

33. ©2017 Real-Time Innova?ons, Inc.
sd Client Agent
XRCE AgentXRCE Client
RESOURCE_STATUS(DataWriter)
RESOURCE_STATUS(Session)
ASSERT_RESOURCE(Session)
WRITE_DATA(DataWriter)
WRITE_DATA(DataWriter)
ASSERT_RESOURCE(DataWriter)
DELETE_RESOURCE(DataWriter, Session)
WRITE_DATA(DataWriter)
ASSERT_RESOURCE iden?fies (preconfigured)
resource by means its Name and retrieves
resourceId
Note: Using CREATE instead of
ASSERT_RESOURCE can create the resource
from its representa?on

34. ©2017 Real-Time Innova?ons, Inc.
sd Minimal Publisher
XRCE Client XRCE Agent
DELETE_RESOURCE()
WRITE_DATA()
ASSERT_RESOURCE(Session)
WRITE_DATA()
RESOURCE_STATUS()
WRITE_DATA()
The resourceId for specific objects (e.g. a
DataWriter) can also be pre-configured.
This saves having to ASSERT it.

35. ©2017 Real-Time Innova?ons, Inc.
sd Minimal Subscriber
XRCE Client XRCE Agent
DATA()
DATA()
ASSERT_RESOURCE(Session)
DELETE_RESOURCE()
DATA()
CONFIG_READ()

36. Real-Time Innova?ons, Inc.
Transport Model

37. Transport requirements
• Packet-oriented or connec?on oriented
• It does not need to provide reliability
– Messages may be dropped
• It does not need to provide ordering
– Messages may arrive out of order
• It does not need to provide no?fica?on of dropped messages
• Must support transport-level security (e.g. TLS or DTLS)
©2017 Real-Time Innova?ons, Inc.

38. Reliability
• Reliability per stream, mul?ple (256) streams
– Buil?n streams 0-5 are present on every session
– Even-numbered streamIds are best-efforts.
– Reliable streams use HB/ ACKNACK protocol
• Similar to DDS/RTPS
• Mul?ple Writers can share same stream as configured by client
• Mul?ple Readers can also share the same stream
©2017 Real-Time Innova?ons, Inc.

39. Real-Time Innova?ons, Inc.
Security

40. Security approach
• Assumes transport-level security is in place
• Message Header includes (4-Byte) secret ClientKey
• Agent uses ClientKey to determine client permissions and
Iden?ty
– Leverages DDS-Security iden?ty and permissions documents
– Proxy DDS DomainPar?cipant are exclusive to a single Client
©2017 Real-Time Innova?ons, Inc.

41. Real-Time Innova?ons, Inc.
Other

42. Other points
• Reuses CDR (DDS-XTYPES) serializa?on
• Reuses DDS-XML representa?ons
– Can create complex system with O(1) overhead.
– Single named resource can represent the whole system
• Serialized Resource format described in IDL
• Supports fragmenta?on
• Supports mul?ple concurrent sessions
• Full “DDS reachability”
– Any Qos available in DDS
– Any data type available in DDS-XTYPES
©2017 Real-Time Innova?ons, Inc.

43. Real-Time Innova?ons, Inc.
Mee?ng the RFP
requirements

44. RFP mandatory requirements
©2017 Real-Time Innova?ons, Inc.
RFP requirement Submission
DDS-XRCE protocol between a Client and an Agent YES
DDS-XRCE clients to be perceived by DDS Par?cipants as at least
minimum profile DDS Par?cipants
YES, the can full DDS
Par?cipants, not limited to
minimum profile
DDS-XRCE protocol overhead, when sending or receiving user data, shall
not exceed 24 bytes per packet
YES, less than 20 bytes
Address clients opera?ng with sleep/wakeup periods YES, state maintained by
Agent, “stateless” messages
DDS-XRCE protocol for the UDP/IP transport.
DDS-XRCE protocol for the TCP/IP transport
YES, protocol is independent
from transport.

45. ©2017 Real-Time Innova?ons, Inc.
RFP mandatory requirement Submission
6.5.1 DDS-XRCE protocol between a Client and an
Agent
YES. Defined in sec?on 9
6.5.2 DDS-XRCE clients to be perceived by DDS
Par?cipants as at least minimum profile DDS
Par?cipants
YES, the can full DDS Par?cipants, not limited
to minimum profile
See Sec?on 9 and 8.
6.5.3 DDS-XRCE protocol overhead, when sending
or receiving user data, shall not exceed 24
bytes per packet
YES, typically 12 Bytes.
See sec?ons 9.4.2, 9.4.3, and 9.4.4
6.5.4 Address clients opera?ng with sleep/wakeup
periods
YES, state maintained by Agent, “stateless”
messages
See sec?ons 8.2 and 9.4.2
6.5.5
6.5.6
DDS-XRCE protocol for the UDP/IP transport.
DDS-XRCE protocol for the TCP/IP transport
YES, protocol is independent from transport.

46. ©2017 Real-Time Innova?ons, Inc.
RFP non-mandatory requirement Submission
6.6.1 Submi^ers may consider defining support
for transparent user payload compression
YES.
This can be done by the applica?on by
compressing the data before sending to the DDS-
XRCE and a<er receiving it from the DDS-XRCE.
6.6.2 Submi^ers may consider defining a custom
mapping to IEEE 802.15.4 that does not
rely on IP.
YES. This can be done by the applica?on by
compressing the data before sending to the DDS-
XRCE and a<er receiving it from the DDS-XRCE.
6.6.3 Submissions may define the use of the
TCP/IP transport mapping over HTML5
WebSockets.
YES. This can be done by the applica?on by
compressing the data before sending to the DDS-
XRCE and a<er receiving it from the DDS-XRCE.

47. Real-Time Innova?ons, Inc.
Issues to be discussed

48. Message Overhead
• Typically 12 Bytes. Caused by
• 1 Byte sessionId
• 1 Byte streamId
• 2 Byte sequence number
• 1 Byte submessageId
• 1 Byte flags and
• 2 Byte submessage Length
• 2 Byte request_id
• 2 Byte object_id
• We looked at whether it could be further reduced. But doing so would increase complexity and reduce
robustness.
• E.g. e do not need different code paths for mul=ple sessions, reconnec=ons, handling variable-it
encoding
• Considering TCP/IP overhead is 40 bytes, 12 bytes versus 4 Bytes represents only a 6% difference. Not
worth the complexity

49. Example: DATA Message
©2017 Real-Time Innova?ons, Inc.
0 8 16 24 31
+---------------+--------+-------+----------------+----------------+
| session_id | stream_id | sequenceNr | 4
+-------+-------+----------------+----------------+----------------+
| DATA | flags | submessageLength | 8
+---------------+----------------+----------------+----------------+
| request_id | object_id | 12
+---------------+----------------+----------------+----------------+
+ XCDR_Serialization(foo) + 16
+---------------+--------+-------+----------------+----------------+
Payload format
Depends on flags
Serializa?on uses DDS-
XTYPES format

50. Real-Time Innova?ons, Inc.
AB Feedback

51. Received 3 detailed / thorough reviews
• Really looked deep into the spec. Down to font used,
valida?ng XMI, checking IPR mode, etc.
• Only uncovered 2 “significant issues”
– Lack of IPR Mode
– Lack of sec?on describing how mandatory requirements are
met
• These issues has been addressed in the Errata

52. Real-Time Innova?ons, Inc.
Conclusion