Extensible and Dynamic Topic Types for DDS

Extensible and Dynamic
Topic Types for DDS
MARS – Jacksonville, FL – March 2010
Presenter: Rick Warren, RTI
Submitting POCs:
 Angelo Corsaro, PrismTech: angelo.corsaro@prismtech.com
 Gerardo Pardo, RTI: gerardo.pardo@rti.com
 Robert Poth, PrismTech: robert.poth@prismtech.com
 Rick Warren, RTI: rick.warren@rti.com
Supporting POCs:
 Virginie Watine, Thales: virginie.watine@thalesgroup.com
document number: 2010-03-02

Structure of this Presentation
Overview of specification and process
Specification contents
 Very detailed discussion last time
 This presentation focuses on what’s changed
AB feedback
Copyright © 2010 RTI - All rights Reserved 2

DDS Types: Assessment and Challenges
Good
Unmatched type safety
=> fewer defects
Unique type awareness
=> faster integration
Unique type awareness
=> higher performance
Bad
No standard API to
define / access types
at runtime
Hard to change types
without rebuilding,
redeploying
Hard to upgrade
systems one
subsystem at a time
Vendor-Specific
Solutions
OMG-Standard
Solutions

Where We Are
RFP issued: Jun. 2008
Initial submission: Nov 2008
Revised submission extended: Feb. 2009
Revised submission extended: Aug. 2009
Revised submission extended: Nov. 2009
Revised submission deadline: Feb. 2010
 Submitters back on the same page
 Responded to comments from 2 (now 3) AB reviewers
 Discussed with Task Force in Long Beach meeting

Design Principles
Don’t break anybody.
 Many people depend on DDS
 Don’t modify existing APIs
 Don’t modify existing compliance points
 Many people depend on RTPS/DDSI 2
 Don’t modify existing protocols or formats
 Don’t modify existing compliance points
Keep it fast.
 New capabilities must permit efficient implementations.
 Don’t use them? Don’t pay for them.
Keep it orthogonal. Type compatibility doesn’t depend on:
 Types’ definition language(s)
 Data representation(s)
 Programming language(s)
 QoS configuration

Overview
Type System
Type
Representation
Language
Binding
Data
Representation

Overview: Example
Type
Representation
Language
Binding
Data
Representation
IDL:
Foo.idl
struct Foo {
string name;
long ssn;
};
IDL to Language Mapping:
Foo.h
Foo.c
FooTypeSupport.c
struct Foo {
char *name;
int ssn;
};
Foo f = {"hello", 2};
IDL to CDR:
00000006
68656C6C
6F000000
00000002

Conformance Changes
Feedback on previous submission: conformance
criteria, relationships to other specs unclear
New submission more precise, better organized:
 Conformance (part II, section 3):
2 conformance levels correspond to DDS, RTPS
 Programming language conformance:
Type system, type representations, language bindings,
DDS Minimum Profile
 Network interoperability conformance:
Type system, data representations, RTPS
 Relations to other specs (part III, section 8)
 Extensions to DDS:
 “Programming language conformance” sections constitute
new optional conformance level (“profile”)
 No changes to existing DDS profiles
 Extensions to IDL:
 Normative to impl.’s of this spec only;
no change to CORBA conformance

Type System
Extensible, Compatible
 6.5.1: (a) Type System as UML meta-model.
(b) Type substitutability rules.
 6.5.2: Type extensibility, mutability rules
 6.5.3: Type compatibility in presence of change
 6.5.17: Allow type to evolve without breaking interoperability
 6.5.5: Support keys. How does extensibility apply?
More Expressive
 6.5.4: Map data type
 6.5.6, 6.5.19: Sparse data: object contains only subset of fields
defined by its type

Type System Overview
class Type System
NamedElement
Type
+ nested: boolean {readOnly}
PrimitiveType
«enumeration»
TypeKind
Collection
Namespace
Aggregation
Namespace
Enumeration
NamedElement
Namespace
Module
Namespace
BitSet
Alias
ConstructedType
«enumeration»
ExtensibilityKind
FINAL_EXTENSIBILITY {readOnly}
EXTENSIBLE_EXTENSIBILITY {readOnly}
MUTABLE_EXTENSIBILITY {readOnly}
+/container
0..*
{frozen}
+/containedType
*
{addOnly}
+extensibility_kind
1
{frozen}
+element_type
1
{frozen}
+kind
1
{frozen}
+base_type
1

Type System Overview
Entirely familiar from IDL:
 Primitives
 Strings, narrow and wide
 Arrays and sequences
 Aliases (typedefs)
 Unions
 Modules
As in IDL, but extended:
 Structures—including single inheritance
(for substitutability: 6.5.1)
 Enumerations—specify bit width and constant values
New relative to IDL:
 Maps—like std::map or java.util.Map;
required by RFP (6.5.4), fundamental OO collection
 Annotations—for extensibility (6.5.2, 6.5.3)
module Foo {
struct Bar {
long baz;
};
typedef Bar Quux;
}

Type System Changes
Clearer, more-flexible type extensibility kinds
1. Previous kind (Boolean): “extensible”=true:
 Add/remove/reorder type members
 Renamed to “mutable”
2. Previous kind (Boolean): “extensible”=false:
 Add to end, but don’t add elsewhere, remove, or reorder
 Renamed to “extensible”
 Now allow removing from end as well as adding to end
3. New kind “final”:
 Don’t allow any modifications
 Explicit annotations identify all levels in IDL
 Un-annotated types presumed “extensible” (2)
 Can be overridden by IDL compiler option
Several concrete examples added

Type Extensibility Example
@Extensibility(
EXTENSIBLE)
struct Coord2D {
long x;
long y;
};
@Extensibility(
EXTENSIBLE)
struct Coord3D {
long x;
long y;
long z;
};
Assignable

Type Extensibility Example
@Extensibility(
FINAL)
struct Coord2D {
long x;
long y;
};
@Extensibility(
FINAL)
struct Coord3D {
long x;
long y;
long z;
};
Not Assignable

Type Representations
 IDL
 XML, XSD
 TypeObject
Static Definition
 6.5.11: Programming-language-
independent type serialization format(s)
 6.5.9: Represent types in IDL and XML
 6.5.7: Support IDL subset used by DDS 1.2
Dynamic Definition
 6.5.12: Type discovery

IDL Type Representation Changes
Added optional pragmas to delimit IDL segments
using new extended syntax
#pragma dds_xtopics begin [<version_number>]
// IDL definitions
#pragma dds_xtopics end
 Document conformance requirements for human readers
 Help non-conforming compilers recognize problems
sooner, give more meaningful errors
 No particular errors or warnings mandated

Use struct syntax for inheritance, not valuetype
struct MyStruct : MyBaseStruct { … };
 Type system doesn’t include all valuetype features, so
syntax was confusing
 Unclear whether valuetype could extend struct

Cleaner user-defined annotation syntax
 Previously used methods, like Java annotations:
@Annotation local interface MyAnnotation {
long my_annotation_member();
};
 Now uses attributes:
@Annotation local interface MyAnnotation {
attribute long my_annotation_member;
};
 Still uses local interface (alternative: struct)
 Emphasizes that no runtime state exists
 Allows future support for multiple inheritance, if desired

XML Type Representations (No Change)
XML Type Representation
 Terse XML syntax designed ground-up for this Type
System
 Easy to read, plays nicely with auto-completing editors
XSD Type Representation
 Based on existing IDL-to-WSDL mapping specification
 Extended for new Type System concepts from this spec

TypeObject Changes
Type Representation suitable for DDS discovery
Renamed from “TypeCode”
 Name similarity to CORBA TypeCode deemed A Bad
Thing: this spec doesn’t depend on that one
 .Net also has “TypeCode,” something else entirely…
Improved string sharing to make representations
much more compact

Data Representations
 Extended CDR
 XML

CDR Data Representation (No Change)
Requirements:
 Continue supporting existing (compact) CDR used by user-defined
types (6.5.16)
 Continue supporting existing (extensible, parameterized) CDR
used by discovery types (6.5.16)
 Provide extensibility to user-defined types (6.5.2, 6.5.3, 6.5.17)—
all Data Representations should have equivalent expressiveness
Solution: Make existing extensible, parameterized CDR
available to all
 Same rules for every type, built-in or user-defined
 100% compatible with existing types and serialization
Parameterized CDR: structs & unions marked “mutable”
 Each member == parameter
 Member ID == parameter ID
Compact CDR: everything else

XML Data Representation (No Change)
Optional Requirement (6.6.1):
Dynamic data access based on XML or JSON
Given: XSD Type Representation
Data object == XML document
Validated by XSD Representation of its type
SAX, DOM-based access follow naturally
New RTPS encapsulation ID lets you use XML
on the network
struct OriginalLandData {
long x;
long y;
};
<OriginalLandData>
<x>5</x>
<y>42</y>;
</OriginalLandData>

Language Bindings
 Plain Language Binding
 Dynamic Language Binding

Plain Language Binding (No Change)
Defined by existing IDL-to-language mappings:
1. Start with IDL Representation of a type
2. Apply mapping to desired programming language
Mappings expanded to cover new concepts
 e.g. maps, annotations, optional members
 For C, C++, Java as required by RFP (6.5.8)
// IDL
struct
OrigLandData {
long x;
long y;
};
// Java
public class
OrigLandData {
public int x;
public int y;
}

Dynamic Language Binding (No Change)
Classes
 DynamicTypeFactory: Creates new types (6.5.20)
 DynamicType: Introspect type definitions (6.5.13)
 Informed by CORBA TypeCode API
 DynamicData: Introspect objects (6.5.18)
 Informed by CORBA DynAny, JMS MapMessage,
TIBCO Rendezvous self-describing messages
 DynamicTypeSupport: Registers types with DDS (6.5.20)
 DynamicDataWriter: Writes objects of any (single) type,
each represented by a DynamicData object (6.5.21)
 DynamicDataReader: Reads objects of any (single) type,
each represented by a DynamicData object (6.5.21)

Use by DDS
 Data Representation Compatibility
 Type Compatibility
 Built-in Types

Data Representation Compatibility
Reader and writer must agree
on which Data Representation to use
 6.5.14: Encapsulation specification, negotiation
 6.5.15: Is encapsulation request-offer?
 6.6.2: Optional: Custom encapsulation API
Largely unchanged from previous submission
Solution: DataRepresentationQosPolicy
 List of representation IDs (standard = XCDR, XML)
 Applies to both readers and writers
 Included in corresponding built-in topic data types
 Request-Offer semantics
 Writer offers single representation it will use
 Reader requests a list of representations
 Compatible iff writer’s representation is in reader’s list
 Can describe proposed representations, future additions, and
vendor extensions

Type Compatibility
If I write type “Foo” and you read type “Bar”, can
we communicate?
New submission clearer, more flexible
based on feedback on previous submission
3 parts:
1. Physical: type assignability
2. Intentional: type signature
3. Enforcement: QoS policy governs how above are applied
(NEW in this submission)

Type Compatibility Example: What I Want
struct Coord2D
{
long x;
long y;
};
struct Coord3D :
Coord2D
{
long z;
};
Read Write
Write Read

1. Type Assignability
Structural compatibility:
Can sample serialized according to one type be
deserialized according to another?
 Reminder: Rules designed to require no writer-specific
runtime checks at deserialization time
Intrinsic property of types’ definitions
 Governed in part by extensibility kind
 No action required by application programmer

2. Type Signature
Do I intend the types to be compatible?
Lists of equivalent types, base types provided at
registration time
Clarification to previous submission: both reader
and writer declarations taken into account
New since previous submission: lists can contain
patterns, not just literal names

3. Type Consistency Enforcement Policy
QoS policy on Topic, DataReader, DataWriter
governs how (1) and (2) take effect
 RxO: reader and writer must have the same setting
 EXACT_TYPE:  Types must also be equal
(as if extensibility were “final”)
 Registered names must be equal
 Like DDS of today but with extra safety
 Assumed during endpoint matching if remote policy
is unspecified
 EXACT_NAME:  Assignability assumed without inspection
 Registered names must be equal
 This is what DDS guarantees today.
 DECLARED:  Types must be assignable
 Type signatures must be compatible
 Default for conformant implementations
 ASSIGNABLE:  Types must be assignable
 Type names, signatures ignored: talk to any
compatible type

Type Compatibility Example: How to Get It
1. Is Coord2D assignable from Coord3D? Yes.
2. Type Signature:
Coord3DTypeSupport.register_type(
my_participant,
"Coord3D : Coord2D")
3. Type Consistency Enforcement: DECLARED
(default)
struct Coord2D
{
long x;
long y;
};
struct Coord3D :
Coord2D
{
long z;
};

Built-in Types (No Change)
Goal: Improve out-of-box experience for new
users, those with simple needs
Background: Usability gap vs. competition
 Other middlewares allow sending textual or binary data
without explicit type definition, registration
Response: Provide simple types built in
 Pre-defined, pre-registered
 String: single string payload
 Bytes: octet sequence payload
 KeyedString: string payload + string key
 KeyedBytes: octet sequence payload + string key

Summary

AB Feedback
Incorporates feedback from 3 AB reviewers:
Define IDL pragma to delimit extensions: Done
Clarify relationships to other specs: Done
Provide more detail about modules and
namespaces: Done
Make annotation type definition syntax more
natural: Done
Standardize VerbatimText behavior for C, C++,
Java languages: Done
Assorted typos, wording changes, organizational
suggestions: Done

Summary
The proposal:
Satisfies all mandatory and optional requirements and
supports identified real-world use cases
 Without sacrificing backwards compatibility
 Without sacrificing performance
Is easily extensible w/ new type, data representations to
handle additional use cases
Only small changes since previous submission
Incorporates all AB feedback
We will be asking for a vote to recommend adoption.

Q & A

Extensible and Dynamic Topic Types for DDS

More Related Content

What's hot

Similar to Extensible and Dynamic Topic Types for DDS

More from Rick Warren

Recently uploaded

Extensible and Dynamic Topic Types for DDS

Editor's Notes