• Like
  • Save
Extensible and Dynamic Topic Types For DDS (out of date)
Upcoming SlideShare
Loading in...5
×
 

Extensible and Dynamic Topic Types For DDS (out of date)

on

  • 1,822 views

Presentation to a Technical Meeting of the Object Management Group (OMG) describing a revised response to an RFP for improvements to the DDS type system in September 2009. ...

Presentation to a Technical Meeting of the Object Management Group (OMG) describing a revised response to an RFP for improvements to the DDS type system in September 2009.

This presentation is superseded by later ones on the same subject.

Statistics

Views

Total Views
1,822
Views on SlideShare
1,819
Embed Views
3

Actions

Likes
0
Downloads
24
Comments
1

1 Embed 3

http://www.slideshare.net 3

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • A note to my readers: this presentation describes the proposed specification as of September 2009. A newer one, circa December 2009, is available here: http://www.slideshare.net/rickbwarren/extensible-and-dynamic-topic-types-for-dds-updated.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • DDS is unique among pub-sub systems in providing interoperable static type safety. Applications that use data already expect certain contents; by making expectations explicit, enable: type-aware integration (e.g. relational mapping) compact network representation.
  • 5 parts to the spec
  • … but what’s the concrete type of the DataReader? That follows from the type of the sample…
  • Two of them!

Extensible and Dynamic Topic Types For DDS (out of date) Extensible and Dynamic Topic Types For DDS (out of date) Presentation Transcript

  • Extensible and Dynamic Topic Types for DDS MARS – San Antonio, TX – Sept., 2009 Rick Warren, RTI [email_address] document number: 2009-09-08
  • Introduction DDS Types: Assessment and Challenges
    • Good
    • Unmatched type safety => fewer defects
    • Unique type awareness => faster integration
    • Unique type awareness => higher performance
    • Bad
    • No standard API to define / access types at runtime
    • Hard to change types without rebuilding, redeploying
    • Hard to upgrade systems one subsystem at a time
    © 2009 RTI - All rights Reserved Vendor-Specific Solutions OMG-Standard Solutions
  • Introduction Scope
    • Improve Portability
      • Clarify applicability of existing IDL type system
      • Codify existing DDS type system features, especially keys
      • Extend IDL to support new concepts
    • Improve Type System Expressiveness
      • Support maps ( i.e. associative collections)
      • Omit irrelevant fields , if desired
      • Make RTPS Discovery encapsulation first class, usable by any type/topic
    © 2009 RTI - All rights Reserved
  • Introduction Scope
    • Enable Agility
      • Support type evolution , subsystem by subsystem
      • Send and receive data using types unknown at compile time
        • Define and modify types programmatically
        • Discover remote type definitions
        • Get and set data member values by name
    • Simplify Integration
      • XML type definitions connect DDS to the XML-speaking world
      • Built-in types help users start using DDS quickly
    © 2009 RTI - All rights Reserved
  • Introduction Overview
    • Type System : abstract definition of what types can exist
      • Expressed as UML meta-model
      • Mostly familiar from IDL
        • IDL-compatible primitives
        • Strings and aliases (typedefs)
        • Arrays, sequences, and maps
        • Structures ( incl. aspects of valuetypes ) and unions
        • Enumerations and bit sets
        • Annotations
    • Type Representations : languages for describing types
      • IDL (with extensions)
      • XML and XSD
      • TypeCode
    © 2009 RTI - All rights Reserved
  • Introduction Overview
    • Data Representations : languages for describing samples
      • CDR (with extensions)
      • XML ( TBD )
    • Language Binding : programming APIs
      • “ Plain language”: extension of existing IDL-to- language bindings
      • Dynamic: reflective API for types and samples, defined in UML (PIM) and IDL (PSM)
    • Use by DDS : defines how DDS uses all of the above
      • Everything else is DDS-independent!
      • (so far, maybe only 90% independent)
    © 2009 RTI - All rights Reserved
  • Introduction Overview © 2009 RTI - All rights Reserved
  • Introduction Overview: Example © 2009 RTI - All rights Reserved Type Representation Language Binding Data Representation IDL: Foo.idl struct Foo { string name; long ssn; };
    • IDL to Language Mapping:
    • Foo.h
    • Foo.c
    • FooTypeSupport.c
      • struct Foo {
      • char *name;
      • int ssn;
      • };
    • Foo f = {"hello", 2};
    IDL to CDR: 00000006 68656C6C 6F000000 00000002 TypeSystem implicitly defined by the Type Representation
  • Introduction Technology Highlights
    • Dynamic Language Binding : dynamic types, dynamic data
    • XML Type Representations : XML and XSD
    • IDL Type Representation : …with extensions
    • CDR Data Representation : extensible binary data
    © 2009 RTI - All rights Reserved
  • Dynamic Types, Dynamic Data Dynamic Types: Overview
    • Summary : Define and/or modify type definitions at runtime
      • Informed by CORBA TypeCode
      • Interoperable with statically defined types
    • Small number of fundamental classes:
      • DynamicType : A programmatic type description
      • DynamicTypeFactory : Creates new types
      • DynamicTypeSupport : Registers types with DDS
    © 2009 RTI - All rights Reserved
  • Dynamic Types, Dynamic Data Dynamic Types: Example #1
    • Create the equivalent:
    • struct MyType {
    • long my_integer;
    • };
    • C++ code:
    • DynamicTypeFactory* factory =
    • DynamicTypeFactory::get_instance() ;
    • DynamicType* type = factory-> create_type (
    • TypeDescriptor (" MyType ", STRUCTURE_KIND ));
    • type-> add_member (
    • MemberDescriptor (
    • " my_integer ",
    • factory-> get_primitive_type (
    • INTEGER_KIND )));
    © 2009 RTI - All rights Reserved
  • Dynamic Types, Dynamic Data Dynamic Types: Example #2
    • Subscribe to a type you’ve never seen before
    • Java code:
    • DynamicType type =
    • myPubBuiltinTopicData.type ;
    • DynamicTypeSupport support =
    • DynamicDataTypeFactory.get_instance().
    • create_type_support ( type );
    • support. register_type (
    • myParticipant ,
    • myPubBuiltinTopicData.type_name );
    • // … init QoS based on built-in topic data
    • Topic topic = myParticipant.create_topic (
    • myPubBuiltinTopicData.name,
    • myPubBuiltinTopicData.type_name,
    • myTopicQos, myTopicListener);
    © 2009 RTI - All rights Reserved
  • Dynamic Types, Dynamic Data Dynamic Data: Overview
    • Summary : Introspect and modify samples at runtime
      • Informed by CORBA DynAny , JMS MapMessage , TIBCO Rendezvous self-describing messages
    • Small number of fundamental classes:
      • DynamicData : A sample of any (single) type
      • DynamicDataFactory : “Instantiates” a dynamic type
      • DynamicDataWriter : Writes samples of any (single) type, each represented by a DynamicData object
      • DynamicDataReader : Reads samples of any (single) type, each represented by a DynamicData object
    © 2009 RTI - All rights Reserved
  • Dynamic Types, Dynamic Data Dynamic Data: Names and IDs
    • DynamicData gets / sets member values in 2 ways:
      • By name : Obvious, human-readable approach
      • By ID number : Potentially faster, more efficient search
    • Example in C++:
    • DynamicData* sample = DynamicDataFactory::get_instance()-> create_data(my_type) ;
    • long my_value = 5;
    • sample-> set_int ( &quot; my_integer &quot;, // Specify name or ID; MEMBER_ID_ANY , // no need for both. my_value ); // <-- Value to set.
    © 2009 RTI - All rights Reserved
  • XML Type Representations XML Type Representations
    • #1: XSD
    • Based on IDL-to-WSDL mapping
    • Allows type sharing between DDS and web services
    • But…
    • Very verbose
    • Hard to read, hard to write, hard to parse
    • #2: XML
    • Straightforward mapping of IDL into XML
    • Suitable for embedding into other XML documents ( e.g. QoS profile files)
    • Easy to read, easy to write, easy to parse
    © 2009 RTI - All rights Reserved
  • XML Type Representations XML Type Representations: Examples
    • #1: XSD
    • <xsd:complexType name=&quot;T&quot;>
    • <xsd:sequence>
    • <xsd:element name=&quot;value&quot; minOccurs=&quot;1&quot; maxOccurs=&quot;1&quot; type=&quot;char&quot;/>
    • </xsd:sequence>
    • </xsd:complexType>
    • #2: XML
    • <struct name=&quot;T&quot;>
    • <member name=&quot;value&quot; type=&quot;char&quot;/>
    • </struct>
    © 2009 RTI - All rights Reserved
  • IDL Type Representation IDL Extension: Annotations
    • Summary : Bringing Java-style annotations to IDL
      • Don’t just solve today’s problem;
      • … also provide an extensible tool we can reuse tomorrow
    • Example :
    • @AnAnnotationAppliedToTheType
    • struct TheType
    • @AnAnnotationAppliedToTheMember
    • long the_member;
    • };
    © 2009 RTI - All rights Reserved
  • IDL Type Representation IDL Extension: Annotations
    • Define annotations:
      • @Annotation local interface MyAnnotation {
      • long value() default 32;
      • };
      • * Why “@Annotation” instead of “@interface”? Coming up…
    • Apply annotations:
      • struct MyStruct {
      • @MyAnnotation long my_field;
      • };
    • Code gen translates to Java annotations one-to-one
    © 2009 RTI - All rights Reserved
  • IDL Type Representation IDL Extension: Compatible Annotation Syntax
    • Assumption : Some vendors will implement this spec sooner than others
    • Assumption : Users share IDL files among products from multiple vendors
    • Problem : How to take advantage of new features w/o breaking old IDL compilers with new syntax?
    • Solution : Put annotations in single-line “comment,” e.g. :
      • You can do this: @Key long my_key_field;
      • … or equivalently: long my_key_field; //@Key
    © 2009 RTI - All rights Reserved
  • IDL Type Representation IDL Extension: Built-in Annotations
    • @Key
      • A member is part of its enclosing type’s key
    • @ID
      • Specify the ID of a particular type member
      • If omitted, ID is assigned automatically
    • @Optional
      • A type member may be omitted from data samples
    • @Extensible
      • A type might change in the future,
      • … so DDS will use a more-verbose, more-flexible encapsulation
    © 2009 RTI - All rights Reserved
  • CDR Data Representation Extensible Binary Data: Challenge
    • Problem : CDR encoded data looks like this:
      • 10110010001011100001110101000111
    • … So how do I know if I’ve added / removed a field?
    • Possible Solution #1 : Include type definition with every sample
      • Uses too much bandwidth
      • Parsing is too slow
    • Possible Solution #2 : Have DataWriter publish its type definition; DataReader uses that to parse samples
      • Parsing is still too slow
    © 2009 RTI - All rights Reserved
  • CDR Data Representation Extensible Binary Data: Solution
    • Solution : Use member IDs w/ RTPS param CDR
      • +----+----+----+----+----+----+----+----+
      • | ID (16) | size (16) |
      • +----+----+----+----+----+----+----+----+
      • | data (size) ... |
      • +----+----+----+----+----+----+----+----+
    • Benefits :
      • Skip fields you don’t understand
      • Easily determine which fields are (not) present
      • Reorder fields
      • Brings RTPS simple discovery into type system mainstream
    • Cost : 4 bytes (+ alignment) per member
    © 2009 RTI - All rights Reserved
  • CDR Data Representation Extensible Binary Data: Details
    • Types marked as “extensible” encapsulated w/ parameterized CDR, all others w/ “plain” CDR
      • Extensible types can contain objects of non-extensible types and vice versa
      • 100% compatible with existing types and serialization (…since built-in topic data are the only “extensible” types that exist today  )
    • Lots of members in your type? Data sizes > 64 KB?
      • +----+----+----+----+----+----+----+----+
      • | ID sentinel (16) | size = 8 (16) |
      • +----+----+----+----+----+----+----+----+
      • | extended ID (32) |
      • +----+----+----+----+----+----+----+----+
      • | extended size (32) |
      • +----+----+----+----+----+----+----+----+
      • | data (extended size) ... |
      • +----+----+----+----+----+----+----+----+
    © 2009 RTI - All rights Reserved
  • CDR Data Representation Extensible Binary Data: Type Versioning
    • Problem : How can types change over time – and at different times from subsystem to subsystem?
    • Possible Solution : Give types version number, keep track of past versions and who is using which version.
      • Too slow : must consult per-remote-writer definition to deserialize
      • Requires tracking lots of state – including every version of every type that’s ever existed ( e.g. for late joiners)
      • Requires network propagation of all type information: increased bandwidth use, possible security objections
    • Solution : Do nothing . Member IDs already provide compact per-member “version.”
      • Adding / changing a member? Define a new ID. Old applications can ignore it.
      • “ Removing” a member? Just stop sending it. Doesn’t invalidate existing persisted data.
    © 2009 RTI - All rights Reserved
  • Summary: Specification Overview © 2009 RTI - All rights Reserved
  • Summary: Goals and Benefits
    • Portable, consistent type system
      • Standardized syntax for keys and other DDS concepts
      • General-purpose extensibility (annotations) reusable by future specifications, vendors, and users
      • Same features usable by user-defined types as built-in types
    • Dynamic access to type definitions
      • Introspect types for remote publications, subscriptions
      • Define and modify types at runtime
      • Publish and subscribe to dynamically defined types
      • Interoperate with statically defined types and across vendors
    • Simplified integration of DDS into other systems
      • XML and XSD type definitions to integrate w/ XML-centric systems
      • Built-in types for simplified programming, especially for new users
    • Fast : little or no performance overhead in steady state
    © 2009 RTI - All rights Reserved
  • Summary: Progress
    • Many more details than previous submission
      • Full specification of IDL extensions
      • Unification of CDR and parameterized CDR data representations
      • Concrete dynamic type, dynamic data APIs
      • Refinements to automatic type conversions
    • Some work remains
      • XML data representation
      • Refinements to XML type representation grammars
      • Refinements to endpoint matching rules
    • Targeted for completion by Dec. 2009 meeting
    © 2009 RTI - All rights Reserved
  • Q & A © 2009 RTI - All rights Reserved