Your SlideShare is downloading. ×
0
Extensible and Dynamic Topic Types for DDS MARS – San Antonio, TX – Sept., 2009 Rick Warren, RTI [email_address] document ...
Introduction DDS Types: Assessment and Challenges <ul><li>Good </li></ul><ul><li>Unmatched type safety => fewer defects </...
Introduction Scope  <ul><li>Improve Portability </li></ul><ul><ul><li>Clarify  applicability of existing IDL type system <...
Introduction Scope <ul><li>Enable Agility </li></ul><ul><ul><li>Support type evolution , subsystem by subsystem </li></ul>...
Introduction Overview <ul><li>Type System : abstract definition of what types can exist </li></ul><ul><ul><li>Expressed as...
Introduction Overview <ul><li>Data Representations : languages for describing samples </li></ul><ul><ul><li>CDR (with exte...
Introduction Overview © 2009 RTI - All rights Reserved
Introduction Overview: Example © 2009 RTI - All rights Reserved Type Representation Language Binding Data Representation I...
Introduction Technology Highlights <ul><li>Dynamic Language Binding : dynamic types, dynamic data </li></ul><ul><li>XML Ty...
Dynamic Types, Dynamic Data Dynamic Types: Overview <ul><li>Summary : Define and/or modify type definitions at runtime </l...
Dynamic Types, Dynamic Data Dynamic Types: Example #1 <ul><li>Create the equivalent: </li></ul><ul><li>struct MyType { </l...
Dynamic Types, Dynamic Data Dynamic Types: Example #2 <ul><li>Subscribe to a type you’ve never seen before </li></ul><ul><...
Dynamic Types, Dynamic Data Dynamic Data: Overview <ul><li>Summary : Introspect and modify samples at runtime </li></ul><u...
Dynamic Types, Dynamic Data Dynamic Data: Names and IDs <ul><li>DynamicData  gets / sets member values in 2 ways: </li></u...
XML Type Representations XML Type Representations <ul><li>#1: XSD </li></ul><ul><li>Based on IDL-to-WSDL mapping </li></ul...
XML Type Representations XML Type Representations: Examples <ul><li>#1: XSD </li></ul><ul><li><xsd:complexType name=&quot;...
IDL Type Representation IDL Extension: Annotations <ul><li>Summary : Bringing Java-style annotations to IDL </li></ul><ul>...
IDL Type Representation IDL Extension: Annotations <ul><li>Define annotations: </li></ul><ul><ul><li>@Annotation local int...
IDL Type Representation IDL Extension: Compatible Annotation Syntax <ul><li>Assumption : Some vendors will implement this ...
IDL Type Representation IDL Extension: Built-in Annotations <ul><li>@Key </li></ul><ul><ul><li>A member is part of its enc...
CDR Data Representation Extensible Binary Data: Challenge <ul><li>Problem : CDR encoded data looks like this: </li></ul><u...
CDR Data Representation Extensible Binary Data: Solution <ul><li>Solution : Use member IDs w/ RTPS param CDR </li></ul><ul...
CDR Data Representation Extensible Binary Data: Details <ul><li>Types marked as “extensible” encapsulated w/ parameterized...
CDR Data Representation Extensible Binary Data: Type Versioning <ul><li>Problem : How can types change over time – and at ...
Summary: Specification Overview © 2009 RTI - All rights Reserved
Summary: Goals and Benefits <ul><li>Portable, consistent type system </li></ul><ul><ul><li>Standardized syntax for keys  a...
Summary: Progress <ul><li>Many more details than previous submission </li></ul><ul><ul><li>Full specification of IDL exten...
Q & A © 2009 RTI - All rights Reserved
Upcoming SlideShare
Loading in...5
×

Extensible and Dynamic Topic Types For DDS (out of date)

1,208

Published on

Presentation to a Technical Meeting of the Object Management Group (OMG) describing a revised response to an RFP for improvements to the DDS type system in September 2009.

This presentation is superseded by later ones on the same subject.

Published in: Technology, Business
1 Comment
0 Likes
Statistics
Notes
  • A note to my readers: this presentation describes the proposed specification as of September 2009. A newer one, circa December 2009, is available here: http://www.slideshare.net/rickbwarren/extensible-and-dynamic-topic-types-for-dds-updated.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total Views
1,208
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
25
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide
  • DDS is unique among pub-sub systems in providing interoperable static type safety. Applications that use data already expect certain contents; by making expectations explicit, enable: type-aware integration (e.g. relational mapping) compact network representation.
  • 5 parts to the spec
  • … but what’s the concrete type of the DataReader? That follows from the type of the sample…
  • Two of them!
  • Transcript of "Extensible and Dynamic Topic Types For DDS (out of date)"

    1. 1. Extensible and Dynamic Topic Types for DDS MARS – San Antonio, TX – Sept., 2009 Rick Warren, RTI [email_address] document number: 2009-09-08
    2. 2. Introduction DDS Types: Assessment and Challenges <ul><li>Good </li></ul><ul><li>Unmatched type safety => fewer defects </li></ul><ul><li>Unique type awareness => faster integration </li></ul><ul><li>Unique type awareness => higher performance </li></ul><ul><li>Bad </li></ul><ul><li>No standard API to define / access types at runtime </li></ul><ul><li>Hard to change types without rebuilding, redeploying </li></ul><ul><li>Hard to upgrade systems one subsystem at a time </li></ul>© 2009 RTI - All rights Reserved Vendor-Specific Solutions OMG-Standard Solutions
    3. 3. Introduction Scope <ul><li>Improve Portability </li></ul><ul><ul><li>Clarify applicability of existing IDL type system </li></ul></ul><ul><ul><li>Codify existing DDS type system features, especially keys </li></ul></ul><ul><ul><li>Extend IDL to support new concepts </li></ul></ul><ul><li>Improve Type System Expressiveness </li></ul><ul><ul><li>Support maps ( i.e. associative collections) </li></ul></ul><ul><ul><li>Omit irrelevant fields , if desired </li></ul></ul><ul><ul><li>Make RTPS Discovery encapsulation first class, usable by any type/topic </li></ul></ul>© 2009 RTI - All rights Reserved
    4. 4. Introduction Scope <ul><li>Enable Agility </li></ul><ul><ul><li>Support type evolution , subsystem by subsystem </li></ul></ul><ul><ul><li>Send and receive data using types unknown at compile time </li></ul></ul><ul><ul><ul><li>Define and modify types programmatically </li></ul></ul></ul><ul><ul><ul><li>Discover remote type definitions </li></ul></ul></ul><ul><ul><ul><li>Get and set data member values by name </li></ul></ul></ul><ul><li>Simplify Integration </li></ul><ul><ul><li>XML type definitions connect DDS to the XML-speaking world </li></ul></ul><ul><ul><li>Built-in types help users start using DDS quickly </li></ul></ul>© 2009 RTI - All rights Reserved
    5. 5. Introduction Overview <ul><li>Type System : abstract definition of what types can exist </li></ul><ul><ul><li>Expressed as UML meta-model </li></ul></ul><ul><ul><li>Mostly familiar from IDL </li></ul></ul><ul><ul><ul><li>IDL-compatible primitives </li></ul></ul></ul><ul><ul><ul><li>Strings and aliases (typedefs) </li></ul></ul></ul><ul><ul><ul><li>Arrays, sequences, and maps </li></ul></ul></ul><ul><ul><ul><li>Structures ( incl. aspects of valuetypes ) and unions </li></ul></ul></ul><ul><ul><ul><li>Enumerations and bit sets </li></ul></ul></ul><ul><ul><ul><li>Annotations </li></ul></ul></ul><ul><li>Type Representations : languages for describing types </li></ul><ul><ul><li>IDL (with extensions) </li></ul></ul><ul><ul><li>XML and XSD </li></ul></ul><ul><ul><li>TypeCode </li></ul></ul>© 2009 RTI - All rights Reserved
    6. 6. Introduction Overview <ul><li>Data Representations : languages for describing samples </li></ul><ul><ul><li>CDR (with extensions) </li></ul></ul><ul><ul><li>XML ( TBD ) </li></ul></ul><ul><li>Language Binding : programming APIs </li></ul><ul><ul><li>“ Plain language”: extension of existing IDL-to- language bindings </li></ul></ul><ul><ul><li>Dynamic: reflective API for types and samples, defined in UML (PIM) and IDL (PSM) </li></ul></ul><ul><li>Use by DDS : defines how DDS uses all of the above </li></ul><ul><ul><li>Everything else is DDS-independent! </li></ul></ul><ul><ul><li>(so far, maybe only 90% independent) </li></ul></ul>© 2009 RTI - All rights Reserved
    7. 7. Introduction Overview © 2009 RTI - All rights Reserved
    8. 8. Introduction Overview: Example © 2009 RTI - All rights Reserved Type Representation Language Binding Data Representation IDL: Foo.idl struct Foo { string name; long ssn; }; <ul><li>IDL to Language Mapping: </li></ul><ul><li>Foo.h </li></ul><ul><li>Foo.c </li></ul><ul><li>FooTypeSupport.c </li></ul><ul><ul><li>struct Foo { </li></ul></ul><ul><ul><li>char *name; </li></ul></ul><ul><ul><li>int ssn; </li></ul></ul><ul><ul><li>}; </li></ul></ul><ul><li>Foo f = {&quot;hello&quot;, 2}; </li></ul>IDL to CDR: 00000006 68656C6C 6F000000 00000002 TypeSystem implicitly defined by the Type Representation
    9. 9. Introduction Technology Highlights <ul><li>Dynamic Language Binding : dynamic types, dynamic data </li></ul><ul><li>XML Type Representations : XML and XSD </li></ul><ul><li>IDL Type Representation : …with extensions </li></ul><ul><li>CDR Data Representation : extensible binary data </li></ul>© 2009 RTI - All rights Reserved
    10. 10. Dynamic Types, Dynamic Data Dynamic Types: Overview <ul><li>Summary : Define and/or modify type definitions at runtime </li></ul><ul><ul><li>Informed by CORBA TypeCode </li></ul></ul><ul><ul><li>Interoperable with statically defined types </li></ul></ul><ul><li>Small number of fundamental classes: </li></ul><ul><ul><li>DynamicType : A programmatic type description </li></ul></ul><ul><ul><li>DynamicTypeFactory : Creates new types </li></ul></ul><ul><ul><li>DynamicTypeSupport : Registers types with DDS </li></ul></ul>© 2009 RTI - All rights Reserved
    11. 11. Dynamic Types, Dynamic Data Dynamic Types: Example #1 <ul><li>Create the equivalent: </li></ul><ul><li>struct MyType { </li></ul><ul><li>long my_integer; </li></ul><ul><li>}; </li></ul><ul><li>C++ code: </li></ul><ul><li>DynamicTypeFactory* factory = </li></ul><ul><li>DynamicTypeFactory::get_instance() ; </li></ul><ul><li>DynamicType* type = factory-> create_type ( </li></ul><ul><li>TypeDescriptor (&quot; MyType &quot;, STRUCTURE_KIND )); </li></ul><ul><li>type-> add_member ( </li></ul><ul><li>MemberDescriptor ( </li></ul><ul><li>&quot; my_integer &quot;, </li></ul><ul><li>factory-> get_primitive_type ( </li></ul><ul><li>INTEGER_KIND ))); </li></ul>© 2009 RTI - All rights Reserved
    12. 12. Dynamic Types, Dynamic Data Dynamic Types: Example #2 <ul><li>Subscribe to a type you’ve never seen before </li></ul><ul><li>Java code: </li></ul><ul><li>DynamicType type = </li></ul><ul><li>myPubBuiltinTopicData.type ; </li></ul><ul><li>DynamicTypeSupport support = </li></ul><ul><li>DynamicDataTypeFactory.get_instance(). </li></ul><ul><li>create_type_support ( type ); </li></ul><ul><li>support. register_type ( </li></ul><ul><li>myParticipant , </li></ul><ul><li>myPubBuiltinTopicData.type_name ); </li></ul><ul><li>// … init QoS based on built-in topic data </li></ul><ul><li>Topic topic = myParticipant.create_topic ( </li></ul><ul><li>myPubBuiltinTopicData.name, </li></ul><ul><li>myPubBuiltinTopicData.type_name, </li></ul><ul><li>myTopicQos, myTopicListener); </li></ul><ul><li>… </li></ul>© 2009 RTI - All rights Reserved
    13. 13. Dynamic Types, Dynamic Data Dynamic Data: Overview <ul><li>Summary : Introspect and modify samples at runtime </li></ul><ul><ul><li>Informed by CORBA DynAny , JMS MapMessage , TIBCO Rendezvous self-describing messages </li></ul></ul><ul><li>Small number of fundamental classes: </li></ul><ul><ul><li>DynamicData : A sample of any (single) type </li></ul></ul><ul><ul><li>DynamicDataFactory : “Instantiates” a dynamic type </li></ul></ul><ul><ul><li>DynamicDataWriter : Writes samples of any (single) type, each represented by a DynamicData object </li></ul></ul><ul><ul><li>DynamicDataReader : Reads samples of any (single) type, each represented by a DynamicData object </li></ul></ul>© 2009 RTI - All rights Reserved
    14. 14. Dynamic Types, Dynamic Data Dynamic Data: Names and IDs <ul><li>DynamicData gets / sets member values in 2 ways: </li></ul><ul><ul><li>By name : Obvious, human-readable approach </li></ul></ul><ul><ul><li>By ID number : Potentially faster, more efficient search </li></ul></ul><ul><li>Example in C++: </li></ul><ul><li>DynamicData* sample = DynamicDataFactory::get_instance()-> create_data(my_type) ; </li></ul><ul><li>long my_value = 5; </li></ul><ul><li>sample-> set_int ( &quot; my_integer &quot;, // Specify name or ID; MEMBER_ID_ANY , // no need for both. my_value ); // <-- Value to set. </li></ul>© 2009 RTI - All rights Reserved
    15. 15. XML Type Representations XML Type Representations <ul><li>#1: XSD </li></ul><ul><li>Based on IDL-to-WSDL mapping </li></ul><ul><li>Allows type sharing between DDS and web services </li></ul><ul><li>But… </li></ul><ul><li>Very verbose </li></ul><ul><li>Hard to read, hard to write, hard to parse </li></ul><ul><li>#2: XML </li></ul><ul><li>Straightforward mapping of IDL into XML </li></ul><ul><li>Suitable for embedding into other XML documents ( e.g. QoS profile files) </li></ul><ul><li>Easy to read, easy to write, easy to parse </li></ul>© 2009 RTI - All rights Reserved
    16. 16. XML Type Representations XML Type Representations: Examples <ul><li>#1: XSD </li></ul><ul><li><xsd:complexType name=&quot;T&quot;> </li></ul><ul><li><xsd:sequence> </li></ul><ul><li><xsd:element name=&quot;value&quot; minOccurs=&quot;1&quot; maxOccurs=&quot;1&quot; type=&quot;char&quot;/> </li></ul><ul><li></xsd:sequence> </li></ul><ul><li></xsd:complexType> </li></ul><ul><li>#2: XML </li></ul><ul><li><struct name=&quot;T&quot;> </li></ul><ul><li><member name=&quot;value&quot; type=&quot;char&quot;/> </li></ul><ul><li></struct> </li></ul>© 2009 RTI - All rights Reserved
    17. 17. IDL Type Representation IDL Extension: Annotations <ul><li>Summary : Bringing Java-style annotations to IDL </li></ul><ul><ul><li>Don’t just solve today’s problem; </li></ul></ul><ul><ul><li>… also provide an extensible tool we can reuse tomorrow </li></ul></ul><ul><li>Example : </li></ul><ul><li>@AnAnnotationAppliedToTheType </li></ul><ul><li>struct TheType </li></ul><ul><li>@AnAnnotationAppliedToTheMember </li></ul><ul><li>long the_member; </li></ul><ul><li>}; </li></ul>© 2009 RTI - All rights Reserved
    18. 18. IDL Type Representation IDL Extension: Annotations <ul><li>Define annotations: </li></ul><ul><ul><li>@Annotation local interface MyAnnotation { </li></ul></ul><ul><ul><li>long value() default 32; </li></ul></ul><ul><ul><li>}; </li></ul></ul><ul><ul><li>* Why “@Annotation” instead of “@interface”? Coming up… </li></ul></ul><ul><li>Apply annotations: </li></ul><ul><ul><li>struct MyStruct { </li></ul></ul><ul><ul><li>@MyAnnotation long my_field; </li></ul></ul><ul><ul><li>}; </li></ul></ul><ul><li>Code gen translates to Java annotations one-to-one </li></ul>© 2009 RTI - All rights Reserved
    19. 19. IDL Type Representation IDL Extension: Compatible Annotation Syntax <ul><li>Assumption : Some vendors will implement this spec sooner than others </li></ul><ul><li>Assumption : Users share IDL files among products from multiple vendors </li></ul><ul><li>Problem : How to take advantage of new features w/o breaking old IDL compilers with new syntax? </li></ul><ul><li>Solution : Put annotations in single-line “comment,” e.g. : </li></ul><ul><ul><li>You can do this: @Key long my_key_field; </li></ul></ul><ul><ul><li>… or equivalently: long my_key_field; //@Key </li></ul></ul>© 2009 RTI - All rights Reserved
    20. 20. IDL Type Representation IDL Extension: Built-in Annotations <ul><li>@Key </li></ul><ul><ul><li>A member is part of its enclosing type’s key </li></ul></ul><ul><li>@ID </li></ul><ul><ul><li>Specify the ID of a particular type member </li></ul></ul><ul><ul><li>If omitted, ID is assigned automatically </li></ul></ul><ul><li>@Optional </li></ul><ul><ul><li>A type member may be omitted from data samples </li></ul></ul><ul><li>@Extensible </li></ul><ul><ul><li>A type might change in the future, </li></ul></ul><ul><ul><li>… so DDS will use a more-verbose, more-flexible encapsulation </li></ul></ul>© 2009 RTI - All rights Reserved
    21. 21. CDR Data Representation Extensible Binary Data: Challenge <ul><li>Problem : CDR encoded data looks like this: </li></ul><ul><ul><li>10110010001011100001110101000111 </li></ul></ul><ul><li>… So how do I know if I’ve added / removed a field? </li></ul><ul><li>Possible Solution #1 : Include type definition with every sample </li></ul><ul><ul><li>Uses too much bandwidth </li></ul></ul><ul><ul><li>Parsing is too slow </li></ul></ul><ul><li>Possible Solution #2 : Have DataWriter publish its type definition; DataReader uses that to parse samples </li></ul><ul><ul><li>Parsing is still too slow </li></ul></ul>© 2009 RTI - All rights Reserved
    22. 22. CDR Data Representation Extensible Binary Data: Solution <ul><li>Solution : Use member IDs w/ RTPS param CDR </li></ul><ul><ul><li>+----+----+----+----+----+----+----+----+ </li></ul></ul><ul><ul><li>| ID (16) | size (16) | </li></ul></ul><ul><ul><li>+----+----+----+----+----+----+----+----+ </li></ul></ul><ul><ul><li>| data (size) ... | </li></ul></ul><ul><ul><li>+----+----+----+----+----+----+----+----+ </li></ul></ul><ul><li>Benefits : </li></ul><ul><ul><li>Skip fields you don’t understand </li></ul></ul><ul><ul><li>Easily determine which fields are (not) present </li></ul></ul><ul><ul><li>Reorder fields </li></ul></ul><ul><ul><li>Brings RTPS simple discovery into type system mainstream </li></ul></ul><ul><li>Cost : 4 bytes (+ alignment) per member </li></ul>© 2009 RTI - All rights Reserved
    23. 23. CDR Data Representation Extensible Binary Data: Details <ul><li>Types marked as “extensible” encapsulated w/ parameterized CDR, all others w/ “plain” CDR </li></ul><ul><ul><li>Extensible types can contain objects of non-extensible types and vice versa </li></ul></ul><ul><ul><li>100% compatible with existing types and serialization (…since built-in topic data are the only “extensible” types that exist today  ) </li></ul></ul><ul><li>Lots of members in your type? Data sizes > 64 KB? </li></ul><ul><ul><li>+----+----+----+----+----+----+----+----+ </li></ul></ul><ul><ul><li>| ID sentinel (16) | size = 8 (16) | </li></ul></ul><ul><ul><li>+----+----+----+----+----+----+----+----+ </li></ul></ul><ul><ul><li>| extended ID (32) | </li></ul></ul><ul><ul><li>+----+----+----+----+----+----+----+----+ </li></ul></ul><ul><ul><li>| extended size (32) | </li></ul></ul><ul><ul><li>+----+----+----+----+----+----+----+----+ </li></ul></ul><ul><ul><li>| data (extended size) ... | </li></ul></ul><ul><ul><li>+----+----+----+----+----+----+----+----+ </li></ul></ul>© 2009 RTI - All rights Reserved
    24. 24. CDR Data Representation Extensible Binary Data: Type Versioning <ul><li>Problem : How can types change over time – and at different times from subsystem to subsystem? </li></ul><ul><li>Possible Solution : Give types version number, keep track of past versions and who is using which version. </li></ul><ul><ul><li>Too slow : must consult per-remote-writer definition to deserialize </li></ul></ul><ul><ul><li>Requires tracking lots of state – including every version of every type that’s ever existed ( e.g. for late joiners) </li></ul></ul><ul><ul><li>Requires network propagation of all type information: increased bandwidth use, possible security objections </li></ul></ul><ul><li>Solution : Do nothing . Member IDs already provide compact per-member “version.” </li></ul><ul><ul><li>Adding / changing a member? Define a new ID. Old applications can ignore it. </li></ul></ul><ul><ul><li>“ Removing” a member? Just stop sending it. Doesn’t invalidate existing persisted data. </li></ul></ul>© 2009 RTI - All rights Reserved
    25. 25. Summary: Specification Overview © 2009 RTI - All rights Reserved
    26. 26. Summary: Goals and Benefits <ul><li>Portable, consistent type system </li></ul><ul><ul><li>Standardized syntax for keys and other DDS concepts </li></ul></ul><ul><ul><li>General-purpose extensibility (annotations) reusable by future specifications, vendors, and users </li></ul></ul><ul><ul><li>Same features usable by user-defined types as built-in types </li></ul></ul><ul><li>Dynamic access to type definitions </li></ul><ul><ul><li>Introspect types for remote publications, subscriptions </li></ul></ul><ul><ul><li>Define and modify types at runtime </li></ul></ul><ul><ul><li>Publish and subscribe to dynamically defined types </li></ul></ul><ul><ul><li>Interoperate with statically defined types and across vendors </li></ul></ul><ul><li>Simplified integration of DDS into other systems </li></ul><ul><ul><li>XML and XSD type definitions to integrate w/ XML-centric systems </li></ul></ul><ul><ul><li>Built-in types for simplified programming, especially for new users </li></ul></ul><ul><li>Fast : little or no performance overhead in steady state </li></ul>© 2009 RTI - All rights Reserved
    27. 27. Summary: Progress <ul><li>Many more details than previous submission </li></ul><ul><ul><li>Full specification of IDL extensions </li></ul></ul><ul><ul><li>Unification of CDR and parameterized CDR data representations </li></ul></ul><ul><ul><li>Concrete dynamic type, dynamic data APIs </li></ul></ul><ul><ul><li>Refinements to automatic type conversions </li></ul></ul><ul><li>Some work remains </li></ul><ul><ul><li>XML data representation </li></ul></ul><ul><ul><li>Refinements to XML type representation grammars </li></ul></ul><ul><ul><li>Refinements to endpoint matching rules </li></ul></ul><ul><li>Targeted for completion by Dec. 2009 meeting </li></ul>© 2009 RTI - All rights Reserved
    28. 28. Q & A © 2009 RTI - All rights Reserved
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×