Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Extensible and Dynamic Topic Types for DDS

on

  • 1,555 views

Presentation to the OMG MARS Task Force from March of 2010 on the "Extensible and Dynamic Topic Types for DDS" (DDS-XTypes) specification. Following this presentation, the Task Force voted to ...

Presentation to the OMG MARS Task Force from March of 2010 on the "Extensible and Dynamic Topic Types for DDS" (DDS-XTypes) specification. Following this presentation, the Task Force voted to recommend the adoption of this specification.

Statistics

Views

Total Views
1,555
Views on SlideShare
1,555
Embed Views
0

Actions

Likes
1
Downloads
25
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Lots of material in a short amount of time, so here’s the format:If I’m not clear, let me know right away.If you have a quick question, go ahead and ask itIf you have a longer question of discussion point, we’ll talk about it at the end.
  • DDS is unique among pub-sub systems in providing interoperable static type safety.Applications that use data already expect certain contents; by making expectations explicit, enable: type-aware integration (e.g. relational mapping) compact network representation.
  • Many systems are already committed to, deployed on DDS 1.2, RTPS 2 (such as, many international navies). Breaking compatibility with those specs makes integration really hard, costs a lot of time and money for users an vendors alike. Don’t do it.People choose DDS for its performance. Don’t break that either.
  • Already, DDS conformance and RTPS conformance are independent. This spec pertains to both programming APIs and network representation, so it makes the same distinction.
  • Remember why we decided to use parameterized encoding for discovery in the first place:We knew it would change in future versions.We knew vendors would add extensions.But we created a special enclave: built-in topic data types are extensible, but no one else.This spec generalizes the model that has proven itself over a decade.
  • Nothing very new here: IDL  WSDL let you do this before. Now it’s official.
  • C, C++, Java covers almost all DDS users.Ada is a tiny fraction.Scripting languages are covered by Web-Enabled.
  • This is one common use case. After we talk about the different pieces, we’ll see how to accomplish it.
  • “DECLARED” is the default for conforming implementations. Call get_default_qos and you will get this.When a conforming implementation talks to a non-conforming implementation, this policy will be missing, so according to the type defaulting rules, it will assume “EXACT_TYPE.”

Extensible and Dynamic Topic Types for DDS Extensible and Dynamic Topic Types for DDS Presentation Transcript

  • Extensible and DynamicTopic Types for DDSMARS – Jacksonville, FL – March 2010
    Presenter: Rick Warren, RTI
    Submitting POCs:
    • Angelo Corsaro, PrismTech: angelo.corsaro@prismtech.com
    • Gerardo Pardo, RTI: gerardo.pardo@rti.com
    • Robert Poth, PrismTech: robert.poth@prismtech.com
    • Rick Warren, RTI: rick.warren@rti.com
    Supporting POCs:
    • VirginieWatine, Thales: virginie.watine@thalesgroup.com
    document number: 2010-03-02
  • Structure of this Presentation
    Overview of specification and process
    Specification contents
    Very detailed discussion last time
    This presentation focuses on what’s changed
    AB feedback
    Copyright © 2010 RTI - All rights Reserved
    2
  • DDS Types: Assessment and Challenges
    Good
    Bad
    Unmatched type safety => fewer defects
    Unique type awareness => faster integration
    Unique type awareness => higher performance
    No standard API to define / access types at runtime
    Hard to change types without rebuilding, redeploying
    Hard to upgrade systems one subsystem at a time
    Copyright © 2010 RTI - All rights Reserved
    3
    Vendor-Specific Solutions
    OMG-Standard Solutions
  • Where We Are
    RFP issued: Jun. 2008
    Initial submission: Nov 2008
    Revised submission extended: Feb. 2009
    Revised submission extended: Aug. 2009
    Revised submission extended: Nov. 2009
    Revised submission deadline: Feb. 2010
    Submitters back on the same page
    Responded to comments from 2 (now 3) AB reviewers
    Discussed with Task Force in Long Beach meeting
    Copyright © 2010 RTI - All rights Reserved
    4
  • Design Principles
    Don’t break anybody.
    Many people depend on DDS
    Don’t modify existing APIs
    Don’t modify existing compliance points
    Many people depend on RTPS/DDSI 2
    Don’t modify existing protocols or formats
    Don’t modify existing compliance points
    Keep it fast.
    New capabilities must permit efficient implementations.
    Don’t use them? Don’t pay for them.
    Keep it orthogonal. Type compatibility doesn’t depend on:
    Types’ definition language(s)
    Data representation(s)
    Programming language(s)
    QoS configuration
    Copyright © 2010 RTI - All rights Reserved
    5
  • Overview
    Copyright © 2010 RTI - All rights Reserved
    6
    Type System
    Type Representation
    Language Binding
    Data Representation
  • Overview: Example
    Copyright © 2010 RTI - All rights Reserved
    7
    Type
    Representation
    IDL:
    Foo.idl
    struct Foo {
    string name;
    long ssn;
    };
    Language
    Binding
    IDL to Language Mapping:
    Foo.h
    Foo.c
    FooTypeSupport.c
    struct Foo {
    char *name;
    int ssn;
    };
    Foo f = {"hello", 2};
    Data
    Representation
    IDL to CDR:
    00000006
    68656C6C
    6F000000
    00000002
  • Conformance Changes
    Feedback on previous submission: conformance criteria, relationships to other specs unclear
    New submission more precise, better organized:
    Conformance (part II, section 3):2 conformance levels correspond to DDS, RTPS
    Programming language conformance:Type system, type representations, language bindings, DDS Minimum Profile
    Network interoperability conformance:Type system, data representations, RTPS
    Relations to other specs (part III, section 8)
    Extensions to DDS:
    “Programming language conformance” sections constitute new optional conformance level (“profile”)
    No changes to existing DDS profiles
    Extensions to IDL:
    Normative to impl.’s of this spec only;no change to CORBA conformance
    Copyright © 2010 RTI - All rights Reserved
    8
  • Type System
    Extensible, Compatible
    6.5.1: (a) Type System as UML meta-model. (b) Type substitutability rules.
    6.5.2: Type extensibility, mutability rules
    6.5.3: Type compatibility in presence of change
    6.5.17: Allow type to evolve without breaking interoperability
    6.5.5: Support keys. How does extensibility apply?
    More Expressive
    6.5.4: Map data type
    6.5.6, 6.5.19: Sparse data: object contains only subset of fields defined by its type
    Copyright © 2010 RTI - All rights Reserved
    9
  • Type System Overview
    Copyright © 2010 RTI - All rights Reserved
    10
  • Type System Overview
    Entirely familiar from IDL:
    Primitives
    Strings, narrow and wide
    Arrays and sequences
    Aliases (typedefs)
    Unions
    Modules
    As in IDL, but extended:
    Structures—including single inheritance(for substitutability: 6.5.1)
    Enumerations—specify bit width and constant values
    New relative to IDL:
    Maps—like std::mapor java.util.Map;required by RFP (6.5.4), fundamental OO collection
    Annotations—for extensibility (6.5.2, 6.5.3)
    Copyright © 2010 RTI - All rights Reserved
    11
    module Foo {
    struct Bar {
    long baz;
    };
    typedef Bar Quux;
    }
  • Type System Changes
    Clearer, more-flexible type extensibility kinds
    Previous kind (Boolean): “extensible”=true:
    Add/remove/reorder type members
    Renamed to “mutable”
    Previous kind (Boolean): “extensible”=false:
    Add to end, but don’t add elsewhere, remove, or reorder
    Renamed to “extensible”
    Now allow removing from end as well as adding to end
    New kind “final”:
    Don’t allow any modifications
    Explicit annotations identify all levels in IDL
    Un-annotated types presumed “extensible” (2)
    Can be overridden by IDL compiler option
    Several concrete examples added
    Copyright © 2010 RTI - All rights Reserved
    12
  • Type Extensibility Example
    @Extensibility(
    EXTENSIBLE)
    struct Coord2D {
    long x;
    long y;
    };
    @Extensibility(
    EXTENSIBLE)
    struct Coord3D {
    long x;
    long y;
    long z;
    };
    Copyright © 2010 RTI - All rights Reserved
    13
    Assignable
  • Type Extensibility Example
    @Extensibility(
    FINAL)
    struct Coord2D {
    long x;
    long y;
    };
    @Extensibility(
    FINAL)
    struct Coord3D {
    long x;
    long y;
    long z;
    };
    Copyright © 2010 RTI - All rights Reserved
    14
    Not Assignable
  • Type Representations
    • IDL
    • XML, XSD
    • TypeObject
    Copyright © 2010 RTI - All rights Reserved
    15
    Static Definition
    • 6.5.11: Programming-language-independent type serialization format(s)
    • 6.5.9: Represent types in IDL and XML
    • 6.5.7: Support IDL subset used by DDS 1.2
    Dynamic Definition
    • 6.5.12: Type discovery
  • IDL Type Representation Changes
    Added optional pragmas to delimit IDL segments using new extended syntax
    #pragmadds_xtopics begin [<version_number>]
    // IDL definitions
    #pragmadds_xtopics end
    Document conformance requirements for human readers
    Help non-conforming compilers recognize problems sooner, give more meaningful errors
    No particular errors or warnings mandated
    Copyright © 2010 RTI - All rights Reserved
    16
  • IDL Type Representation Changes
    Use struct syntax for inheritance, not valuetype
    structMyStruct : MyBaseStruct { … };
    Type system doesn’t include all valuetype features, so syntax was confusing
    Unclear whether valuetype could extend struct
    Copyright © 2010 RTI - All rights Reserved
    17
  • IDL Type Representation Changes
    Cleaner user-defined annotation syntax
    Previously used methods, like Java annotations:
    @Annotation local interface MyAnnotation {
    long my_annotation_member();
    };
    Now uses attributes:
    @Annotation local interface MyAnnotation {
    attribute long my_annotation_member;
    };
    Still uses local interface (alternative: struct)
    Emphasizes that no runtime state exists
    Allows future support for multiple inheritance, if desired
    Copyright © 2010 RTI - All rights Reserved
    18
  • XML Type Representations (No Change)
    XML Type Representation
    Terse XML syntax designed ground-up for this Type System
    Easy to read, plays nicely with auto-completing editors
    XSD Type Representation
    Based on existing IDL-to-WSDL mapping specification
    Extended for new Type System concepts from this spec
    Copyright © 2010 RTI - All rights Reserved
    19
  • TypeObject Changes
    Type Representation suitable for DDS discovery
    Renamed from “TypeCode”
    Name similarity to CORBA TypeCode deemed A Bad Thing: this spec doesn’t depend on that one
    .Net also has “TypeCode,” something else entirely…
    Improved string sharing to make representations much more compact
    Copyright © 2010 RTI - All rights Reserved
    20
  • Data Representations
    • Extended CDR
    • XML
    Copyright © 2010 RTI - All rights Reserved
    21
  • CDR Data Representation (No Change)
    Requirements:
    Continue supporting existing (compact) CDR used by user-defined types (6.5.16)
    Continue supporting existing (extensible, parameterized) CDR used by discovery types (6.5.16)
    Provide extensibility to user-defined types (6.5.2, 6.5.3, 6.5.17)—all Data Representations should have equivalent expressiveness
    Solution: Make existing extensible, parameterized CDR available to all
    Same rules for every type, built-in or user-defined
    100% compatible with existing types and serialization
    Parameterized CDR: structs & unions marked “mutable”
    Each member == parameter
    Member ID == parameter ID
    Compact CDR: everything else
    Copyright © 2010 RTI - All rights Reserved
    22
  • XML Data Representation (No Change)
    Optional Requirement (6.6.1):Dynamic data access based on XML or JSON
    Given: XSD Type Representation
    Data object == XML document
    Validated by XSD Representation of its type
    SAX, DOM-based access follow naturally
    New RTPS encapsulation ID lets you use XML on the network
    Copyright © 2010 RTI - All rights Reserved
    23
    structOriginalLandData {
    long x;
    long y;
    };
    <OriginalLandData>
    <x>5</x>
    <y>42</y>;
    </OriginalLandData>
  • Language Bindings
    • Plain Language Binding
    • Dynamic Language Binding
    Copyright © 2010 RTI - All rights Reserved
    24
  • Plain Language Binding (No Change)
    Defined by existing IDL-to-language mappings:
    Start with IDL Representation of a type
    Apply mapping to desired programming language
    Mappings expanded to cover new concepts
    e.g. maps, annotations, optional members
    For C, C++, Java as required by RFP (6.5.8)
    Copyright © 2010 RTI - All rights Reserved
    25
    // IDL
    structOrigLandData {
    long x;
    long y;
    };
    // Java
    public class OrigLandData {
    public intx;
    public inty;
    }
  • Dynamic Language Binding (No Change)
    Classes
    DynamicTypeFactory: Creates new types (6.5.20)
    DynamicType: Introspect type definitions (6.5.13)
    Informed by CORBA TypeCode API
    DynamicData: Introspect objects (6.5.18)
    Informed by CORBA DynAny, JMS MapMessage,TIBCO Rendezvous self-describing messages
    DynamicTypeSupport: Registers types with DDS (6.5.20)
    DynamicDataWriter: Writes objects of any (single) type, each represented by a DynamicData object (6.5.21)
    DynamicDataReader: Reads objects of any (single) type, each represented by a DynamicData object (6.5.21)
    Copyright © 2010 RTI - All rights Reserved
    26
  • Use by DDS
    • Data Representation Compatibility
    • Type Compatibility
    • Built-in Types
    Copyright © 2010 RTI - All rights Reserved
    27
  • Data Representation Compatibility
    Reader and writer must agree on which Data Representation to use
    6.5.14: Encapsulation specification, negotiation
    6.5.15: Is encapsulation request-offer?
    6.6.2: Optional: Custom encapsulation API
    Largely unchanged from previous submission
    Solution: DataRepresentationQosPolicy
    List of representation IDs (standard = XCDR, XML)
    Applies to both readers and writers
    Included in corresponding built-in topic data types
    Request-Offer semantics
    Writer offers single representation it will use
    Reader requests a list of representations
    Compatible iff writer’s representation is in reader’s list
    Can describe proposed representations, future additions, and vendor extensions
    Copyright © 2010 RTI - All rights Reserved
    28
  • Type Compatibility
    If I write type “Foo” and you read type “Bar”, can we communicate?
    New submission clearer, more flexiblebased on feedback on previous submission
    3 parts:
    Physical: type assignability
    Intentional: type signature
    Enforcement: QoS policy governs how above are applied (NEW in this submission)
    Copyright © 2010 RTI - All rights Reserved
    29
  • Type Compatibility Example: What I Want
    struct Coord2D
    {
    long x;
    long y;
    };
    struct Coord3D :
    Coord2D
    {
    long z;
    };
    Copyright © 2010 RTI - All rights Reserved
    30
    Read Write
    Write Read
  • 1. Type Assignability
    Structural compatibility:Can sample serialized according to one type be deserialized according to another?
    Reminder: Rules designed to require no writer-specific runtime checks at deserialization time
    Intrinsic property of types’ definitions
    Governed in part by extensibility kind
    No action required by application programmer
    Copyright © 2010 RTI - All rights Reserved
    31
  • 2. Type Signature
    Do I intend the types to be compatible?
    Lists of equivalent types, base types provided at registration time
    Clarification to previous submission: both reader and writer declarations taken into account
    New since previous submission: lists can contain patterns, not just literal names
    Copyright © 2010 RTI - All rights Reserved
    32
  • 3. Type Consistency Enforcement Policy
    QoS policy on Topic, DataReader, DataWritergoverns how (1) and (2) take effect
    RxO: reader and writer must have the same setting
    Copyright © 2010 RTI - All rights Reserved
    33
  • Type Compatibility Example: How to Get It
    Is Coord2D assignable from Coord3D? Yes.
    Type Signature:Coord3DTypeSupport.register_type(my_participant, "Coord3D : Coord2D")
    Type Consistency Enforcement: DECLARED (default)
    Copyright © 2010 RTI - All rights Reserved
    34
    struct Coord2D
    {
    long x;
    long y;
    };
    struct Coord3D :
    Coord2D
    {
    long z;
    };
  • Built-in Types (No Change)
    Goal: Improve out-of-box experience for new users, those with simple needs
    Background: Usability gap vs. competition
    Other middlewares allow sending textual or binary data without explicit type definition, registration
    Response: Provide simple types built in
    Pre-defined, pre-registered
    String: single string payload
    Bytes: octet sequence payload
    KeyedString: string payload + string key
    KeyedBytes: octet sequence payload + string key
    Copyright © 2010 RTI - All rights Reserved
    35
  • Summary
    Copyright © 2010 RTI - All rights Reserved
    36
  • AB Feedback
    Incorporates feedback from 3 AB reviewers:
    Define IDL pragma to delimit extensions: Done
    Clarify relationships to other specs: Done
    Provide more detail about modules and namespaces: Done
    Make annotation type definition syntax more natural: Done
    Standardize VerbatimText behavior for C, C++, Java languages: Done
    Assorted typos, wording changes, organizational suggestions: Done
    Copyright © 2010 RTI - All rights Reserved
    37
  • Summary
    The proposal:
    Satisfies all mandatory and optional requirements andsupports identified real-world use cases
    Without sacrificing backwards compatibility
    Without sacrificing performance
    Is easily extensible w/ new type, data representations to handle additional use cases
    Only small changes since previous submission
    Incorporates all AB feedback
    We will be asking for a vote to recommend adoption.
    Copyright © 2010 RTI - All rights Reserved
    38
  • Q & A
    Copyright © 2010 RTI - All rights Reserved
    39