From the Tactical Edge to the Enterprise: Integrating DDS and JMS Rick Warren, Principal Engineer
Introduction <ul><li>Rick Warren </li></ul><ul><li>Principal Engineer, RTI </li></ul><ul><li> [email_address] </li></ul><...
Problem Statement <ul><li>JMS established in enterprise applications </li></ul><ul><ul><li>Integrates web services, legacy...
Problem Statement <ul><li>Large JMS-compatible ecosystem already exists </li></ul><ul><li>DDS-compatible ecosystem still d...
Solution Alternatives <ul><li>What do you mean by “JMS”? </li></ul><ul><li>“ JMS” as a middleware implementation </li></ul...
Solution Alternatives <ul><li>What do you mean by “JMS”? </li></ul><ul><li>“ JMS” as a portability API </li></ul><ul><ul><...
Solution Alternatives: JMS PSM for DDS <ul><li>JMS front end to leverage existing JMS-compatible components </li></ul><ul>...
System Architecture © 2009 Real-Time Innovations, Inc.
Performance and Fault Tolerance: DDS    JMS Bridge <ul><li>Leverages existing components and infrastructure while isolat...
Performance and Fault Tolerance: DDS    JMS Bridge <ul><li>Performance inversely proportional to number of clients per b...
Throughput of Data-Centric Implementation |   Traditional JMS Implementations     | JMS PSM for DDS © 2009 Real-Time Inn...
Data Model © 2009 Real-Time Innovations, Inc.  http://blogs.rti.com/2009/04/20/designing-information-models-for-distribute...
<ul><li>Data-Centric Model </li></ul><ul><li>Deliver updates to the data itself </li></ul><ul><ul><li>Most effective: data...
DDS    JMS Data Interoperability <ul><li>Specialized Mapping </li></ul><ul><ul><li>Build custom transformation in system...
DDS Representation of JMS Messages <ul><li>Any JMS  Message  can be sent to any  Destination </li></ul><ul><ul><li>BytesMe...
DDS Representation of JMS Messages: Message Properties <ul><li>struct  Message { </li></ul><ul><li>sequence <NameValuePair...
DDS Representation of JMS Messages: Message Body <ul><li>struct  Message { </li></ul><ul><li>sequence<NameValuePair> prope...
JMS Representation of DDS Messages: Message Body <ul><li>struct  MyType </li></ul><ul><li>long   my_long; </li></ul><ul><l...
JMS Representation of DDS Messages: Sample Metadata and Instance Lifecycle <ul><li>DDS Meta-Data: </li></ul><ul><li>Key fi...
JMS Representation of DDS Messages: Sample Metadata and Instance Lifecycle <ul><li>Publication : DDS  unregister()  and  d...
Destinations and Delivery © 2009 Real-Time Innovations, Inc.
A Topic is a Topic is a Topic… <ul><li>DDS and JMS both describe  topics  for 1-to-many comm. </li></ul><ul><li>DDS specif...
A Topic is a Topic is a Topic… <ul><li>DDS Topic = JMS Topic </li></ul><ul><li>Simple topic name </li></ul><ul><li>Specifi...
A Queue is a … ? <ul><li>JMS defines queues  in addition to topics; DDS doesn’t </li></ul><ul><ul><li>At-least-once delive...
Conclusions © 2009 Real-Time Innovations, Inc.
Bridged Integration Leverages Existing JMS Subsystems <ul><li>Preserves  investment in JMS design, which may or may not be...
JMS PSM for DDS Empowers Data-Centric Integration with Enterprise Infrastructure <ul><li>Requires  JMS provider with data-...
Information Mapping Concerns are the Same, Regardless of System Architecture <ul><li>Information Model Components </li></u...
Needed: Standards Leadership <ul><li>QoS profiles  allow configuration where JMS API is lacking </li></ul><ul><li>Extensib...
Q & A © 2009 Real-Time Innovations, Inc.
Upcoming SlideShare
Loading in …5
×

From the Tactical Edge to the Enterprise: Integrating DDS and JMS

1,347 views

Published on

Presentation to a Real-Time Workshop of the Object Management Group (OMG) describing concerns, approaches, and best practices for integrating systems based on DDS and JMS middleware in July, 2009.

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,347
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
62
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • Context for this talk: Many people trying to find best practices for: 1. Taking data from “edge” systems (real-time and/or tactical systems), 2. Transforming data into situational awareness/business intelligence 3. By fusing, correlating, mining that data Technically, means connecting edge systems to DBs, CEP engines, app servers, etc. People doing this now, but in ad hoc way. Very broad problem; this talk is narrow: integrating messaging and data distribution across subsystems.
  • What do I mean by a “data-centric core” and “JMS PSM”?
  • The performance penalty will be decreased somewhat if one or both of the adapters shown can run in-process with the broker (such a configuration may or may not be possible; see below). However, if the system depends on the broker for other critical functions, running these adapters in-process means potentially exposing other system components to failures in those adapters. The performance penalty will be increased if the DDS implementation and/or the JMS implementation require per-node daemons, as these introduce additional data copies, context switches, potential priority inversions, and potential points of failure.
  • Company Confidential
  • Benefits of data-centric, “JMS PSM” approach should now be clear. Remaining concerns mostly apply to either approach. First: data model differences.
  • Before we can talk about the particulars of data definition and representation, we have to understand the different communication models typical of each API. … but just as you can write object-oriented code in C or procedural code in Java, you can design message-centric or data-centric systems around either JMS or DDS.
  • On to the particulars: how do we transform the data? Two approaches considered in detail: JMS-centric: IDL representation of JMS messages DDS-centric: JMS Message implementation in terms of DDS-compatible data
  • Approach #1: IDL definition for all JMS message types, all unioned together
  • Two main parts of the message: properties and body. Zoom in on properties first.
  • Now zoom in on body. Before we move on, a summary of where we stand: If problem is just to represent JMS system in DDS, then don’t need to worry about keys/instances, instance lifecycle, etc.: JMS doesn’t have those. … but then DDS system needs to know it’s talking to JMS, and approach loses a lot of value. Next part of the presentation provides a more general mapping.
  • Approach #2: Define JMS message interfaces to access DDS-compatible data Currently, implementation would be DDS-vendor-specific Extensible Topics specification will enable DDS-vendor-agnostic implementation (mars/08-06-22)
  • Presentation of DDS-compatible meta-data through JMS API eases data-centric design in JMS … but doesn’t require it: ignore these properties and design message-centric system
  • Instance lifecycle not totally transparent to JMS application. May be necessary to allow user to turn DDS lifecycle support on/off.
  • Talked about data and messages. But where do we send those messages? Next step: delivering messages to destinations.
  • JMS topic names may be simple or hierarchical and may have any syntax: flat, directory, dot-delimited, LDAP/other URL, etc. Changing JMS vendors can entail info model change: applies to any vendor change … but possibly even more so when moving from traditional JMS to JMS with more data-centric capabilities.
  • JMS gives great freedom to vendors. Could be bad wrt JMS portability, but one good thing: allows JMS topics to be defined in terms of DDS topics OMG can show leadership in JMS configuration; QoS profile format from DDS/LwCCM spec is relevant but not perfect match (e.g. terminology differences)
  • Can previous model (DDS destination == JMS destination) work for queues too? Queues don’t promise much – this is also a benefit when mapping to DDS.
  • Each ESB and CEP vendor has their own adapter APIs. JMS adapters usually come out of the box; DDS adapters don’t. DDS adapters for some products may be available from your DDS vendor. Otherwise, build your own.
  • In the shorter term, if you have an existing design and code base to preserve, and if you can live with the limitations of bridging, then bridge. In the longer term, if you’re making a strategic investment, work towards a unified information model based on a data-centric JMS implementation.
  • QoS profiles and libraries done: DDS for Lightweight CCM (ptc/2009-02-02), UML Profile for DDS Extensible Topics in progress: mars/08-06-22 Java 5 PSM discussed in SIG, but no process started
  • From the Tactical Edge to the Enterprise: Integrating DDS and JMS

    1. 1. From the Tactical Edge to the Enterprise: Integrating DDS and JMS Rick Warren, Principal Engineer
    2. 2. Introduction <ul><li>Rick Warren </li></ul><ul><li>Principal Engineer, RTI </li></ul><ul><li> [email_address] </li></ul><ul><li> http://blogs.rti.com/author/rtirick/ </li></ul><ul><li>R&D, previously Services </li></ul><ul><li>Integration : usability, portability, scalability, *ility </li></ul><ul><li>Agenda </li></ul><ul><li>DDS+JMS Integration Challenge </li></ul><ul><li>DDS & JMS: Towards Best Practices </li></ul><ul><ul><li>System Architecture </li></ul></ul><ul><ul><li>Data Model </li></ul></ul><ul><ul><li>Destinations </li></ul></ul><ul><li>Closing Remarks </li></ul><ul><li>Q & A </li></ul>© 2009 Real-Time Innovations, Inc. Assumption: Some knowledge of DDS and JMS
    3. 3. Problem Statement <ul><li>JMS established in enterprise applications </li></ul><ul><ul><li>Integrates web services, legacy applications, etc. </li></ul></ul><ul><ul><li>App servers, ESBs provide adapters </li></ul></ul><ul><li>DDS, newcomer, successful at edge and moving in </li></ul><ul><ul><li>For example: </li></ul></ul><ul><ul><ul><li>Sensor networks </li></ul></ul></ul><ul><ul><ul><li>Distributed real-time control </li></ul></ul></ul><ul><ul><ul><li>Military tactical systems </li></ul></ul></ul><ul><ul><li>Offers things JMS doesn’t: </li></ul></ul><ul><ul><ul><li>Improved determinism </li></ul></ul></ul><ul><ul><ul><li>QoS for explicit configuration and resource management </li></ul></ul></ul><ul><ul><ul><li>Cross-vendor interoperability </li></ul></ul></ul>© 2009 Real-Time Innovations, Inc.
    4. 4. Problem Statement <ul><li>Large JMS-compatible ecosystem already exists </li></ul><ul><li>DDS-compatible ecosystem still developing </li></ul><ul><li>… So how do I plug my DDS app into a JMS-compatible system? </li></ul>© 2009 Real-Time Innovations, Inc.
    5. 5. Solution Alternatives <ul><li>What do you mean by “JMS”? </li></ul><ul><li>“ JMS” as a middleware implementation </li></ul><ul><ul><li>e.g. WebLogic JMS, TIBCO EMS </li></ul></ul><ul><ul><li>JMS and DDS subsystems can’t communicate directly: different protocols (even across JMS implementations) </li></ul></ul><ul><ul><li> Build a bridge; could be based on ESB or CEP engine </li></ul></ul>© 2009 Real-Time Innovations, Inc. RTPS ??? DDS App Adapter Broker Adapter JMS App
    6. 6. Solution Alternatives <ul><li>What do you mean by “JMS”? </li></ul><ul><li>“ JMS” as a portability API </li></ul><ul><ul><li>Raison d'être for JMS specification </li></ul></ul><ul><ul><li>Implement JMS API atop data-centric middleware core </li></ul></ul><ul><ul><li>aka “JMS PSM” for DDS </li></ul></ul>© 2009 Real-Time Innovations, Inc. DDS App JMS App RTPS
    7. 7. Solution Alternatives: JMS PSM for DDS <ul><li>JMS front end to leverage existing JMS-compatible components </li></ul><ul><li>Data-centric back end for interop. with DDS systems </li></ul>© 2009 Real-Time Innovations, Inc. JMS API Data-Centric Core RTPS App Servers, ESBs, etc. Real-Time Systems
    8. 8. System Architecture © 2009 Real-Time Innovations, Inc.
    9. 9. Performance and Fault Tolerance: DDS  JMS Bridge <ul><li>Leverages existing components and infrastructure while isolating subsystems </li></ul><ul><li>Broker is handy place to locate data translation, impedance mismatching, and other services </li></ul>DDS App © 2009 Real-Time Innovations, Inc. Adapter Broker Adapter JMS App copy copy pt of failure pt of failure pt of failure
    10. 10. Performance and Fault Tolerance: DDS  JMS Bridge <ul><li>Performance inversely proportional to number of clients per broker </li></ul>DDS App © 2009 Real-Time Innovations, Inc. Adapter Broker Adapter JMS App DDS App DDS App JMS App JMS App
    11. 11. Throughput of Data-Centric Implementation |  Traditional JMS Implementations  | JMS PSM for DDS © 2009 Real-Time Innovations, Inc. http://www.rti.com/products/jms/latency-throughput-benchmarks.html
    12. 12. Data Model © 2009 Real-Time Innovations, Inc. http://blogs.rti.com/2009/04/20/designing-information-models-for-distributed-applications/
    13. 13. <ul><li>Data-Centric Model </li></ul><ul><li>Deliver updates to the data itself </li></ul><ul><ul><li>Most effective: data lives in middleware </li></ul></ul><ul><ul><li>Leverages configuration (QoS)-driven behavior </li></ul></ul><ul><li>… Transparently, according to a well-defined data type </li></ul><ul><li>… Using user-defined interfaces </li></ul><ul><li>Message-Centric Model </li></ul><ul><li>Deliver messages about changes to the data </li></ul><ul><ul><li>Most effective: data lives in app-managed store </li></ul></ul><ul><ul><li>Behaviors implemented in application code </li></ul></ul><ul><li>… Opaquely, according to an implicit or ad hoc “data type” </li></ul><ul><li>… Using middleware-defined interfaces </li></ul>© 2009 Real-Time Innovations, Inc. Many distributed systems contain both, regardless of the API(s) they use. Data-Centric vs. Message-Centric Design http://blogs.rti.com/2009/06/03/thinking-differently-about-messaging/
    14. 14. DDS  JMS Data Interoperability <ul><li>Specialized Mapping </li></ul><ul><ul><li>Build custom transformation in system-specific bridge </li></ul></ul><ul><li>General-Purpose Mappings </li></ul><ul><ul><li>Pack DDS-compatible data objects into JMS ObjectMessage </li></ul></ul><ul><ul><li>Define DDS-compatible type for representing JMS messages </li></ul></ul><ul><ul><li>Implement Message interfaces reflectively in terms of DDS-compatible data </li></ul></ul>© 2009 Real-Time Innovations, Inc.  More Specific More Generic   Application JMS Provider 
    15. 15. DDS Representation of JMS Messages <ul><li>Any JMS Message can be sent to any Destination </li></ul><ul><ul><li>BytesMessage </li></ul></ul><ul><ul><li>MapMessage </li></ul></ul><ul><ul><li>ObjectMessage </li></ul></ul><ul><ul><li>StreamMessage </li></ul></ul><ul><ul><li>TextMessage </li></ul></ul><ul><li>… but DDS Topic is strongly typed </li></ul>© 2009 Real-Time Innovations, Inc. module JMS { struct BytesBody { … }; struct StreamBody { … }; struct MapBody { … }; struct TextBody { … }; union switch (BodyType) { case BYTES: BytesBody bytes; case STREAM: StreamBody stream; case MAP: MapBody map; case TEXT: TextBody text; } } IDL Definition
    16. 16. DDS Representation of JMS Messages: Message Properties <ul><li>struct Message { </li></ul><ul><li>sequence <NameValuePair> </li></ul><ul><li>properties; </li></ul><ul><li>… ; </li></ul><ul><li>}; </li></ul><ul><li>struct NameValuePair { </li></ul><ul><li>string name; </li></ul><ul><li>union switch (ValueKind) { </li></ul><ul><li>case SHORT_KIND: </li></ul><ul><li>short short_value; </li></ul><ul><li>case LONG_KIND: </li></ul><ul><li>long long_value; </li></ul><ul><li>case …: </li></ul><ul><li>} value; </li></ul><ul><li>}; </li></ul><ul><li>Any Message can contain arbitrary name-value pairs </li></ul>© 2009 Real-Time Innovations, Inc. <ul><li>IDL is verbose </li></ul><ul><li>Content filtering is a challenge </li></ul>
    17. 17. DDS Representation of JMS Messages: Message Body <ul><li>struct Message { </li></ul><ul><li>sequence<NameValuePair> properties; </li></ul><ul><li>union switch (BodyKind) { </li></ul><ul><li>case BYTES_KIND: </li></ul><ul><li>sequence < octet > bytes; </li></ul><ul><li>case MAP_KIND: </li></ul><ul><li>sequence <NameValuePair> map; </li></ul><ul><li>case OBJECT_KIND: </li></ul><ul><li>sequence < octet > object; </li></ul><ul><li>case TEXT_KIND: </li></ul><ul><li>string text; </li></ul><ul><li>case …: </li></ul><ul><li>} body; </li></ul><ul><li>}; </li></ul>© 2009 Real-Time Innovations, Inc. Not transparent : DDS subsystem coupled to JMS concepts
    18. 18. JMS Representation of DDS Messages: Message Body <ul><li>struct MyType </li></ul><ul><li>long my_long; </li></ul><ul><li>string my_string; </li></ul><ul><li>}; </li></ul><ul><li>int intVal = msg.getInt( &quot;my_long&quot;); </li></ul><ul><li>msg.setString( &quot;my_string&quot;, &quot;my value&quot;); </li></ul>© 2009 Real-Time Innovations, Inc. IDL: JMS:
    19. 19. JMS Representation of DDS Messages: Sample Metadata and Instance Lifecycle <ul><li>DDS Meta-Data: </li></ul><ul><li>Key fields </li></ul><ul><li>Sample state, view state, instance state </li></ul><ul><li>Source and reception time stamps </li></ul><ul><li>Generation counts and ranks </li></ul><ul><li>Proposal : </li></ul><ul><li>JMS_ * message properties </li></ul><ul><li>* JMS app must be aware of DDS if it uses these, but it doesn’t have to </li></ul>© 2009 Real-Time Innovations, Inc.
    20. 20. JMS Representation of DDS Messages: Sample Metadata and Instance Lifecycle <ul><li>Publication : DDS unregister() and dispose() </li></ul><ul><li> send() message with a property set </li></ul><ul><li>Subscription : Detecting unregistration/disposal </li></ul><ul><li> Receive (body-less?) message with a property set </li></ul><ul><li>Open Issue : Not transparent to JMS applications  May not be compatible with existing JMS subsystems </li></ul>© 2009 Real-Time Innovations, Inc.
    21. 21. Destinations and Delivery © 2009 Real-Time Innovations, Inc.
    22. 22. A Topic is a Topic is a Topic… <ul><li>DDS and JMS both describe topics for 1-to-many comm. </li></ul><ul><li>DDS specification rigorously defines what a Topic is </li></ul><ul><ul><li>Name: literal string </li></ul></ul><ul><ul><li>Data type </li></ul></ul><ul><ul><li>QoS configuration: fully specified, application-visible </li></ul></ul><ul><li>JMS specification leaves Topic concept to vendors </li></ul><ul><ul><li>Names of unspecified syntax; info model flat or hierarchical </li></ul></ul><ul><ul><li>No data type </li></ul></ul><ul><ul><li>Config. vendor-specific; not available through std. interfaces </li></ul></ul><ul><ul><li> Changing JMS vendors can entail information model changes: configuration, code, and/or design changes! </li></ul></ul>© 2009 Real-Time Innovations, Inc.
    23. 23. A Topic is a Topic is a Topic… <ul><li>DDS Topic = JMS Topic </li></ul><ul><li>Simple topic name </li></ul><ul><li>Specified data type </li></ul><ul><ul><li>Enforce message contents based on type definition </li></ul></ul><ul><ul><li>… or use “self-describing” type as in previous above </li></ul></ul><ul><li>DDS-compatible configuration </li></ul>© 2009 Real-Time Innovations, Inc.
    24. 24. A Queue is a … ? <ul><li>JMS defines queues in addition to topics; DDS doesn’t </li></ul><ul><ul><li>At-least-once delivery : messages must be kept until ack’ed </li></ul></ul><ul><ul><li>Only relevant message ordering is per-producing Session </li></ul></ul><ul><ul><li>Behavior with respect to concurrent consumers is unspecified </li></ul></ul><ul><li>JMS queues can be mapped to DDS topics </li></ul><ul><ul><li>DDS addition beneficial : at-least-once delivery (or at-least- n ) </li></ul></ul><ul><ul><ul><li>Proposal : min_acknowledgements = [ 1 , n ], LENGTH_UNLIMITED </li></ul></ul></ul><ul><ul><ul><li>In DurabilityQosPolicy ? </li></ul></ul></ul><ul><ul><li>Existing 1-to-many delivery is compliant; does it match use case? </li></ul></ul><ul><li>No def’n of exactly-once delivery in either spec Open issue for application developers </li></ul>© 2009 Real-Time Innovations, Inc.
    25. 25. Conclusions © 2009 Real-Time Innovations, Inc.
    26. 26. Bridged Integration Leverages Existing JMS Subsystems <ul><li>Preserves investment in JMS design, which may or may not be portable </li></ul><ul><li>Requires adapter implementations specific to broker implementation </li></ul><ul><li>Limits levels of performance and integration across DDS/JMS interface </li></ul><ul><li>May limit fault tolerance of combined system </li></ul>DDS App © 2009 Real-Time Innovations, Inc. Adapter Broker Adapter JMS App
    27. 27. JMS PSM for DDS Empowers Data-Centric Integration with Enterprise Infrastructure <ul><li>Requires JMS provider with data-centric core </li></ul><ul><li>May require data-centric application design </li></ul><ul><li>Provides maximum performance and fault tolerance </li></ul><ul><li>Provides interoperability: DDS  DDS, DDS  JMS, JMS  JMS </li></ul><ul><li>Supports high performance and tight integration with unified data model </li></ul>© 2009 Real-Time Innovations, Inc. JMS API App Servers, ESBs, etc. Data-Centric Core RTPS Real-Time Systems JMS API App Servers, ESBs, etc. Data-Centric Core RTPS Real-Time Systems
    28. 28. Information Mapping Concerns are the Same, Regardless of System Architecture <ul><li>Information Model Components </li></ul><ul><ul><li>Message/sample contents : body and properties </li></ul></ul><ul><ul><li>Instances and instance lifecycle </li></ul></ul><ul><ul><li>Message/sample destinations : topics, queues, and configuration </li></ul></ul><ul><li>Mapping Considerations </li></ul><ul><ul><li>Completeness : Are all data and meta-data needed from both APIs? </li></ul></ul><ul><ul><li>Transparency : Does DDS have to understand JMS or vice versa? </li></ul></ul><ul><ul><li>Data-centric vs. message-centric design : Do instances, keys, etc. need to be represented? </li></ul></ul>© 2009 Real-Time Innovations, Inc. http://blogs.rti.com/2009/04/30/data-transparency-why-you-should-care/
    29. 29. Needed: Standards Leadership <ul><li>QoS profiles allow configuration where JMS API is lacking </li></ul><ul><li>Extensible Topics specification will improve portability of DDS  JMS data type interoperability implementation </li></ul><ul><li>Java 5 PSM for DDS would improve portability </li></ul><ul><ul><li>Easier to create ESB, CEP adapters for multiple DDS impls. </li></ul></ul><ul><ul><li>To what extent should it be aligned with JMS? </li></ul></ul><ul><li>OMG/JCP collaboration could improve portability and integration for combined DDS + JMS implementation(s) </li></ul>© 2009 Real-Time Innovations, Inc. Done In Progress Hypothetical
    30. 30. Q & A © 2009 Real-Time Innovations, Inc.

    ×