Integration Patterns for Mission Critical Systems


Published on

An increasing number of mission- and business-critical systems rely on the OMG
DDS as the technology for distributing and managing data. DDS trivializes
system integration in those cases where a shared data model exists.
Yet, integrating DDS-based systems that rely on different information models,
or DDS-based systems with systems that use other technologies is today performed
either point-to-point, or taking advantage of integration technologies such as
Enterprise Service Bus (ESB). None of these approaches is optimal for mission-
or business-critical applications because of the quadratic complexity of
point-to-point integration, and because of the inefficiency and lack of
QoS preservation and transformation of ESBs. With the trend toward Systems
of Systems and the increasing need for sharing information across multiple
systems and technologies these limitations are no longer sustainable either
from a cost or from a time-to-market perspective.

At the same time, well architected System of Systems often rely on
the same language of Integration Patters in order to deal with problems
that stem from systems asymmetry, data aggregation, correlation, sampling,
data projections, etc.

This presentation will (1) introduce a pattern language for Integration of Mission
Critical System of Systems, (2) explain for each pattern the problem it solves,
the context in which it arises and how the conflicting forces are often balanced,
and (3) an implementation of these patterns as provided by the OpenSplice Gateway
route definition DSL.

Published in: Technology, Economy & Finance
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Integration Patterns for Mission Critical Systems

  1. 1. Integration Patterns [For Mission/Business Critical System of Systems]OpenSplice DDS Angelo CORSARO, Ph.D. Chief Technology Officer OMG DDS Sig Co-Chair PrismTech
  2. 2. 18:45 CET OpenSplice DDS Hans vant Hag, OpenSplice DDS Product Today - The V6 Manager, PrismTech game changers 19:10 CET OpenSplice DDS Dr Angelo Corsaro, OpenSplice DDS CTO, Tomorrow - Whats PrismTech coming next 19:35 CET Buffet and Hands on with OpenSplice DDS Thank you for registering to attend PrismTech’s OpenSplice DDS User Conference which will be held in Paris, France on Tuesday 17 April 2012 at 20.05 CET The Community - Dr Angelo Corsaro, OpenSplice DDS CTO, 6.00pm Central European Time. Your contributions PrismTech and Hans vant Hag, OpenSplice enriching DDS Product Manager, PrismTech We very much look forward to meeting both new and experienced OpenSplice OpenSplice DDS DDS Users at the event. Please see below some logistical information for the event. 20:30 CET OpenSplice DDS OpenSplice DDS Users Copyright  2011,  PrismTech  –    All  Rights  Reserved. Event Location User Success Stories The OpenSplice DDS User Conference is free to attend and will be held at: 20:45 CET The OpenSplice All GRAND  ETOILE  D’OR  ROOM  (first level) DDS Forum – Your Hotel Concorde La Fayette - questions 3 Place du General Koenig, answered and your 75017, ideas discussed Paris,OpenSplice DDS France. 21:30 CET Finish The hotel is conveniently located close to the Champs-Elysees and the business district of La Defense. Detailed directions to the hotel via different travel modes is available from: Special Offer The OpenSplice DDS User Conference is being held in collaboration with the Event Agenda Object Management Group (OMG) Real-Time, Embedded, and Enterprise Scale Time-Critical Systems Workshop, which will run from Tuesday 17 April until The agenda for the event will be as follows: Thursday 19 April. Time Agenda Presenter PrismTech is pleased to announce that as a platinum sponsor of the Workshop, 18:00 CET Drinks and we are able to offer you a special discounted rate. Canapes Simply click on the link - - 18:15 CET The Progress of Dr Richard Soley, OMG and enter the  code  “rts”  where  it  prompts  you  to  do  so. DDS The code gives you a €295 saving off the standard conference rate as well as 18:30 CET The Success Story Keith Steele, CEO, PrismTech pricing options for individual days. that is OpenSplice DDS
  3. 3. Scope and Context This presentation will focus on Integration Patterns for systems Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ based on DDS ☐ The patterns presented address the challenges arising whenOpenSplice DDS integrating multiple DDS-based systems or when bridging across technology ☐ Finally, it is worth keeping in mind that Mission/Business- Critical systems have specific QoS requirement that should not be affected by integration!
  4. 4. Integration Challenges [1/2] ☐ Data Integration Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Enable data to flow across systems/sub-systems ☐ Data Policing Control which data is allowed to get in/out of a given system/subsystemOpenSplice DDS ☐ ☐ Impedance/QoS Adaptation ☐ Different network bandwidth ☐ Different processing power ☐ Ensure that QoS properties of parts are preserved when integrated into the whole
  5. 5. Integration Challenges [2/2] Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Technology Integration ☐ Enable data to flow across multiple formats and communication stacks ☐ SecurityOpenSplice DDS ☐ Secure information exchange across system/sub-system boundaries
  6. 6. Data Integration PatternsOpenSplice DDS
  7. 7. Data Integration Patterns [1/2] Data Integration Patterns are a synthesis of the most commonly used Copyright  2011,  PrismTech  –    All  Rights  Reserved. techniques for integrating systems that don’t share the same representation for some given entities ☐ Some common Data Integration Patterns areOpenSplice DDS ☐ Content Enricher ☐ Content Filter ☐ Type Transformation ☐ Canonical Data Model
  8. 8. Content Enricher Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Enriches a data type by adding additional information required by the target system/subsystem ☐ Example: Compute speed and enrich positionOpenSplice DDS Content Enricher struct Dynamics2D { struct CartesianCoord2D { long x; long x; long y; long y; long dx; }; long dy; System A }; System B
  9. 9. Content Filter ☐ Projects a data type by removing information not required by the Copyright  2011,  PrismTech  –    All  Rights  Reserved. target system/subsystem ☐ Example: Going from Dynamics to PositionOpenSplice DDS Content Filter struct Dynamics2D { struct CartesianCoord2D { long x; long x; long y; long y; long dx; }; long dy; System A }; System B
  10. 10. Canonical Data Model Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The Canonical Data Model Pattern is commonly used in DDS systems. In this case, data is always transformed into the representation used by the “Canonical Model”. This approach is very useful as it reduces the integration complexity Canonical Data ModelOpenSplice DDS ☐ Example: Coordinate Systems Type Transformer struct CartesianCoord2D { struct PolarCoord2D { long x; long rho; long y; float theta; }; }; System A System B
  11. 11. Impedance/QoS Adaptation PatternsOpenSplice DDS
  12. 12. Impedance/QoS Adaptation Patterns Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Impedance/QoS Adaptation Patterns provide solutions to the integration of systems/subsystems that have non-negligible difference w.r.t. their computational and/or networking capabilitiesOpenSplice DDS ☐ Commonly Used Patterns are: ☐ Impedance/QoS Mediator ☐ Rate Adaptation ☐ Compressor
  13. 13. Impedance Mediator Copyright  2011,  PrismTech  –    All  Rights  Reserved. System A (“Fast System”) System B (“Slow System”) Data Reader Data Writer Data Reader TopicD TopicDOpenSplice DDS TopicA TopicA Data TopicB TopicB Reader Data Writer TopicC TopicC ... Data Data Writer Reader
  14. 14. Impedance Mediator System A (“Fast System”) System B (“Slow System”) Data Copyright  2011,  PrismTech  –    All  Rights  Reserved. Reader Data Writer Data Reader TopicD TopicD TopicA TopicA DataOpenSplice DDS TopicB TopicB Reader Data Writer TopicC TopicC ... Data Data Writer Reader ☐ Direct integration of the two systems could lead to serious issues w.r.t. the “Fast” System as slow readers could slow down writers. This is not acceptable on a large class of Mission/Business Critical Systems
  15. 15. Impedance Mediator System A (“Fast System”) System B (“Slow System”) Data Copyright  2011,  PrismTech  –    All  Rights  Reserved. Reader Data Writer Data Reader TopicD TopicD TopicA TopicA Data MediatorOpenSplice DDS TopicB TopicB Reader Data Writer TopicC TopicC ... Data Data Writer Reader ☐ The mediator becomes the point of elasticity of the integrated system ☐ It decouples the fast from the slow system and allows for further injection of integration policies, such as, down-sampling (rate-adaptor), DDS QoS adaptation and compression
  16. 16. OpenSplice DDS Security
  17. 17. Security Patterns ☐ Security Integration Patterns provide common solutions to implement secure information exchanged across system/subsystems Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ The most commonly used security pattern in integration is the combination of the Border Security Model with Secure data TunnelingOpenSplice DDS Unsecured Environment Unsecured System A Unsecured Environment Secure Communication Environment System B System C Unsecured Environment
  18. 18. OpenSplice GatewayOpenSplice DDS
  19. 19. OpenSplice Gateway ☐ Provides DDS applications with unprecedented JMS Copyright  2011,  PrismTech  –    All  Rights  Reserved. integration capabilities by supporting high performance format and content XM ST PP RE transformation, routing and QoS mediation OpenSplice Gateway Leverages Apache Camel integration P Cu ☐ AMQ sto framework and its support for over 80OpenSplice DDS m connectors DDSI-RTPS ☐ OpenSplice Gateway is the best choice for integrating DDS-interoperable applications with proprietary as well as standard-based Supported Connectors Include: messaging technologies, such as JMS and - JMS - HTTP AMQP, as well as user applications leveraging - REST - AMQP - CometD - XMPP Web standards such as  W3C Web Services, - CFX - Hibernate REST and HTML5 WebSockets - TCP & UDP Sockets - Custom
  20. 20. Architecture OpenSplice Gateway Based on camel-ospl Copyright  2011,  PrismTech  –    All  Rights  Reserved. • DataReader & Component DataWriter • Typed data OpenSplice DDS • No native or camel-ddsi generated code Component No marshallingOpenSplice DDS • (raw data buffer) JDDSI Camel DDS Processors • DDSI demarshaller / marshaller • Data transformation • Dynamic Poll Enricher • QoS adaptations
  21. 21. Key Elements ☐ Components: Provide connectivity to a given technology Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Endpoints: Represent a source/destination of messages from/to a components ☐ Exchanges: Encapsulate data and meta coming from and end-point ☐ Routes: Define path from input endpoints to output endpointsOpenSplice DDS ☐ Processors: Allow to perform transformation to the data, its format, etc. route endpoint exchange processor
  22. 22. Components and Endpoints ☐ A component is a factory of Endpoint instances. Over 80 Copyright  2011,  PrismTech  –    All  Rights  Reserved. components are supported ☐ Endpoints are specified using URI: dds:TopicName:DomainID/TopicType?QoSOpenSplice DDS ☐ ☐ cometd://host:port/channelname ☐ jms:[topic:]destinationName ☐ [tcp|udp|vm]:host[:port] ☐ xmpp://host:port/room ☐ …
  23. 23. Routes Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ A route defines a path from an in endpoint to an out endpoint. ☐ Route can be defined in Spring, or using either the Java orOpenSplice DDS Scala DSL (I prefer the Scala DSL ;-)
  24. 24. Sample Route in Scala val route= new RouteBuilder {! Copyright  2011,  PrismTech  –    All  Rights  Reserved. “dds:Foo:1/FooType” to “dds:Bar:2/FooType” ! }! DDS Endpoint! Topic Name! DomainID! TypeName!OpenSplice DDS val route= new RouteBuilder {! “ddsi:Foo:1/FooType” to “dds:Bar:2/FooType” ! }! DDSI Endpoint! Topic Name! DomainID! TypeName!
  25. 25. Enterprise Integration Patterns ☐ EIP provide pre-specified ways of integrating Copyright  2011,  PrismTech  –    All  Rights  Reserved. endpoints Content Based Recipient List Router ☐ The most common integration patterns are supported, such as, multicast, filtering, Message A BOpenSplice DDS correlation, content-routing, load- FIlter Correlation ID balancing, etc. ☐ Multicast Ex: Splitter Aggregator val shapesRoute = new RouteBuilder {! “dds:Foo:1/FooType” ! "to (“dds:Bar:2/BarType”, “jms:topic:Foo”)! }!
  26. 26. Integrating Patterns in ActionOpenSplice DDS
  27. 27. iShapes Application ☐ To explore the features Copyright  2011,  PrismTech  –    All  Rights  Reserved. provided by the OpenSplice Gateway we’ll use the simd- cxx ishapes application ☐ Three TopicsOpenSplice DDS ☐ Circle, Square, Triangle ☐ One Type struct ShapeType { string color; long x; long y; Spotted shapes represent subscriptions long shapesize; }; Pierced shapes represent publications #pragma keylist ShapeType color
  28. 28. Data IntegrationOpenSplice DDS
  29. 29. Topic/Domain Bridging DDS 1 val in = "ddsi:Circle:0/ShapeType" Domain 0 Copyright  2011,  PrismTech  –    All  Rights  Reserved. 2 val out = "ddsi:Square:1/ShapeType" 3 Topic “Circle” 4 val shapesRoute = new RouteBuilder { 5 in to out 6 }OpenSplice DDS ☐ When using a DDSI endpoint de- marshaling is lazy. Thus in this example Topic “Square” data is never de-marshalled but simply sent-over to the target domain, with the DDS result that domain bridging is very Domain 1 efficient!
  30. 30. Type Transformation DDS Domain 0 Copyright  2011,  PrismTech  –    All  Rights  Reserved. 1 val shapeRoute0 = new RouteBuilder { 2 in unmarshal(cdrData) process ({ Topic 3 e => foo2Bar(e.getIn.getBody(classOf[Foo]))}) “Foo” 4 to out 5 }OpenSplice DDS ☐ process allows to perform arbitrary modifications to the exchange Topic content “Bar” ☐ As such it can be used to implement DDS Domain 1 arbitrary type transformations
  31. 31. OpenSplice DDS Data Policing
  32. 32. Content Router DDS 1 2 // Define endpoints val inEndpoint = "ddsi:"+ inTopic +":"+ inDomain +"/" + shapeType Domain 0 Copyright  2011,  PrismTech  –    All  Rights  Reserved. 3 val outEndpoint1 = "ddsi:"+ outTopic1 +":"+ outDomain1 +"/" + shapeType Topic 4 val outEndpoint2 = "ddsi:"+ outTopic2 +":"+ outDomain2 +"/" + shapeType 5 “Circle” 6 val cdrData = new DataFormatDefinition(new CDRDataFormat) 7 val shapesRoute = new RouteBuilder { 8 inEndpoint unmarshal(cdrData) choice { 9 when ( e => { 10 val s = e.getIn.getBody(classOf[ShapeType]) if (x >= y) if (x <= y)OpenSplice DDS 11 if (s.x > s.y) true else false 12 }) to outEndpoint1 13 14 when (e => { 15 val s = e.getIn.getBody(classOf[ShapeType]) 16 if (s.x < s.y) true else false 17 }) to outEndpoint2 Topic Topic 18 “Circle” “Circle” 19 otherwise { to (outEndpoint1, outEndpoint2) } 20 } DDS DDS 21 Domain 1 Domain 2 22 }
  33. 33. Load Balancing DDS 1 // Define endpoints Domain 0 Copyright  2011,  PrismTech  –    All  Rights  Reserved. 2 val inEndpoint = "ddsi:"+ inTopic +":"+ inDomain +"/" + shapeType 3 val outEndpoint1 = "ddsi:"+ outTopic1 +":"+ outDomain1 +"/" + shapeType Topic 4 val outEndpoint2 = "ddsi:"+ outTopic2 +":"+ outDomain2 +"/" + shapeType “Circle” 5 6 // Define a Route using the Java DSL 7 val shapesRoute = new RouteBuilder { 8 override def configure() =OpenSplice DDS 9 from(inEndpoint).sample(period, TimeUnit.MILLISECONDS) load balance 10 loadBalance() roundRobin() 11 to(outEndpoint1, outEndpoint2) 12 } Topic Topic “Circle” “Circle” DDS DDS Domain 1 Domain 2
  34. 34. Impedance QoS AdaptationOpenSplice DDS
  35. 35. Data Sampling DDS Domain 0 Topic Copyright  2011,  PrismTech  –    All  Rights  Reserved. 1 // Define endpoints “Circle” 2 val inEndpoint = "ddsi:"+ inTopic +":"+ inDomain +"/" + shapeType 3 val outEndpoint = "ddsi:"+ outTopic +":"+ outDomain +"/" + shapeType 4 5 // Define a Route using the Java DSL 1 per 500 msec 6 val shapesRoute = new RouteBuilder {OpenSplice DDS 7 override def configure() = 8 from(inEndpoint) sample(period, TimeUnit.MILLISECONDS) 9 to(outEndpoint) 10 } Topic “Circle” DDS Domain 1
  36. 36. Technology IntegrationOpenSplice DDS
  37. 37. JMS Integration DDS Domain 0 Topic Copyright  2011,  PrismTech  –    All  Rights  Reserved. “Circle” 1 val dds = "ddsi:Circle:0/ShapeType" 2 val jms = 3 "jms:topic:circle?jmsMessageType=Text&deliveryPersistent=false" 4 5 val shapeRoute = new RouteBuider { To JsonOpenSplice DDS 6 from(dds) unmarshal("cdr") marshal().json() to(jms) 7 } Topic “Circle” JMS
  38. 38. WebSocket Integration 1 // Define endpoints 2 val inEndpoint = Copyright  2011,  PrismTech  –    All  Rights  Reserved. 3 "ddsi:"+ inTopic +":"+ inDomain +"/" + shapeType 4 val outEndpoint = 5 "websocket://"+inTopic.toLowerCase + "?sendToAll=true" 6 7 // Define a Route using the Scala DSL 8 val shapesRoute = new RouteBuilder { 9 override def configure() =OpenSplice DDS 10 from(inEndpoint) unmarshal("cdr") marshal() json() to(outEndpoint) To JSON 11 } Topic “Circle” ☐ This example requires Camel 2.10 DDS Domain 0
  39. 39. OpenSplice DDS Security
  40. 40. TCP/UDP Tunneling DDS Domain 0 ☐ Integrate different DDS Domains via TCP (or UDP) Topic “Circle” tunnel ☐ Per Topic bridging ☐ Unidirectional or bidirectional GW 1 ☐ Possibly adding SSL/TLS TCPOpenSplice DDS GW 2 1 // on GW1: 2 from("ddsi:Circle:0/ShapeType") 3 to("netty:tcp://localhost:6789?sync=false"); Topic “Circle” 1 // on GW2: 2 from("netty:tcp://localhost:6789?sync=false") 3 to("ddsi:Circle:0/ShapeType"); DDS Domain 0
  41. 41. HTTP/HTTPS Tunneling DDS Domain 0 ☐ Integrate different DDS Domains via HTTP/HTTPS Topic “Circle” tunneling ☐ Per Topic bridging ☐ Unidirectional or bidirectional GW 1 HTTPOpenSplice DDS 1 // on GTW1 2 from("ddsi:Circle:0/ShapeType") GW 2 3 unmarshal("cdr") 4 marshal().json() 5 to("jetty:https://localhost:5001/circle"); Topic “Circle” 1 // on GTW1 2 3 from("jetty:https://localhost:5001/circle") unmarshal().json() DDS 4 marshal("cdr") Domain 0 5 to("ddsi:Circle:0/ShapeType");
  42. 42. Concluding RemarksOpenSplice DDS
  43. 43. Concluding Remarks The OpenSplice Gateway, through its tight integration with Copyright  2011,  PrismTech  –    All  Rights  Reserved. ☐ Apache Camel, provides the best solution for integrating DDS- based Systems with just about anything elseOpenSplice DDS ☐ The OpenSplice Gateway architecture minimizes the integration overhead by being lazy -- in other terms, only performing work when this is necessary for the kind of transformation required ☐ In brief the OpenSplice Gateway provides you with a rich set of Integration Patterns and great performances!
  44. 44. References Copyright  2011,  PrismTech  –    All  Rights  Reserved. OpenSplice | DDS ¥ #1 OMG DDS Implementation ¥ Integration Framework ¥ Open Source ¥ Open Source ¥ ¥ camel.apache.orgOpenSplice DDS Escalier ¥ Fastest growing JVM Language ¥ Scala API for OpenSplice DDS ¥ Open Source ¥ Open Source ¥ ¥
  45. 45. :: Connect with Us :: ¥ ¥ ¥@acorsaro ¥ ¥ ¥@prismtechOpenSplice DDS ¥ ¥ ¥ ¥