Patterns of Data Distribution


Published on

These are slides from a talk I gave on 19 April, 2012, at the Object Management Group's (OMG's) Real-Time Workshop in Paris, France. The purpose of the talk was to describe the ways in which building applications is different from building platforms and systems, especially with respect to patterns of communication. Specifically, the recognized messaging patterns make sense at an application layer but are often too limiting and brittle within the software infrastructure itself.

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • This talk is not specific to any one middleware technology.
  • Each message delivered to all available consumers.
  • Each message is delivered to only a subset (usually one) of a set of consumers that all share a certain role.
  • Each message is delivered to one recipient, which sends a related message back to the original sender.
  • Sender has a job to do, so posts a job to a queue: Point-to-Point pattern. A receiver picks it up.The one that processes the job posts a response back; there’s a layered Request-Reply Pattern.Other consumers use different patterns: consider monitoring and recording scenarios.The queues are really topics! Does the original sender know that?
  • If you remember nothing else from this talk, remember this slide.You could substitute “portability” for “applications” and/or “interoperability” for “platforms”.
  • Patterns of Data Distribution

    1. 1. Your systems. Working as one.Patterns ofData DistributionWorkshop on Real-Time, Embedded and Rick WarrenEnterprise-Scale Time-Critical Systems Dir. Tech. Solutions, Principal EngineerApril 2012; Paris
    2. 2. System Lifecycle …Because we still1. Build cool small app. think like this2. Needs more capability! Scale out. …But mostly we3. Interop., maintenance, and screw up here reuse are hard! Refactor out platform.Repeat until you’re ready to… Mature4. Build new capabilities atop ecosystems are unmodified platform. already here © 2012 RTI 2
    3. 3. Think Different.Building Applications Building Platforms• Designed as a unit • Connect independently• Managed as a unit developed applications• …To serve a single business • Focus on interoperability and function heterogeneous versioning • …To solve business function(s) different than its constituent partsWe will discussjust one aspect here: Mostly from thisPatterns of data distribution perspective © 2012 RTI 3
    4. 4. Distribution Pattern Overview Publish- Point to Request- Subscribe Point Reply* See Hohpe and Woolf, 5 “Enterprise Integration Patterns”,
    5. 5. Pattern: Publish-Subscribe Subscriber Subscriber Publisher Topic Message … SubscriberScenario for App Use Variation Points• Distributing data • Topic structure: flat vs. hierarchical • Subscriber quorum © 2012 RTI 6
    6. 6. Pattern: Point to Point Receiver Receiver Sender Queue Message … ReceiverScenario for App Use Variation Points• Distributing computation • Allowed current consumers • Message visibility to consumers of other roles © 2012 RTI 7
    7. 7. Pattern: Request-Reply Request “Method”? Channel Message Requester Replier Reply Channel MessageScenarios for App Use Variation Points• Sending / acknowledging • # Repliers commands • # Replies per Replier – Called “Command Pattern”• Pulling data (often large) • Synch. vs. asych. • RMI vs. messaging API © 2012 RTI 8
    8. 8. Middleware ComparisonMiddleware standards hard code specific patterns.• JMS • HTTP – Publish-Subscribe – Request-Reply (data-centric) • TopicPublisher • Messaging: GET, PUT, POST, • TopicSubscriber DELETE, etc. – Point to Point • CORBA • QueueProducer – Request-Reply (imperative) • QueueConsumer/QueueBrow • RMI, user-defined ser – Point to Point (partial) – Request-Reply (idiomatic) • oneway call to predefined • Based on JMSReplyTo header recipient• DDS – Publish-Subscribe • DataWriter • DataReader © 2012 RTI 9
    9. 9. Combining Patterns Subscriber Recording Subscriber Monitoring Receiver Receiver Queue Sender Topic Request Channel … Publisher Reply Channel ReceiverRequester Replier Topic © 2012 RTI 10
    10. 10. Combining Patterns SubscriberInsight: Recording• Data is Platforms are Subscriber concerned with this Monitoring global• Patterns Receiver Applications are are local concerned with this Receiver Queue Sender Topic Request Channel … Publisher Reply Channel Receiver Requester Replier Topic © 2012 RTI 11
    11. 11. Identifying Super-Patterns• Build platform with super-patterns Super-pattern. [noun] Pattern from which all others in same category can be derived, by: 1. Adding constraints and/or 2. Composing it with itself• Build applications (1) based on derived patterns or (2) isolate logic from pattern altogether – For programming ease of use – For correctness – For manageability © 2012 RTI 12
    12. 12. This is the Super-Pattern• Someone produces some data• Some other(s) observe(s) that data Subscriber Subscriber Publisher Topic Message … Subscriber• Apply platform concerns here – Interoperability – Connectivity – Governance © 2012 RTI 13
    13. 13. We Can Constrain It• Apply delivery policy to limit observation based on Receiver role(s)• Encompasses concurrent consumers and multiple roles Receiver Receiver Sender Queue Message … Receiver• Apply application concerns here – Portability – Ease of (business logic) development © 2012 RTI 14
    14. 14. We Can Compose It• Request, Reply Channels each instance of super-pattern• Encompasses multiple Repliers, multiple Replies, additional producers and consumers Request Channel Message Requester Replier Reply Channel Message• Apply application concerns here – Portability – Ease of (business logic) development © 2012 RTI 15
    15. 15. Challenges for Platform BuildersChallenge: System ArchitectureArchitects and developers need trainingand support• Many platform developers were trained as application developers, not systems engineers• Many people think imperatively: “Do this, then do this, …” © 2012 RTI 16
    16. 16. Challenges for Platform BuildersChallenge: Performance• General, reconfigurable implementation typically has overhead vs. highly specialized one• …But beware premature optimization © 2012 RTI 17
    17. 17. Challenges for Platform BuildersChallenge: Application InterfaceOption #1: Provide Derived Option #2: Separate Apps from Patterns? Patterns?• Application developers will • Example (JMS MDB): demand many of them! void onMessage( – Pub-Sub, Point to Message event) { Point, Request-Reply // biz logic… – Others? }• Pro: Perceived as • Pro: Elegant; one thing to familiar, easy to use learn; easy to reconfigure• Con: Many ways to do same • Con: Requires inversion of thing; less reconfigurable; control => requires heavy work for vendors container, retraining © 2012 RTI 18
    18. 18. Summary• From app programming POV, certain problems feel natural to solve with certain patterns• From platform or systems POV, handling each pattern in independent ad hoc ways results in brittle, poorly performing systems• Solution: Build platform in terms of fundamental super-patterns 1. Option: Derive high-level patterns from super- patterns; expose those to apps 2. Option: Invert control; enforce patterns within platform, not apps; make app developers cope © 2012 RTI 19
    19. 19. P.S. Exercise for the ReaderFind the Super-Patterns:• Topology • Invocation – Peer-to-Peer – Event-Driven – Hub and Spoke – Imperative – etc. – etc. © 2012 RTI 20