Communication Patterns Using Data-Centric Publish/Subscribe
Upcoming SlideShare
Loading in...5
×
 

Communication Patterns Using Data-Centric Publish/Subscribe

on

  • 4,986 views

Fundamental to any distributed system are communication patterns: point-to-point, request-reply, transactional queues, and publish-subscribe. Large distributed systems often employ two or more ...

Fundamental to any distributed system are communication patterns: point-to-point, request-reply, transactional queues, and publish-subscribe. Large distributed systems often employ two or more communication patterns. Using a single middleware that supports multiple communication patterns is a very cost-effective way of developing and maintaining large distributed systems. This talk will begin with an introduction of Data Distribution Service (DDS) – an OMG standard – that supports data-centric publish-subscribe communication for real-time distributed systems. DDS separates state management and distribution from application logic and supports discoverable data models. The talk will then describe how RTI Connext Messaging goes beyond vanilla DDS and implements various communication patterns including request-reply, command-response, and guaranteed delivery. You will also learn how these patterns can be combined to create interesting variations when the underlying substrate is as powerful as DDS. We’ll also discuss APIs for creating high-performance applications using the request-reply communication pattern.

Statistics

Views

Total Views
4,986
Views on SlideShare
3,398
Embed Views
1,588

Actions

Likes
1
Downloads
97
Comments
0

9 Embeds 1,588

http://blogs.rti.com 1570
http://feeds2.feedburner.com 7
http://www.linkedin.com 5
http://translate.googleusercontent.com 1
http://feeds.feedburner.com 1
http://www.theofficialboard.com 1
http://blogs.rti.com.sixxs.org 1
http://thisninja 1
http://www.slashdocs.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Communication Patterns Using Data-Centric Publish/Subscribe Communication Patterns Using Data-Centric Publish/Subscribe Presentation Transcript

  • Sumant Tambe, Ph.D.Senior Software Research EngineerReal-Time Innovations, Inc.www.rti.com
  • Agenda• Communication Models• Data Distribution Service Standard• Data-Centricity 101 – Demo• Communication Patterns – Request/Reply – Guaranteed Delivery10/11/2012 2
  • Middleware Communication Models10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 3 View slide
  • RPC Vs. Pub-Sub/Messaging/Data-Distribution• RPC (WebServices, CORBA, DCOM) offer a remote method abstraction – Familiar OO programming model – Results in a tightly-coupled system • Forces synchronous invocations • Imposes global object model • Limited QoS(appearance of local method call) • Lack robustness: cascading points of failure – Typically built on top of TCP: • Impacts scalability and time-determinism – Best-suited to smaller, closely-coupled systems• Pub-Sub (Messaging Data-Distribution) offer a queue-based and/or replicated-data model – Subsystems are decoupled in time, space, and synchronization • Contracts established by verifying QoS compatibility – Supports a variety of transports including multicast UDP – Better suited for high-performance and real-time 10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 4 View slide
  • Queue Vs. Pub/Sub• Queue – Message sent to Queue – Multiple readers can read from the queue – Each message is delivered to ONLY one reader • Readers “affect each other” – Apps: • Job Scheduling, Load Balancing, Collaboration• Pub/Sub – Message sent to Topic – Multiple readers can subscribe to Topic with or without filters – Each message delivered to ALL subscribers that pass filter • Readers are decoupled – Apps: • Notifications, Information Distribution 10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 5
  • Pub/Sub Vs. Data-Distribution• Pub-Sub – Only messages no concept of data – Each message is interpreted without context – Messages must be delivered FIFO or according to some “priority” attribute – No Caching of data – Simple QoS: filters, durability, lifespan• Data-Distribution – Messages represent update to data-objects – Data-Objects identify by a key Data Bus – Middleware maintains state of each object – Objects are cached. Applications can read at leisure – Smart QoS • Ownership, History (per key), Deadline – Subsumes Pub-Sub 10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 6
  • DDS: Standards-based Integration Infrastructure for Critical Applications Streaming Sensors Events Data Real-Time Enterprise Actuators Applications Applications© 2009 Real-Time Innovations, Inc.
  • Systems that interact with the Real World • Must adapt to changing environment • Cannot stop processing the information • Live within world-imposed timing Beyond traditional interpretation of real-time© 2010 Real-Time Innovations, Inc.
  • Real-Time Data Distribution10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 9
  • Family of Specifications2008 2009 2010 2010 2012 2013UML Profile DDS for DDS DDS-STD-C++ DDS Security RPC over DDSfor DDS Lw CCM X-Types DDS-JAVA5 App 2004 App App DDS Spec DDS 2006 DDS DDSImplementation DDS Implementation Implementation Interoperablity Network / TCP / UDP / IP © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 10
  • Broad Adoption• Vendor independent Cross-vendor portability – API for portability – Wire protocol for interoperability• Multiple implementations – 10 of API DDS API – 8 support RTPS• Heterogeneous Middleware – C, C++, Java, .NET (C#, C++/CLI) – Linux, Windows, VxWorks, other DDS Real-Time embedded & realtime Publish-Subscribe Wire Protocol (RTPS)• Loosely coupled Cross-vendor interoperability© 2009 Real-Time Innovations, Inc. 11
  • DDS adopted by key programs• European Air Traffic Control – DDS used to interoperate ATC centers• UK Generic Vehicle Architecture – Mandates DDS for vehicle comm. – Mandates DDS-RTPS for interop.• DISR (DoD Information Technology Standards Registry) – Mandates DDS for Pub-Sub API and Interop.• US Navy Open Architecture – Mandates DDS for Pub-Sub• SPAWAR NESI – Mandates DDS for Pub-Sub SOA © 2010 Real-Time Innovations, Inc.
  • Evolution of DataBusData-centricity basics
  • Everyday Example: Schedule Meeting via Emails Alternative Process #1 (message-centric): 1. Email: “Meeting Monday at 10:00.” 2. Email: “Here’s dial-in info for meeting…” 3. Email: “Meeting moved to Tuesday” 4. You: “Where do I have to be? When?” 5. You: (sifting through email messages…) 14
  • Everyday Example: Schedule Meeting Using a Calendar Alternative Process #2: 1. Calendar: (add meeting Monday at 10:00) 2. Calendar: (add dial-in info) 3. Calendar: (move meeting to Tuesday) 4. You: “Where do I have to be? When?” 5. You: (check calendar. Contains consolidated-state)The difference is state!The infrastructure consolidates changes and maintains it 15
  • What’s the Difference? State.• Objects have identity and “State” (“data”) is a attributes – The meeting will run 1:00– snapshot of those 2:00 in the conference room. attributes and – My friend’s phone number is characteristics. 555-1234 his email is… – The car is blue and is traveling north from Sunnyvale at 65 mph. If the infrastructure• …whether they exist in maintains the state, the real world, in the the application does computer, or both not need to re-• …whether or not we construct it… observe or acknowledge them
  • Why is it better to have the (data-centric) middleware manage the state?• Reconstructing the state of an object is hard – Must infer based on all previous messages – Maintaining all these messages is expensive – Each app makes these inferences  duplicate effort• Reconstructing state is not robust – Many copies of state  may be different  bugs vs. Uniform operations on state  fewer bugs – Any missing change compromises integrity• State awareness results in better performance – Middleware can be smart about what to send and when• Data-type awareness simplifies programming – Middleware supports direct definition and instantiation of the data- model
  • DDS Communication Model10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 18
  • Demo: Publish-Subscribe© 2009 Real-Time Innovations, Inc.
  • DDS Real-Time Quality of Service (QoS) Attributes QoS Policy QoS Policy DURABILITY USER DATA User QoS HISTORY TOPIC DATAVolatility READER DATA LIFECYCLE GROUP DATA WRITER DATA LIFECYCLE PARTITION Presentation LIFESPAN PRESENTATION Infrastructure ENTITY FACTORY DESTINATION ORDER RESOURCE LIMITS OWNERSHIP Redundancy RELIABILITY OWNERSHIP STRENGTH Delivery TIME BASED FILTER LIVELINESS Transport DEADLINE LATENCY BUDGET CONTENT FILTERS TRANSPORT PRIORITY
  • Communication PatternsBeyond DDS10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 21
  • Why Patterns• From app programming POV – Patterns solve specific problems nicely• From platform or systems POV – Handling each pattern in independent ad hoc ways results in brittle, poorly performing systems• Solution: Build platform in terms of fundamental patterns – Derive high-level patterns from basic patterns; expose those to apps © 2012 RTI 22
  • Communication PatternRequest/Reply
  • Introduction to Request/Reply• Publish/Subscribe (Pub/Sub) – Fundamental pattern in RTI Connext – Pub/Sub shines where there is • Push model • Unidirectional stream of data • One-to-many (multicast)• Request/Reply – Client-server model built on top of pub/sub – Request/Reply shines where there is • Pull model • Bidirectional communication • A need for remote code execution – E.g., Get/set configuration, send commands10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 24
  • Request-Reply Request Request TopicRequestor (Foo) Replier Reply Topic Reply (Bar) © 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
  • Correlation Correlation Message ID 1 3 2 1 Requests Replier 1 1 2 3Requestor Replies Correlation ID © 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
  • How Does Correlation Work Requester App Replier App sample_id1 Data VGUID1 VSN1 Foo Data Foo Write Writer Reader (VGUID1) Part of Foo ProcessingSampleInfo VSN1 VGUID1 Bar Read Data Data Bar Reader Writer VGUID2 VSN2 VGUID1 VSN1 Bar (VGUID2) sample_id-id2 related_sample_id1 Index Condition “@related_sample_id.VSN == VSN1” Content-based Filter “@related_sample_id.VGUID == VGUID1” Filtering Implementation details10/11/2012 © 2012 RTI RTI • COMPANY CONFIDENTIAL © 2012 • COMPANY CONFIDENTIAL 27 27
  • Single-Request Multiple-Reply RequestRequestor Replier 1 2 3 Replies Sequence ID © 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
  • Multiple Repliers Reply Replier ARequester Replier B Request Replier C © 2011 Real-Time Innovations, Inc.
  • Replier “Hello, World!” (Server-side)using namespace connext;DDS::DomainParticipantFactory factory = DDS::DomainParticipantFactory::get_instance();DDS::DomainParticipant * participant = factory->create_participant(domain_id, DDS::PARTICIPANT_QOS_DEFAULT, NULL, DDS::STATUS_MASK_NONE);std::auto_otr<Replier<Foo, Bar> > replier (new Replier<Foo, Bar>(participant, "TestService"));Sample<Foo> request;// Receive one requestbool received = replier->receive_request(request, MAX_WAIT);if (!received) { std::cout << "Requests not received" << std::endl; return false;}// A WriteSample automatically initializes and finalizes// the data using the TypeSupport (in this case BarTypeSupport)WriteSample<Bar> reply;if (request.info().valid_data) { sprintf(reply.data().message, "Reply for %s", request.data().message); // Send a reply for that request replier->send_reply(reply, request.identity()); // Note: a replier can send more than one reply for the same request}10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 30
  • Requester “Hello, World!” (Client-side) using namespace connext; // Create a DomainParticipant DDS::DomainParticipant * participant = ...; // Create a Requester std::auto_ptr<Requester<Foo, Bar> > requester (new Requester<Foo, Bar>(participant, "TestService")); // Send request WriteSample<Foo> request; strcpy(request.data().message, "A Request"); requester->send_request(request); if (requester->wait_for_replies(2, MAX_WAIT, request.identity())) { LoanedSamples<Bar> replies = requester->take_replies(request.identity()); // If there are no replies, the container is empty for(LoanedSamples<Bar>::const_iterator it = replies.begin(); it != replies.end(); ++it) { if (it->info().valid_data) { std::cout << "Received reply: " << it->data().message << std::endl; } } }10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 31
  • Implementing Request/Reply Pattern Steps With Vanilla DDS API With BIGPINE API Define request and reply types Manual Manual Create Participant Manual Manual Pollute application data model with a Manual Not necessary correlation-id Create publisher/subscriber Manual Automatic Create datareader/datawriter Manual Automatic Register request/reply types Manual Automatic Create request/reply topics Manual Automatic Setup CFT to filter unwanted replies Manual Automatic Correlation of request/reply samples Manual Semi-Automatic Memory management of samples Manual Automatic Lifecycle management of entities Manual Automatic Language friendly (idiomatic API) Partially Yes Scalable Depends on impl Yes10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 32
  • Salient Features of Connext Request/Reply• New API, built on top of DDS• Data-centric• Scalable• Simple to use, abstracts many DDS details• Idiomatic, language-friendly API (C++, Java, and C#)• It’s not RPC, though can be the foundation for an RPC solution 10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 33
  • Scalable Request-Reply Requester 1 Reply 1 Replier Requester 2 Replies Reply 2 …Thousands of clients! Requester N Reply N © 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
  • Combining Patterns Subscriber Wire Tap Request Request TopicRequestor Replier Reply Topic Reply Subscriber © 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
  • Communication PatternGuaranteed Delivery
  • The Guaranteed Delivery Problem• Delivery of samples from a Producer to the Intended Consumers such that the delivery is robust to multiple kind of failures• Guaranteed delivery does not imply complete certainty  Continuum of Delivery Guarantees © 2012 RTI • COMPANY CONFIDENTIAL
  • The Guaranteed Delivery Scenario • Producer ‘P’ publish a Sample ‘S’ that should be received by Consumer ‘C’ Consumer process failure Producer process failure (the process crashes after the sample is(the producer process crashes after the received by the middleware and before it is sample has been written) processed by the application) Sample ‘S’ Producer ‘P’ Consumer ‘C’ (DataWriter) (DataReader) Network and transport failures (packets are lost due to temporary network failures or Late joiner consumer transport/middleware buffer overflows) (the consumer is not running when the sample is written) © 2012 RTI • COMPANY CONFIDENTIAL
  • Guaranteed Delivery App-level ackPublisher Subscriber Message Message Durable Subscriber Message Message Disk © 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
  • DDS Durability QoS PrimerDDS Durability controls data availability with respect tolate joining datareaders• VOLATILE – No need to keep data samples for late joiners• TRANSIENT_LOCAL – Data instances available for late joiners – Data does not survive beyond datawriter• TRANSIENT – Data is not tied to the lifecycle of the datawriter. – A separate service (persistence service) keep data in memory• PERSISTENT – Data is kept on permanent storage, so that they can outlive a system session 10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 40
  • Beyond DDS: Required Subscriptions• DataWriter can be configured with: – a list of required subscriptions • Required subscriptions are named subscriptions identified by a role name• DataWriter must store a sample until both: – Acknowledged by all active reliable readers – Acknowledged by all required subscriptions• DataReader identifies itself: – As serving a required subscription – Uniquely via a virtual GUID
  • Beyond DDS: Durable Subscriptions• Concept – It is a required subscription with durability >= TRANSIENT – Features that allows to keep data until received by durable subscriptions for which readers may or may not exist at any given time – Data is not lost even if the DataWriter crashes• Benefits and Use Cases • In combination with other features durable subscriptions provides guaranteed messaging• Requires persistence service – To persist data – To persist existence of durable subscriptions and their stats
  • Beyond DDS: Collaborative DataWriters• Concept: – Multiple DataWriters can relay samples from a common stream (data source). The DataReaders reconstruct the original stream and deliver to the application the complete set of samples• Benefits & Use cases: – High availability and fault tolerance • Multiple DataWriters publishing the same samples – Load balancing and data partition • Multiple DataWriters publishing different sets of samples from the same source – Scalability • Partition across different machines and cores
  • Delivery Models: Best-Effort to Guaranteed!Use Case Features in RTI Connext Messaging 5.0.0Periodic data from live producers Best effort reliabilityEvent data from live producers Reliable reliabilityLast value cache from live producers (entire Transient local durability +state of the datawriter cache) Reliable reliabilityDurable last value cache (datawriter may Persistent durability +come and go) Reliable reliability + Persistence ServiceGuaranteed messaging from live producers Transient local durability + Reliable reliability + Required subscriptions + Application level ACKDurable Guaranteed messaging Persistent durability + Reliable reliability + Collaborative DataWriters + Persistence Service + Durable subscriptions + Application level ACK © 2012 RTI • COMPANY CONFIDENTIAL
  • References• Real-Time Innovations, Inc. – www.rti.com• RTI Connext DDS – www.rti.com/products/dds• RTI Connext Request-Reply API – Just Google!• OMG DDS Specifications – www.omg.org/technology/documents/dds_spec_catalog.htm 10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 45
  • Thank you © 2012 RTI • COMPANY CONFIDENTIAL