Communication Patterns Using Data-Centric Publish/Subscribe

1. Sumant Tambe, Ph.D.Senior Software Research EngineerReal-Time Innovations, Inc.www.rti.com

2. Agenda• Communication Models• Data Distribution Service Standard• Data-Centricity 101 – Demo• Communication Patterns – Request/Reply – Guaranteed Delivery10/11/2012 2

4. RPC Vs. Pub-Sub/Messaging/Data-Distribution• RPC (WebServices, CORBA, DCOM) offer a remote method abstraction – Familiar OO programming model – Results in a tightly-coupled system • Forces synchronous invocations • Imposes global object model • Limited QoS(appearance of local method call) • Lack robustness: cascading points of failure – Typically built on top of TCP: • Impacts scalability and time-determinism – Best-suited to smaller, closely-coupled systems• Pub-Sub (Messaging Data-Distribution) offer a queue-based and/or replicated-data model – Subsystems are decoupled in time, space, and synchronization • Contracts established by verifying QoS compatibility – Supports a variety of transports including multicast UDP – Better suited for high-performance and real-time 10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 4

5. Queue Vs. Pub/Sub• Queue – Message sent to Queue – Multiple readers can read from the queue – Each message is delivered to ONLY one reader • Readers “affect each other” – Apps: • Job Scheduling, Load Balancing, Collaboration• Pub/Sub – Message sent to Topic – Multiple readers can subscribe to Topic with or without filters – Each message delivered to ALL subscribers that pass filter • Readers are decoupled – Apps: • Notifications, Information Distribution 10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 5

6. Pub/Sub Vs. Data-Distribution• Pub-Sub – Only messages no concept of data – Each message is interpreted without context – Messages must be delivered FIFO or according to some “priority” attribute – No Caching of data – Simple QoS: filters, durability, lifespan• Data-Distribution – Messages represent update to data-objects – Data-Objects identify by a key Data Bus – Middleware maintains state of each object – Objects are cached. Applications can read at leisure – Smart QoS • Ownership, History (per key), Deadline – Subsumes Pub-Sub 10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 6

8. Systems that interact with the Real World • Must adapt to changing environment • Cannot stop processing the information • Live within world-imposed timing Beyond traditional interpretation of real-time© 2010 Real-Time Innovations, Inc.

10. Family of Specifications2008 2009 2010 2010 2012 2013UML Profile DDS for DDS DDS-STD-C++ DDS Security RPC over DDSfor DDS Lw CCM X-Types DDS-JAVA5 App 2004 App App DDS Spec DDS 2006 DDS DDSImplementation DDS Implementation Implementation Interoperablity Network / TCP / UDP / IP © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 10

11. Broad Adoption• Vendor independent Cross-vendor portability – API for portability – Wire protocol for interoperability• Multiple implementations – 10 of API DDS API – 8 support RTPS• Heterogeneous Middleware – C, C++, Java, .NET (C#, C++/CLI) – Linux, Windows, VxWorks, other DDS Real-Time embedded & realtime Publish-Subscribe Wire Protocol (RTPS)• Loosely coupled Cross-vendor interoperability© 2009 Real-Time Innovations, Inc. 11

12. DDS adopted by key programs• European Air Traffic Control – DDS used to interoperate ATC centers• UK Generic Vehicle Architecture – Mandates DDS for vehicle comm. – Mandates DDS-RTPS for interop.• DISR (DoD Information Technology Standards Registry) – Mandates DDS for Pub-Sub API and Interop.• US Navy Open Architecture – Mandates DDS for Pub-Sub• SPAWAR NESI – Mandates DDS for Pub-Sub SOA © 2010 Real-Time Innovations, Inc.

13. Evolution of DataBusData-centricity basics

14. Everyday Example: Schedule Meeting via Emails Alternative Process #1 (message-centric): 1. Email: “Meeting Monday at 10:00.” 2. Email: “Here’s dial-in info for meeting…” 3. Email: “Meeting moved to Tuesday” 4. You: “Where do I have to be? When?” 5. You: (sifting through email messages…) 14

15. Everyday Example: Schedule Meeting Using a Calendar Alternative Process #2: 1. Calendar: (add meeting Monday at 10:00) 2. Calendar: (add dial-in info) 3. Calendar: (move meeting to Tuesday) 4. You: “Where do I have to be? When?” 5. You: (check calendar. Contains consolidated-state)The difference is state!The infrastructure consolidates changes and maintains it 15

16. What’s the Difference? State.• Objects have identity and “State” (“data”) is a attributes – The meeting will run 1:00– snapshot of those 2:00 in the conference room. attributes and – My friend’s phone number is characteristics. 555-1234 his email is… – The car is blue and is traveling north from Sunnyvale at 65 mph. If the infrastructure• …whether they exist in maintains the state, the real world, in the the application does computer, or both not need to re-• …whether or not we construct it… observe or acknowledge them

17. Why is it better to have the (data-centric) middleware manage the state?• Reconstructing the state of an object is hard – Must infer based on all previous messages – Maintaining all these messages is expensive – Each app makes these inferences  duplicate effort• Reconstructing state is not robust – Many copies of state  may be different  bugs vs. Uniform operations on state  fewer bugs – Any missing change compromises integrity• State awareness results in better performance – Middleware can be smart about what to send and when• Data-type awareness simplifies programming – Middleware supports direct definition and instantiation of the data- model

20. DDS Real-Time Quality of Service (QoS) Attributes QoS Policy QoS Policy DURABILITY USER DATA User QoS HISTORY TOPIC DATAVolatility READER DATA LIFECYCLE GROUP DATA WRITER DATA LIFECYCLE PARTITION Presentation LIFESPAN PRESENTATION Infrastructure ENTITY FACTORY DESTINATION ORDER RESOURCE LIMITS OWNERSHIP Redundancy RELIABILITY OWNERSHIP STRENGTH Delivery TIME BASED FILTER LIVELINESS Transport DEADLINE LATENCY BUDGET CONTENT FILTERS TRANSPORT PRIORITY

22. Why Patterns• From app programming POV – Patterns solve specific problems nicely• From platform or systems POV – Handling each pattern in independent ad hoc ways results in brittle, poorly performing systems• Solution: Build platform in terms of fundamental patterns – Derive high-level patterns from basic patterns; expose those to apps © 2012 RTI 22

23. Communication PatternRequest/Reply

24. Introduction to Request/Reply• Publish/Subscribe (Pub/Sub) – Fundamental pattern in RTI Connext – Pub/Sub shines where there is • Push model • Unidirectional stream of data • One-to-many (multicast)• Request/Reply – Client-server model built on top of pub/sub – Request/Reply shines where there is • Pull model • Bidirectional communication • A need for remote code execution – E.g., Get/set configuration, send commands10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 24

27. How Does Correlation Work Requester App Replier App sample_id1 Data VGUID1 VSN1 Foo Data Foo Write Writer Reader (VGUID1) Part of Foo ProcessingSampleInfo VSN1 VGUID1 Bar Read Data Data Bar Reader Writer VGUID2 VSN2 VGUID1 VSN1 Bar (VGUID2) sample_id-id2 related_sample_id1 Index Condition “@related_sample_id.VSN == VSN1” Content-based Filter “@related_sample_id.VGUID == VGUID1” Filtering Implementation details10/11/2012 © 2012 RTI RTI • COMPANY CONFIDENTIAL © 2012 • COMPANY CONFIDENTIAL 27 27

30. Replier “Hello, World!” (Server-side)using namespace connext;DDS::DomainParticipantFactory factory = DDS::DomainParticipantFactory::get_instance();DDS::DomainParticipant * participant = factory->create_participant(domain_id, DDS::PARTICIPANT_QOS_DEFAULT, NULL, DDS::STATUS_MASK_NONE);std::auto_otr<Replier<Foo, Bar> > replier (new Replier<Foo, Bar>(participant, "TestService"));Sample<Foo> request;// Receive one requestbool received = replier->receive_request(request, MAX_WAIT);if (!received) { std::cout << "Requests not received" << std::endl; return false;}// A WriteSample automatically initializes and finalizes// the data using the TypeSupport (in this case BarTypeSupport)WriteSample<Bar> reply;if (request.info().valid_data) { sprintf(reply.data().message, "Reply for %s", request.data().message); // Send a reply for that request replier->send_reply(reply, request.identity()); // Note: a replier can send more than one reply for the same request}10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 30

31. Requester “Hello, World!” (Client-side) using namespace connext; // Create a DomainParticipant DDS::DomainParticipant * participant = ...; // Create a Requester std::auto_ptr<Requester<Foo, Bar> > requester (new Requester<Foo, Bar>(participant, "TestService")); // Send request WriteSample<Foo> request; strcpy(request.data().message, "A Request"); requester->send_request(request); if (requester->wait_for_replies(2, MAX_WAIT, request.identity())) { LoanedSamples<Bar> replies = requester->take_replies(request.identity()); // If there are no replies, the container is empty for(LoanedSamples<Bar>::const_iterator it = replies.begin(); it != replies.end(); ++it) { if (it->info().valid_data) { std::cout << "Received reply: " << it->data().message << std::endl; } } }10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 31

32. Implementing Request/Reply Pattern Steps With Vanilla DDS API With BIGPINE API Define request and reply types Manual Manual Create Participant Manual Manual Pollute application data model with a Manual Not necessary correlation-id Create publisher/subscriber Manual Automatic Create datareader/datawriter Manual Automatic Register request/reply types Manual Automatic Create request/reply topics Manual Automatic Setup CFT to filter unwanted replies Manual Automatic Correlation of request/reply samples Manual Semi-Automatic Memory management of samples Manual Automatic Lifecycle management of entities Manual Automatic Language friendly (idiomatic API) Partially Yes Scalable Depends on impl Yes10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 32

33. Salient Features of Connext Request/Reply• New API, built on top of DDS• Data-centric• Scalable• Simple to use, abstracts many DDS details• Idiomatic, language-friendly API (C++, Java, and C#)• It’s not RPC, though can be the foundation for an RPC solution 10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 33

36. Communication PatternGuaranteed Delivery

37. The Guaranteed Delivery Problem• Delivery of samples from a Producer to the Intended Consumers such that the delivery is robust to multiple kind of failures• Guaranteed delivery does not imply complete certainty  Continuum of Delivery Guarantees © 2012 RTI • COMPANY CONFIDENTIAL

38. The Guaranteed Delivery Scenario • Producer ‘P’ publish a Sample ‘S’ that should be received by Consumer ‘C’ Consumer process failure Producer process failure (the process crashes after the sample is(the producer process crashes after the received by the middleware and before it is sample has been written) processed by the application) Sample ‘S’ Producer ‘P’ Consumer ‘C’ (DataWriter) (DataReader) Network and transport failures (packets are lost due to temporary network failures or Late joiner consumer transport/middleware buffer overflows) (the consumer is not running when the sample is written) © 2012 RTI • COMPANY CONFIDENTIAL

40. DDS Durability QoS PrimerDDS Durability controls data availability with respect tolate joining datareaders• VOLATILE – No need to keep data samples for late joiners• TRANSIENT_LOCAL – Data instances available for late joiners – Data does not survive beyond datawriter• TRANSIENT – Data is not tied to the lifecycle of the datawriter. – A separate service (persistence service) keep data in memory• PERSISTENT – Data is kept on permanent storage, so that they can outlive a system session 10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 40

41. Beyond DDS: Required Subscriptions• DataWriter can be configured with: – a list of required subscriptions • Required subscriptions are named subscriptions identified by a role name• DataWriter must store a sample until both: – Acknowledged by all active reliable readers – Acknowledged by all required subscriptions• DataReader identifies itself: – As serving a required subscription – Uniquely via a virtual GUID

42. Beyond DDS: Durable Subscriptions• Concept – It is a required subscription with durability >= TRANSIENT – Features that allows to keep data until received by durable subscriptions for which readers may or may not exist at any given time – Data is not lost even if the DataWriter crashes• Benefits and Use Cases • In combination with other features durable subscriptions provides guaranteed messaging• Requires persistence service – To persist data – To persist existence of durable subscriptions and their stats

43. Beyond DDS: Collaborative DataWriters• Concept: – Multiple DataWriters can relay samples from a common stream (data source). The DataReaders reconstruct the original stream and deliver to the application the complete set of samples• Benefits & Use cases: – High availability and fault tolerance • Multiple DataWriters publishing the same samples – Load balancing and data partition • Multiple DataWriters publishing different sets of samples from the same source – Scalability • Partition across different machines and cores

44. Delivery Models: Best-Effort to Guaranteed!Use Case Features in RTI Connext Messaging 5.0.0Periodic data from live producers Best effort reliabilityEvent data from live producers Reliable reliabilityLast value cache from live producers (entire Transient local durability +state of the datawriter cache) Reliable reliabilityDurable last value cache (datawriter may Persistent durability +come and go) Reliable reliability + Persistence ServiceGuaranteed messaging from live producers Transient local durability + Reliable reliability + Required subscriptions + Application level ACKDurable Guaranteed messaging Persistent durability + Reliable reliability + Collaborative DataWriters + Persistence Service + Durable subscriptions + Application level ACK © 2012 RTI • COMPANY CONFIDENTIAL

45. References• Real-Time Innovations, Inc. – www.rti.com• RTI Connext DDS – www.rti.com/products/dds• RTI Connext Request-Reply API – Just Google!• OMG DDS Specifications – www.omg.org/technology/documents/dds_spec_catalog.htm 10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 45