DDS: A Next-Generation Approach to Building Distributed Real-Time Systems 2011 Masterclass Gerardo Pardo-Castellote, Ph.D....
Outline <ul><li>Day 1 </li></ul><ul><ul><li>Part I </li></ul></ul><ul><ul><ul><li>Concepts </li></ul></ul></ul><ul><ul><ul...
Beyond real-time: Systems that interact directly with the Real World <ul><li>Must adapt to changing environment </li></ul>...
Challenge:  More Data, More Speed, More Sources <ul><li>TRENDS : </li></ul><ul><li>Growing Information Volume </li></ul><u...
“ Real World” Systems are integrated using a  Data-Centric Model <ul><li>Grounded on the “physics” of the problem domain <...
Data Centricity 101
Everyday Example: Calendaring <ul><li>Alternative Process #1: </li></ul><ul><li>Email: “Meeting Monday at 10:00.” </li></u...
Example: Calendaring <ul><li>Alternative Process #2: </li></ul><ul><li>Calendar: ( add meeting Monday at 10:00 ) </li></ul...
Example: FaceBook <ul><li>Alternative Process #1: </li></ul><ul><li>Use the FaceBook web site. </li></ul><ul><li>Alternati...
What’s the Difference?  State . <ul><li>Things have attributes and characteristics </li></ul><ul><ul><li>The meeting will ...
Data-Centricity = <ul><li>Describing the world </li></ul><ul><li>as it is </li></ul><ul><li>Implication :  State  of the w...
Not Data-Centricity = <ul><li>Saying anything else… </li></ul><ul><li>“ Hey you: go do this.” </li></ul><ul><li>“ The thin...
Why is it better to just describe the world? <ul><li>Reconstructing the state of the world is hard </li></ul><ul><ul><li>M...
Why is it better to just describe the world? <ul><li>Computers are stupid </li></ul><ul><ul><li>Integrate based on  what t...
So it’s “better.” Who cares? <ul><li>Faster to implement => Save time and money </li></ul><ul><li>Easier to integrate and ...
Before We Forget: the Definition <ul><li>A data-centric architecture : </li></ul><ul><li>… is based on a  data model  that...
DDS Lets You Observe a Changing World <ul><li>Other data-centric technologies: </li></ul><ul><ul><li>Databases: SQL </li><...
DDS Lets You Observe a Changing World <ul><li>DDS: </li></ul><ul><li>… allows you to observe frequent changes </li></ul><u...
DDS Lets You Observe a Changing World <ul><li>JBC-P replaced home-brew messaging w/ DDS: </li></ul><ul><li>Tracks  20x mor...
DDS Lets You Observe a Changing World <ul><li>Domain : world you’re talking about </li></ul><ul><li>Topic : group of simil...
The OMG DDS Standard
DDS: Standard Data-Centric middleware for Application Integration © 2009 Real-Time Innovations, Inc.  Data Distribution Se...
Open Architecture <ul><li>Vendor independent </li></ul><ul><ul><li>API for portability </li></ul></ul><ul><ul><li>Wire pro...
Family of Specifications © 2009 Real-Time Innovations, Inc.  COMPANY CONFIDENTIAL Network / TCP / UDP / IP Web-Enabled DDS...
DDS adopted by key programs <ul><li>US Navy Open Architecture </li></ul><ul><ul><li>Mandates DDS for Pub-Sub </li></ul></u...
RTI DDS Application Examples <ul><li>Aegis Weapon System </li></ul><ul><li>Lockheed Martin </li></ul><ul><li>Radar, weapon...
RTI DDS Application Examples <ul><li>Multi-ship simulator </li></ul><ul><li>FORCE Technology </li></ul><ul><li>Controls, s...
RTI DDS Application Examples <ul><li>Full-immer sion simulation </li></ul><ul><li>National Highway Transportation Safety A...
Architecture for the next-generation systems <ul><li>Existing technologies are reaching robustness/performance/scalability...
Benefits of the DDS approach <ul><li>Simple & Powerful Data-Centric Pub-Sub Model   </li></ul><ul><ul><li>Reduces Risk and...
DDS Global Data Space And Programming Model
Data-Centric Pub-Sub Model © 2009 Real-Time Innovations, Inc.  COMPANY CONFIDENTIAL Persistence Service Recording Service ...
Data-Centric Model <ul><li>“ Global Data Space ” generalizes Subject-Based Addressing </li></ul><ul><ul><li>Data objects a...
Data-Centric Model <ul><li>“ Global Data Space ” generalizes Subject-Based Addressing </li></ul><ul><ul><li>Data objects a...
Data-Centric Model <ul><li>“ Global Data Space ” generalizes Subject-Based Addressing </li></ul><ul><ul><li>Data objects a...
Demo: Publish-Subscribe © 2009 Real-Time Innovations, Inc.  ShapesDemo
DDS communications model <ul><li>Participants  scope the global data space (domain) </li></ul><ul><li>Topics  define the d...
Demo: Real-Time Quality of Service <ul><li>Content filter </li></ul><ul><li>Time-based filter </li></ul><ul><li>History </...
Real-Time Quality of Service (QoS) © 2009 Real-Time Innovations, Inc.  Volatility User QoS Delivery Presentation Redundanc...
Integrating components to generic middleware technology Copyright © 2010 RTI - All rights Reserved. . Data Model Custom Ma...
Integrating components to data-centric middleware technology Copyright © 2010 RTI - All rights Reserved. . Data Model Stan...
Example: Message-Centric Legacy Define message-sets / handshakes Copyright © 2010 RTI - All rights Reserved. . Publish Sub...
Example: Modern Data-Centric Design Start with Data Model / Schemas / Meaning Copyright © 2010 RTI - All rights Reserved. ...
Example: Modern Data-Centric Design Attach QoS to Data Model <ul><li>Once infrastructure understands data items, can attac...
Domain and Domain Participants Domain 1 Domain 2 Domain 3 Added Func. Multiple Domain System Using Multiple domains for Sc...
<ul><li>DDS Theoretical Foundation: </li></ul><ul><li>Observer Pattern </li></ul><ul><li>Directed State Pattern </li></ul>
Observer Pattern: Use Cases <ul><li>How do I model state that changes over time? </li></ul><ul><li>How do I decouple data ...
Observer Pattern: Introduction <ul><li>Roles </li></ul><ul><ul><li>Subject : Changes its state over time </li></ul></ul><u...
Observer Pattern: Introduction Observer ( Data Bus : Notify) Subject Update Delete Create Read
Observer Pattern: How To <ul><li>Premier pattern in DDS , central to data-centric design! </li></ul><ul><li>Roles </li></u...
Observer Pattern: How To <ul><li>Domain : world you’re talking about </li></ul><ul><ul><li>Best practice : use to isolate ...
Observer Pattern: How To <ul><li>Subject Change Contract : </li></ul><ul><ul><li>History keep last 10 </li></ul></ul><ul><...
Observer Pattern: How To <ul><li>Two variants : </li></ul><ul><li>Publish when it changes </li></ul><ul><ul><li>Uses netwo...
Observer Pattern: Benefits <ul><li>Supports dynamic system composition, evolution </li></ul><ul><ul><li>Subject states don...
Observer Pattern: Challenges <ul><li>Challenge: Large state may change in small ways </li></ul><ul><ul><li>Communicating n...
Observer Pattern: See Also <ul><li>High-Volume patterns </li></ul><ul><li>Low-Latency patterns </li></ul><ul><li>[GoF] “Ob...
<ul><li>DDS Theoretical Foundation: </li></ul><ul><li>Observer Pattern </li></ul><ul><li>Directed State Pattern </li></ul>
Directed State Pattern: Use Cases <ul><li>How can one party request that another party do something? </li></ul><ul><li>How...
Directed State Pattern: Introduction <ul><li>Best practice : keep interactions data-centric </li></ul><ul><ul><li>Insight ...
Directed State Pattern: Introduction <ul><li>3 States : </li></ul><ul><ul><li>Current </li></ul></ul><ul><ul><li>Objective...
Directed State Pattern: How To <ul><li>Reliability QoS policy : </li></ul><ul><ul><li>Current : Reliable  or  Best Effort ...
Directed State Pattern: How To <ul><li>Variant: Simple state, Fast transition </li></ul><ul><li>Current + Requested States...
Directed State Pattern: How To <ul><li>Variant: Complex state, slow transition </li></ul><ul><li>Current, Objective, Reque...
Directed State Pattern: How To <ul><li>Example: Fire Missile </li></ul>Fire Control Fire Effector Launcher = 5 State = Idl...
Directed State Pattern: Benefits <ul><li>Allows components to express intent and communicate about actions </li></ul><ul><...
Directed State Pattern: Challenges <ul><li>Beware degradation into RPC-style request-reply </li></ul><ul><ul><li>Code smel...
Directed State Pattern: Challenges <ul><li>Manage state ownership </li></ul><ul><ul><li>One party established state, anoth...
Directed State Pattern: See Also <ul><li>Patterns </li></ul><ul><ul><li>Observer pattern </li></ul></ul><ul><li>Anti-Patte...
Part I  Summary <ul><li>Real-World Systems are facing information volume, velocity, and mgmt. challenges </li></ul><ul><li...
Day 2: Exercises Gerardo Pardo-Castellote, Ph.D. Co-chair OMG DDS SIG CTO, Real-Time Innovations [email_address] <ul><li>h...
Part II <ul><li>Solving integration use-cases with RTI </li></ul><ul><ul><li>Recording, Persistence, Database Integration,...
RTI DDS builds Higher quality, Lower TCO Systems DDS  Global  Data Space Messaging  & Caching Event  Processing Database  ...
Real-Time Recording Service <ul><li>Applications: </li></ul><ul><ul><li>Future analysis and debugging </li></ul></ul><ul><...
Data Persistence <ul><li>A standalone service that persists data outside of the context of a DataWriter </li></ul>Permanen...
Ownership and High Availability <ul><li>Owner determined per Topic and Key </li></ul><ul><li>Only writer with highest stre...
Relational Database Integration Relational Actions Topic T1 I1 I2 I3 I1 I2 I3 Table T1 Publish-Subscribe Action Write() Re...
RTI Routing Service <ul><li>Selective, real-time data forwarding and transformation </li></ul><ul><li>Can Change Topic Nam...
Global Scalability: LAN to WAN… …without sacrificing Performance and Security DDS Routing Site A Site B Site C Site D WAN ...
Web Accessibility <ul><li>Direct access to real-time data from Web-Based Applications </li></ul>Tactical  Real-Time Data W...
COTS tools:  Excel  – Interacting with your data <ul><li>Display live RTI DDS Data in Excel </li></ul><ul><li>Perform real...
Part II <ul><li>Solving integration use-cases with RTI </li></ul><ul><ul><li>Recording, Persistence, Database Integration,...
Hands-on Example (C++) Type Definition MyType.idl rtiddsgen MyType.h MyTypeSupport.c MyTypePublisher.cpp MyTypeSubscriber....
Exercise #1  - Hello World <ul><li>Define you data type: </li></ul><ul><li>Create a directory “HelloWorld” </li></ul><ul><...
Run rtiddsgen (for C++) <ul><li>rtiddsgen HelloWorld.idl -language C++ -example i86Win32VS2005  </li></ul><ul><li>-replace...
Run rtiddsgen (for Java) <ul><li>rtiddsgen hello.idl -language Java -example i86Win32jdk  </li></ul><ul><li>-replace -ppDi...
Execute the program <ul><li>C++: </li></ul><ul><ul><li>On one window run: </li></ul></ul><ul><ul><ul><li>objsi86Win32VS200...
Modify the program to produce something <ul><li>C++:  Open HelloMsgPublisher.cxx  in VisualStudio </li></ul><ul><li>Java: ...
Playing with rtiddsspy <ul><li>Run rtiddsspy while the other applications are running </li></ul><ul><li>Start and stop app...
Example: Publication // Entities creation DomainParticipant participant =  TheParticipantFactory->create_participant( doma...
Example: Subscription // Entities creation Subscriber  subscriber = domain->create_subscriber(    subscriber_qos, subscrib...
How to Get Data? (Listener-Based) // Listener creation and attachment Listener listener = new MyListener(); reader->set_li...
How to Get Data? (WaitSet-Based) // Creation of condition and attachement Condition  foo_condition =  treader->create_read...
Listeners, Conditions & WaitSets <ul><li>Middleware must notify user application of relevant events: </li></ul><ul><ul><li...
Status Changes <ul><li>DDS defines  </li></ul><ul><li>A set of enumerated STATUS </li></ul><ul><li>The statuses relevant t...
Listeners, Conditions and Statuses <ul><li>A DDS Entity is associated with: </li></ul><ul><ul><li>A listener of the proper...
Listeners & Condition duality <ul><li>A StatusCondition can be selectively activated to respond to any subset of the statu...
Exercise #2 – Shapes Publisher <ul><li>Create a new directory Shapes </li></ul><ul><li>In the Directory create a file call...
rtiddsgen Details <ul><li>rtiddsgen [-d <outdir>] [-language <C|C++|Java|C++/CLI|C#>] </li></ul><ul><li>[-namespace] [-pac...
IDL vs. XML: IDL Example struct   MemberStruct{ short   sData; }  typedef  MemberStructType;   //@top-level false
IDL vs. XML: XML Example <? xml   version =&quot;1.0“ encoding =&quot;UTF-8&quot;?> < types   xmlns:xsi =&quot;http://www....
IDL vs. XSD: XSD Example <? xml   version =&quot;1.0&quot;  encoding =&quot;UTF-8&quot;?> < xsd:schema   xmlns:xsd =&quot;...
Part II <ul><li>Solving integration use-cases with RTI </li></ul><ul><ul><li>Recording, Persistence, Database Integration,...
Tools provide insight into a distributed system <ul><li>RTI Analyzer </li></ul><ul><ul><li>Understand connections and data...
Let’s try the Tools!
XML-Defined Qos Profiles: Overview <ul><li>QoS can be specified in XML file(s) and is automatically loaded when the applic...
Benefits of using QoS profiles <ul><li>Simplifies application programming.  </li></ul><ul><ul><li>Editing XML is simpler t...
More on using QoS profiles <ul><li>Could increase brittleness </li></ul><ul><ul><li>QoS files can change between runs. Wha...
What is a QoS Profile? <ul><li>Set of related Entity-QoS, usually one per entity kind, identified by a name. </li></ul><ul...
What is inside an XML QoS file? <ul><li>A collection of QoS profiles organized inside QoS Libraries </li></ul>Example <ul>...
How are XML Qos files specified? <ul><li>NDDSHOME directory  (first to be loaded) </li></ul><ul><ul><li>$NDDSHOME/resource...
Side note: URL Format <ul><li>URLs are required for either specifying QoS profile location, or for specifying a profile st...
API access to XML QoS Policies <ul><li>If XML QoS policies are used, they are automatically loaded before creating the fir...
QoS Profile Inheritance <ul><li>A QoS profile can inherit the values of another QoS profile </li></ul><ul><li>Base profile...
Modifying the default QoS using XML files <ul><li>The “default” QoS used by DDS can be modified via an explicit call to: <...
Fine-tuning which QoS applies to an Entity via Topic filters <ul><li>Allows a QoS profile to affect only selected DataRead...
XML QoS Notes <ul><li>If both XML and source code are used to configure an entity, the last one set will be used </li></ul...
Configuration of QoS via XML-defined profiles <ul><li>Try it out using the shapes demo! </li></ul>
Part II <ul><li>Solving integration use-cases with RTI </li></ul><ul><ul><li>Recording, Persistence, Database Integration,...
20X Faster than JMS / Broker-based solutions Platform: Linux 2.6 on AMD Athlon, Dual core, 2.2 GHz RTI DDS is about 20X fa...
Extremely low latency and jitter © 2009 Real-Time Innovations, Inc.  COMPANY CONFIDENTIAL Reliable, ordered delivery over ...
Orders of magnitude more scalable than broker-based solutions <ul><li>Going from 1 to 888 subscribers of the same data has...
Realizing Performance & Scalability <ul><li>RTI DDS operates peer-to-peer, without brokers </li></ul><ul><li>RTI DDS uses ...
Realizing Performance & Scalability <ul><li>RTI DDS completely isolates each process and communicates peer-to-peer </li></...
Advanced Scalability & Performance Techniques <ul><li>Latency and Priority Aware message batching </li></ul><ul><li>Conten...
Message Batching write() sender receiver write() sender Send queue Receive queue Send queue Receive queue Without batching...
Reliability with Batching <ul><li>Reliability must work even when messages are batched </li></ul><ul><li>ACK or NACK of in...
Batching has a huge payoff! Intel Core2Duo Single-CPU Dual-Core 2.4GHz, 4MB cache 32-bit CentOS 5 (RHEL 5), 2GB memory, In...
Classic (TCP Style) Reliable Protocol No packet loss situation Company Confidential 01 02 03 04 01 02 03 04, HB 01 02 03 0...
Classic (TCP Style) Reliable Protocol with some packet loss 01 02 03 04 01 02 03 04, HB 01 02 X ACK 1-2, NACK 3 05 06 07 0...
RTI DDS Reliability (Reader Cache + SACK) improves performance when packet loss occurs 01 02 03 04 01 02 03 04, HB 01 02 X...
RTI DDS NACK-only reliability eliminates ACK traffic if there no packet loss 01 02 03 04 01 02 03 04, HB 01 02 03 04 05 06...
RTI DDS NACK-only reliability greatly reduces traffic even with packet loss 01 02 03 04 01 02 03 04, HB 01 02 X 04 NACK 3 ...
Part II <ul><li>Solving integration use-cases with RTI </li></ul><ul><ul><li>Recording, Persistence, Database Integration,...
OMG Standards update <ul><li>DDS API - 2003 </li></ul><ul><li>DDS Wire Interoperability - 2006 </li></ul><ul><li>UML Profi...
Web-Protocol Access to DDS Global Data Space <ul><li>Stateless access of data via  standard Web Technologies & Protocols <...
DDS Security RFP <ul><li>A  PIM and language PSMs for set of “plugins” that provide Authentication, Authorization, Data Ta...
Pub/Sub middleware (DDS) Application Secure OS Secure  Transport DDS Application Secure DDS Global Data Credential Mgmt Au...
App. Other  DDS System Secure DDS  middleware Authentication Plugin Access Control Plugin Data Encryption Plugin Data Tagg...
Part III <ul><li>Application development cycle </li></ul><ul><li>Advanced Exercise </li></ul>
Exercise: How could “chat rooms” be implemented? <ul><li>Different Topics for each Chat room? </li></ul><ul><li>Map to Par...
Exercise: How could we implement Ground control stations that monitor UAVs <ul><li>Different Topics for each UAV? </li></u...
Edit the chat_publisher.cxx <ul><li>Go to the line with comment:  /* Main loop */ </li></ul><ul><ul><li>Add the line: </li...
Exercise #3 – Chat Room <ul><li>Create a new directory Chat </li></ul><ul><li>In the Directory create a file called chat.i...
Edit the chat_publisher.cxx <ul><li>Go to the line with comment:  /* Main loop */ </li></ul><ul><ul><li>Add the line: </li...
Exercise #3 (continued) Use Qos <ul><li>Set RELIABILITY </li></ul><ul><li>Set HISTORY to KEEP_LAST or KEEP_ALL </li></ul><...
Exercise #4 (continued) Use content filters <ul><li>Edit the chat_subscriber.cxx </li></ul><ul><li>Add the lines: </li></u...
Exercise #3 (continued)  Use Exclusive Ownership <ul><li>Set up in pairs edit the chat_publisher.cxx and  use the same “na...
Exercise <ul><li>Record the data </li></ul>
Exercise <ul><li>Use durable data with a persistence service </li></ul>
Exercise <ul><li>Isolate domains and transform data using the routing service </li></ul>
Summary <ul><li>Reduces software lifecycle costs </li></ul><ul><ul><li>Loose coupling </li></ul></ul><ul><ul><li>Replaces ...
Upcoming SlideShare
Loading in...5
×

RTI Data-Distribution Service (DDS) Master Class 2011

5,807

Published on

An in-depth (110 slides) tutorial on DDS and the RTI Data-Distribution Service

Updated for 2011

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,807
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
385
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • Time: 45 minutes Title: The Data Distribution Service (DDS) Standard: A Next-Generation Approach to Building Distributed Real-Time Systems Abstract: DDS has been adopted worldwide by major air force, army, marine and navy programs as an open architecture standard for integrating real-time tactical systems with each other and with enterprise applications such as command and control systems. This breakout will introduce the DDS standard and show how it provides a net-centric, service-oriented approach to meeting the messaging and integration requirements of mission-critical embedded systems. ** Booth demo ** RTI will be demonstrating its real-time publish/subscribe middleware based on the Data Distribution Service (DDS) standard. DDS dramatically reduces software lifecycle costs by making it easy to develop, integrate and scale distributed real-time applications. DDS applications are loosely coupled and can communicate seamlessly across platforms, programming languages and network transports (including shared memory, backplane, LAN, WAN, wireless and satellite links). Supported operating systems include VxWorks, VxWorks MILS 2.0, Linux, Windows, Solaris and AIX. A small-footprint version is available for systems that require DO-178B certification.
  • Make the point that this precept has been fundamentally understood by the GVA, but the concept is empowering as a way to integrate larger systems of systems into a net-centric whole. In fact the data model of the vehicle in def-stan 23-09 can be assessed to determine what parts of its data set could and should be communicated to the wider net-centric environment. 2 nd bullet point – you can note: Otherwise how would crusty old Generals add the value they do in leading combat situations? (Joke – up to you if you use this)
  • Last slide was at one moment in time. Now, longer-term view… Example: with 12 apps, effort is order 12 vs. order 144: order of magnitude savings
  • Mostly for technical folks
  • DDS provides an infrastructure for integrating real-time applications. It also facilitates integrating real-time applications with non-real-time (enterprise) applications, such as command and control systems.
  • Work on the standard began in 2001 and version 1.0 was formally adopted in December 2004. RTI released the first commercial solution to comply with the standardized API in 2005. Implementations: RTI* PrismTech/Thales* MilSOFT* Twin Oaks* OpenDDS Gallium/Kongsberg Boeing SoSCOE *Claim support for wire protocol OCERA ORTE is RTPS only. http://www.ocera.org/download/components/WP7/orte-0.3.1.html
  • Applications that want to contribute information to the Global Data Space can declare their intent to publish the information. Applications that want to access portions of the Global Data Space can declare their intent to subscribe to the information Decoupling in several dimensions: - Space (location): Each side does not need to know the location of the other side. They publish/subscribe to the shared “global data space” - Redundancy: It is possible for the same data to be subscribe by multiple nodes, or to be written by multiple nodes. This is all managed transparently by the infrastructure. - Time: The reception of data does not need to be synchronous with the writing. A subscriber may, if so configured, receive data that was written even before the subscriber joined the network. - Platform: Applications do not have to worry about data representation, processor architecture, Operating System, or even programming on the other side. It is possible for example to publish from a real-time node using the C language and subscribe from a Linux node running Java. Each side is isolated from the details of the other. Mechanisms in place to allow access to data only by specific applications / nodes.
  • Applications that want to contribute information to the Global Data Space can declare their intent to publish the information. Applications that want to access portions of the Global Data Space can declare their intent to subscribe to the information Decoupling in several dimensions: - Space (location): Each side does not need to know the location of the other side. They publish/subscribe to the shared “global data space” - Redundancy: It is possible for the same data to be subscribe by multiple nodes, or to be written by multiple nodes. This is all managed transparently by the infrastructure. - Time: The reception of data does not need to be synchronous with the writing. A subscriber may, if so configured, receive data that was written even before the subscriber joined the network. - Platform: Applications do not have to worry about data representation, processor architecture, Operating System, or even programming on the other side. It is possible for example to publish from a real-time node using the C language and subscribe from a Linux node running Java. Each side is isolated from the details of the other. Mechanisms in place to allow access to data only by specific applications / nodes.
  • Applications that want to contribute information to the Global Data Space can declare their intent to publish the information. Applications that want to access portions of the Global Data Space can declare their intent to subscribe to the information Decoupling in several dimensions: - Space (location): Each side does not need to know the location of the other side. They publish/subscribe to the shared “global data space” - Redundancy: It is possible for the same data to be subscribe by multiple nodes, or to be written by multiple nodes. This is all managed transparently by the infrastructure. - Time: The reception of data does not need to be synchronous with the writing. A subscriber may, if so configured, receive data that was written even before the subscriber joined the network. - Platform: Applications do not have to worry about data representation, processor architecture, Operating System, or even programming on the other side. It is possible for example to publish from a real-time node using the C language and subscribe from a Linux node running Java. Each side is isolated from the details of the other. Mechanisms in place to allow access to data only by specific applications / nodes.
  • Start first window. Publish one instance of each shape (i.e., topic). Start second window. Subscribe to all three shapes. Point out: automatic discovery, peer-to-peer communication. Illustrates one-to-one communication. Start third window. Subscribe to all three shapes. Illustrates one-to-many. Start fourth window. Publish one of each shape, using different colors than are already being published. Illustrates many-to-many. Click “Delete All” and then exit the first window. Notice how well-suited DDS is for dynamic and ad hoc systems. Applications could come and go without impacting other applications. This also provides fault tolerance. Also see how this makes it easy to insert new applications and technology into already deployed systems. Note: keep the three other windows running. Will use them for showing content filter and time-based filter later.
  • In one of the two subscribing windows, delete all of the subscriptions and then subscribe to one shape with a content-filtered topic and another shape with a time based filter. Points: Applications have fine-grained control over which data is received. This optimizes performance and reduces/simplifies application logic Filtering has no impact on either the publisher or other subscriber. It is very loosely coupled. Every application can specify its own requirements. Delete all of the publications in the publishing window and all of subscriptions in the non-filtering window. Publish a shape with Durability, Reliability, History=250 and Deadline=1000. (Publish a shape that is being subscribed with a filter.) In the window with no subscriptions, enable “Show Reliability” and subscribe to the shape being published with Durability, Reliability, History=250 and Deadline=2000. Points: Late joining applications can get the state they need to start processing, including historic data that may be necessary to “prime” algorithms. Even though the QoS of the publisher was changed, the subscriber that was running with the old QoS is still getting data. Subscribers can get data as long as the request QoS is less than or equal to the published QoS. In the subscribing window with Deadline QoS set, view the Output pane and make sure to scroll to the bottom (&lt;Control&gt;&lt;End&gt;). In the publishing window, click-hold on the shape so that it won’t be published. Note the missed deadline message. Point: Applications are notified if timing constraints are not being met so that they can take corrective action and won’t do anything erroneously.
  • The first thing to notice is that the knowledge of your data model that was associated with the data stream in the data-centric technology disappears when you use a message-centric technology. That makes it much harder to develop a generic component such as the Web Integration Service, which much transform arbitrary data types to and from XML, downsample data by based on content, etc. First message arrives. It has the same structure as we saw before, except without a known type definition, the type information must be embedded within the message itself, significantly increasing its size. The second message arrives. It’s in a totally different format than the first! This one is just a blob of binary-encoded data. Maybe the consuming application understands how to decode it and maybe not. Each application connected to the network will have expectations about the formats of the messages it receives. But a messaging infrastructure can’t support those expectations, so they have to be enforced by an organizational policy. I write up a Word document that describes how you should format your messages and email it to you, and you have to follow my instructions. If you make a mistake, we’ll have to debug it at integration time. In a data-centric approach, data type enforcement is built in: developers work with typed objects in their programming languages, errors are detected when the code is compiled before it’s ever deployed, and runtime mismatches that do occur are detected automatically by the middleware. How do I describe a content-based filter on a binary blob? How do I transform it into another format? How do I map it into a database? The third message arrives. It’s in yet a third format: a plain text string. Because the messaging system doesn’t have any concept of object lifecycle, each system has to define its own ad hoc system of sentinels: “create” messages, “dispose” messages, etc. More work, and it makes it much more difficult to leverage something you’ve built for one project on the next project. By comparison, Web Integration Service takes advantage of the built-in lifecycle support in DDS – you saw that when tracks were marked with “X” or “?”. And without any knowledge of your objects or their lifecycle, a messaging infrastructure can only support qualities of service that make sense across an entire topic: for example time-to-live (“lifespan” in language of DDS).
  • From the beginning, the data stream is associated with the schema of the data that will be propagated on that stream. Your applications already have some expectations; if you express those to a data-centric infrastructure, it can help you. For example, you can use this schema to automatically transform data into other formats. (This is how the Routing Service and Web Integration Service work.) The infrastructure can also dissect your data to filter on content (for example “give me updates where x &gt; 5”). “ Key” means “this field establishes the identity of a unique object.” Like the key in a relational database table. In DDS, can be any number of fields of any type(s). New track you’ve never seen before. Notice that since type is already known, only need to send field values, not field names or types. Update to a track you’ve already seen Another new track – notice that the key is different A track you’ve seen before has gone away
  • Effector is responsible for transition algo. In this case, x  y  z. But could have chosen “curved” path x  …  z that never reached y.
  • Time: 45 minutes Title: The Data Distribution Service (DDS) Standard: A Next-Generation Approach to Building Distributed Real-Time Systems Abstract: DDS has been adopted worldwide by major air force, army, marine and navy programs as an open architecture standard for integrating real-time tactical systems with each other and with enterprise applications such as command and control systems. This breakout will introduce the DDS standard and show how it provides a net-centric, service-oriented approach to meeting the messaging and integration requirements of mission-critical embedded systems. ** Booth demo ** RTI will be demonstrating its real-time publish/subscribe middleware based on the Data Distribution Service (DDS) standard. DDS dramatically reduces software lifecycle costs by making it easy to develop, integrate and scale distributed real-time applications. DDS applications are loosely coupled and can communicate seamlessly across platforms, programming languages and network transports (including shared memory, backplane, LAN, WAN, wireless and satellite links). Supported operating systems include VxWorks, VxWorks MILS 2.0, Linux, Windows, Solaris and AIX. A small-footprint version is available for systems that require DO-178B certification.
  • So, what is RTI Routing Service? In a nutshell, Routing Service provides high-performance real-time data forwarding and transformation across DDS Domains, Communities of Interests and Wide Area Networks (WANs) including firewall and NAT traversal. With its plug-in architecture to accommodate new transports and bridging functionality, it has been designed from the ground up to meet custom integration requirements, such as: Bridging between new DDS applications and legacy systems; and Increasing data security by controlled information flows. With the help of our Services team, custom integrations can be easily created and with low risk. My colleague Gordon will explain this further in a little while. As we’re introducing our edition-based packaging, Routing Service will be available as a component of the new RTI Enterprise Edition. Later we’ll give details about how to get access to Enterprise Edition and Routing Service at a significant discount.
  • Hands-On: show rtiddsgen –help output
  • Short, easy to write Efficient, OMG standard
  • AUTOCOMPLETION XML friendly, easy to extend More powerful
  • First step to interoperate with WS Some customers have a lot of sources in XSD or WSDL
  • This is a great tool for seeing the organization of your distributed environment. Having the ability to change QoS settings on the fly and observing how they impact the overall environment is extremely useful Having the ability to determine “WHY” a DataWriter is not talking to a specific DataReader is also very beneficial. This has the ability to show both Node views and Topic views of the system.
  • Note: only one document can be specified with the string_profile variable
  • How it is realized http directly to the data bus
  • RTI is your best partner for a successful DDS deployment: Broadly proven commercial technology &gt;11 years of commercial availability RTI is the de facto standard, with &gt;70% market share Selected for nearly 500 unique applications – many mission critical – many deployed Industry-leading expertise and services capability We know the DDS standard better than anyone We have the most experience putting DDS to use in real applications We have a large team of senior, experienced engineers at your disposal to allow you to leverage our expertise: whether for training, consulting or as a partner in your development Corporate focus and commitment All of RTI’s technology is built on DDS – we are completely committed to it – we sold off our non-DDS business Proven, financially strong DDS business DDS is not acquired or peripheral technology that we could drop or dispose of if it became financially expedient Comprehensive infrastructure built around DDS DDS is only part of your infrastructure and application – RTI provides the most comprehensive overall solution to your real-time application and data management requirements Development tools, database integration, real-time data recording, Complex Event Processing, real-time data visualization dashboards Plus the largest set of partners for additional capabilities like modeling, high-performance transports, real-time operating systems and JVMs Superior architecture and implementation Significantly higher performance: lower latency and higher throughput with less overhead on your compute resources Much more fault tolerant and highly available, with no single points of failure and full redundancy for all services Most flexibility for supporting additional transports, legacy or other non-DDS data types, and custom discovery requirements, Best suited for resource-limited embedded systems Broadest availability across enterprise and embedded platforms Quality by design Mature, formal processes for design, development and Quality Assurance Invest in comprehensive user documentation Proof point: 98% customer satisfaction - extraordinary in any industry In summary, RTI provides: Highest performance and fault tolerance – because of our superior peer-to-peer architecture Fastest time-to-market Leveraging our training, consulting, engineering services and high-quality support organization Superior documentation Most complete infrastructure Flexible implementation Lowest risk Proven commercial technology Most experience and expertise with successful DDS deployment Quality processes and support Corporate focus and commitment to DDS
  • RTI Data-Distribution Service (DDS) Master Class 2011

    1. 1. DDS: A Next-Generation Approach to Building Distributed Real-Time Systems 2011 Masterclass Gerardo Pardo-Castellote, Ph.D. Co-chair OMG DDS SIG CTO, Real-Time Innovations [email_address] http://www.rti.com
    2. 2. Outline <ul><li>Day 1 </li></ul><ul><ul><li>Part I </li></ul></ul><ul><ul><ul><li>Concepts </li></ul></ul></ul><ul><ul><ul><li>Introduction to DDS and RTI </li></ul></ul></ul><ul><ul><ul><li>Overview of Technology </li></ul></ul></ul><ul><ul><ul><li>Run-time services (Persistence, Recording, Routing, …) </li></ul></ul></ul><ul><li>Day 2 </li></ul><ul><ul><li>Part II </li></ul></ul><ul><ul><ul><li>Basic Hand’s on Examples </li></ul></ul></ul><ul><ul><ul><li>Development tools </li></ul></ul></ul><ul><ul><ul><li>Future directions and Standards: </li></ul></ul></ul><ul><ul><li>Part II </li></ul></ul><ul><ul><ul><li>Application development cycle </li></ul></ul></ul><ul><ul><ul><li>Advanced Exercise </li></ul></ul></ul>
    3. 3. Beyond real-time: Systems that interact directly with the Real World <ul><li>Must adapt to changing environment </li></ul><ul><li>Cannot stop processing the information </li></ul><ul><li>Live within world-imposed timing </li></ul><ul><li>Beyond traditional interpretation of real-time </li></ul>© 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
    4. 4. Challenge: More Data, More Speed, More Sources <ul><li>TRENDS : </li></ul><ul><li>Growing Information Volume </li></ul><ul><li>Lowering Decision Latency </li></ul><ul><li>Increasing System Availability </li></ul><ul><li>Accelerating technology insertion and deployment </li></ul><ul><li>Next-generation systems needs : </li></ul><ul><li>Performance </li></ul><ul><li>Scalability </li></ul><ul><li>Robustness & Availability </li></ul><ul><li>Platform Integration & Evolution </li></ul><ul><li>Safety-Critical Certification </li></ul><ul><li>Security </li></ul>© 2009 Real-Time Innovations, Inc.
    5. 5. “ Real World” Systems are integrated using a Data-Centric Model <ul><li>Grounded on the “physics” of the problem domain </li></ul><ul><ul><li>Tied to the nature of the sensors and real objects in the system (vehicles, device types, …) </li></ul></ul><ul><li>Provides governance across disparate teams & organizations </li></ul><ul><ul><li>Central authority can define data model necessary for interoperability </li></ul></ul><ul><li>Increased decoupling from use-cases and components </li></ul><ul><ul><li>Avoids over constraining applications </li></ul></ul><ul><li>Open, Evolvable, Platform-Independent </li></ul><ul><ul><li>The use-cases, algorithms might change between missions or versions of the system </li></ul></ul>Copyright © 2010 RTI - All rights Reserved. COMPANY CONFIDENTIAL. <ul><ul><li>Realizing this data-model requires a middleware infrastructure </li></ul></ul>App App App
    6. 6. Data Centricity 101
    7. 7. Everyday Example: Calendaring <ul><li>Alternative Process #1: </li></ul><ul><li>Email: “Meeting Monday at 10:00.” </li></ul><ul><li>Email: “Meeting moved to Tuesday.” </li></ul><ul><li>Email: “Here’s dial-in info for meeting…” </li></ul><ul><li>Rick: “Where do I have to be? When?” </li></ul><ul><li>Rick: ( sifting through email… ) </li></ul>
    8. 8. Example: Calendaring <ul><li>Alternative Process #2: </li></ul><ul><li>Calendar: ( add meeting Monday at 10:00 ) </li></ul><ul><li>Calendar: ( move meeting to Tuesday ) </li></ul><ul><li>Calendar: ( add dial-in info ) </li></ul><ul><li>Rick: “Where do I have to be? When?” </li></ul><ul><li>Rick: ( check calendar ) </li></ul>
    9. 9. Example: FaceBook <ul><li>Alternative Process #1: </li></ul><ul><li>Use the FaceBook web site. </li></ul><ul><li>Alternative Process #2 : </li></ul><ul><li>The web site doesn’t exist. </li></ul><ul><li>Deduce your friends’ information based on the notifications FaceBook sends you. </li></ul>
    10. 10. What’s the Difference? State . <ul><li>Things have attributes and characteristics </li></ul><ul><ul><li>The meeting will run 1:00–2:00 in the conference room . </li></ul></ul><ul><ul><li>My friend’s phone number is 555-1234 and he’s currently grooming his cat . </li></ul></ul><ul><ul><li>The car is blue and is traveling north from Sunnyvale at 65 mph . </li></ul></ul><ul><li>… whether they exist in the real world, in the computer, or both </li></ul><ul><li>… whether or not we observe or acknowledge them </li></ul><ul><li>“ State” (“data”) is a snapshot of those attributes and characteristics. </li></ul><ul><li>Best Practice: operate on state directly, not dialogs about state. </li></ul>
    11. 11. Data-Centricity = <ul><li>Describing the world </li></ul><ul><li>as it is </li></ul><ul><li>Implication : State of the world can be maintained by infrastructure , not each app </li></ul>the part of you care about at a certain point in time
    12. 12. Not Data-Centricity = <ul><li>Saying anything else… </li></ul><ul><li>“ Hey you: go do this.” </li></ul><ul><li>“ The thing changed in this way.” </li></ul><ul><li>Implication : State must be inferred, reconstructed, managed by each app </li></ul><ul><li>( Sometimes called “ message -centricity ” : focus on what’s said vs. what is ) </li></ul>
    13. 13. Why is it better to just describe the world? <ul><li>Reconstructing the state of the world is hard </li></ul><ul><ul><li>Must infer based on all previous messages </li></ul></ul><ul><ul><li>Maintaining all these messages is expensive </li></ul></ul><ul><ul><li>Each app makes these inferences => duplicate effort </li></ul></ul><ul><li>People make mistakes </li></ul><ul><ul><li>Many copies of state => may be different => bugs </li></ul></ul><ul><ul><li>vs. </li></ul></ul><ul><ul><li>Uniform operations on state => fewer bugs </li></ul></ul>
    14. 14. Why is it better to just describe the world? <ul><li>Computers are stupid </li></ul><ul><ul><li>Integrate based on what to do => must know everything you might do => future extension hard </li></ul></ul><ul><ul><li>Integrate based on what exists => do whatever you like; no one else cares => future extension easy </li></ul></ul><ul><li>Integration effort doesn’t scale </li></ul><ul><ul><li>Integrate at level of function => everything integrates with everything else => effort scales with square of number of apps </li></ul></ul><ul><ul><li>Integrate at level of state => everything integrates with common model => effort scales with number of apps </li></ul></ul>
    15. 15. So it’s “better.” Who cares? <ul><li>Faster to implement => Save time and money </li></ul><ul><li>Easier to integrate and update => Protect your investment </li></ul><ul><li>More reliable systems => Protect your business </li></ul>
    16. 16. Before We Forget: the Definition <ul><li>A data-centric architecture : </li></ul><ul><li>… is based on a data model that is: </li></ul><ul><ul><li>Appropriately documented— i.e. understandable by humans </li></ul></ul><ul><ul><li>Formally defined— i.e. understandable by machines </li></ul></ul><ul><ul><li>Discoverable— i.e. can be found during execution </li></ul></ul><ul><li>The data model is independent of any domain-specific functionality / application . </li></ul><ul><ul><li>i.e. made of nouns , not verbs </li></ul></ul><ul><li>The (instantiated) data model is the only authoritative source of state in the system. </li></ul><ul><li>For example, data structures in IDL file. </li></ul><ul><li>Calendar Event = </li></ul><ul><li>Start Time </li></ul><ul><li>Duration </li></ul><ul><li>Location </li></ul><ul><li>Organizer </li></ul>
    17. 17. DDS Lets You Observe a Changing World <ul><li>Other data-centric technologies: </li></ul><ul><ul><li>Databases: SQL </li></ul></ul><ul><ul><li>Web: HTTP (mostly) </li></ul></ul><ul><li>… assume the world changes slowly </li></ul><ul><li>… use network resources inefficiently </li></ul><ul><li>… are highly centralized </li></ul>App App App App App App Unreliable Failure here kills many apps State Server Slow A few updates/sec Not scalable 100 apps => 100x load
    18. 18. DDS Lets You Observe a Changing World <ul><li>DDS: </li></ul><ul><li>… allows you to observe frequent changes </li></ul><ul><li>… uses network resources efficiently </li></ul><ul><li>… is decentralized </li></ul>State: Global Data Space App App App App App App Fast 100,000’s updates/sec Scalable Load indep. # apps Reliable No single pt. failure Managed with QoS
    19. 19. DDS Lets You Observe a Changing World <ul><li>JBC-P replaced home-brew messaging w/ DDS: </li></ul><ul><li>Tracks 20x more objects with fewer failures </li></ul><ul><li>… with 97% less code ( 1.5M lines  50K ) </li></ul><ul><li>… with 99% less CPU resources ( 88 cores  0.8 ) </li></ul>State: Global Data Space App App App App App App Fast 100,000’s updates/sec Scalable Load indep. # apps Reliable No single pt. failure Managed with QoS
    20. 20. DDS Lets You Observe a Changing World <ul><li>Domain : world you’re talking about </li></ul><ul><li>Topic : group of similar things </li></ul><ul><ul><li>Similar structure (“type”) what </li></ul></ul><ul><ul><li>Similar way they change when over time (“QoS”) how </li></ul></ul><ul><li>Instance : individual thing </li></ul><ul><li>DataWriter : source of observations about part of the world (topic) </li></ul><ul><li>DataReader : observer of part of the world (topic) </li></ul>Domain (e.g. Yellowstone Park) Topic (e.g. bears in the park) Instance (e.g. Yogi the bear)
    21. 21. The OMG DDS Standard
    22. 22. DDS: Standard Data-Centric middleware for Application Integration © 2009 Real-Time Innovations, Inc. Data Distribution Service Streaming Data Sensors Events Real-Time Applications Enterprise Applications Actuators
    23. 23. Open Architecture <ul><li>Vendor independent </li></ul><ul><ul><li>API for portability </li></ul></ul><ul><ul><li>Wire protocol for interoperability </li></ul></ul><ul><li>Multiple implementations </li></ul><ul><ul><li>7 of API </li></ul></ul><ul><ul><li>4 support RTPS (+1 non-DDS) </li></ul></ul><ul><li>Heterogeneous </li></ul><ul><ul><li>C, C++, Java, .NET (C#, C++/CLI) </li></ul></ul><ul><ul><li>Linux, Windows, VxWorks, other embedded & real ­ time </li></ul></ul><ul><li>Loosely coupled </li></ul>© 2009 Real-Time Innovations, Inc. Real-Time Publish-Subscribe Wire Protocol (RTPS) DDS Middleware Library DDS API Cross-vendor portability Cross-vendor interoperability
    24. 24. Family of Specifications © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL Network / TCP / UDP / IP Web-Enabled DDS 2011 DDS Implementation App DDS Implementation App DDS Implementation DDS Spec 2004 DDS Interoperablity 2006 UML Profile for DDS 2008 DDS for Lw CCM 2009 DDS X-Types 2010 DDS Security 2011 2010 DDS-STD-C++ DDS-JAVA5 App
    25. 25. DDS adopted by key programs <ul><li>US Navy Open Architecture </li></ul><ul><ul><li>Mandates DDS for Pub-Sub </li></ul></ul><ul><li>SPAWAR NESI </li></ul><ul><ul><li>Mandates DDS for Pub-Sub SOA </li></ul></ul><ul><li>DISR </li></ul><ul><ul><li>Mandates DDS for Pub-Sub API </li></ul></ul><ul><ul><li>Mandates DDS-RTPS for Pub-Sub Interop </li></ul></ul><ul><li>European Air Traffic Control </li></ul><ul><ul><li>DDS used to interoperate ATC centers </li></ul></ul><ul><li>UK Generic Vehicle Architecture </li></ul><ul><ul><li>Mandates DDS for vehicle comm. </li></ul></ul><ul><ul><li>Mandates DDS-RTPS for interop. </li></ul></ul>
    26. 26. RTI DDS Application Examples <ul><li>Aegis Weapon System </li></ul><ul><li>Lockheed Martin </li></ul><ul><li>Radar, weapons, displays, C2 </li></ul><ul><li>B-1B Bomber </li></ul><ul><li>Boeing </li></ul><ul><li>C2, communications, weapons </li></ul><ul><li>Common Link Integration Processing (CLIP) </li></ul><ul><li>Northrop Grumman </li></ul><ul><li>Standards-compliant interface to legacy and new tactical data links </li></ul><ul><li>Air Force, Navy, B-1B and B-52 </li></ul><ul><li>ScanEagle UAV </li></ul><ul><li>Boeing </li></ul><ul><li>Sensors, ground station </li></ul><ul><li>Advanced Cockpit Ground Control Station </li></ul><ul><li>Predator and SkyWarrior UAS </li></ul><ul><li>General Atomics </li></ul><ul><li>Telemetry data, multiple workstations </li></ul><ul><li>RoboScout </li></ul><ul><li>Base10 </li></ul><ul><li>Internal data bus and link to communications center </li></ul>© 2009 Real-Time Innovations, Inc.
    27. 27. RTI DDS Application Examples <ul><li>Multi-ship simulator </li></ul><ul><li>FORCE Technology </li></ul><ul><li>Controls, simulation display </li></ul><ul><li>Mobile asset tracking </li></ul><ul><li>Wi-Tronix </li></ul><ul><li>GPS, operational status over wireless links </li></ul><ul><li>Highway traffic monitoring </li></ul><ul><li>City of Tokyo </li></ul><ul><li>Roadway sensors, roadside kiosks, control center </li></ul><ul><li>Driver safety </li></ul><ul><li>Volkswagen </li></ul><ul><li>vision systems, analysis, driver information systems </li></ul><ul><li>Medical imaging </li></ul><ul><li>NMR and MRI </li></ul><ul><li>Sensors, RF generators, user interface, control computers </li></ul><ul><li>Automated trading </li></ul><ul><li>Automated Trading Desk (ATD, now Citigroup) </li></ul><ul><li>Market data feed handlers, pricing engines, algorithmic trading applications </li></ul>© 2009 Real-Time Innovations, Inc.
    28. 28. RTI DDS Application Examples <ul><li>Full-immer sion simulation </li></ul><ul><li>National Highway Transportation Safety Authority </li></ul><ul><li>Migrated from CORBA, DCOM for performance </li></ul><ul><li>Air-Traffic Management </li></ul><ul><li>INDRA. </li></ul><ul><li>Deployed in </li></ul><ul><li>UK, Germany, Spain </li></ul><ul><li>Standards, Performance, Scalability </li></ul><ul><li>Indu strial Control </li></ul><ul><li>Schneider Electric </li></ul><ul><li>VxWorks-based PLCs </li></ul><ul><li>communicate via RTI-DDS </li></ul><ul><li>Signal Processing </li></ul><ul><li>PLATH GMBH </li></ul><ul><li>RTI supports modular programming across product line </li></ul><ul><li>Large Telescopes </li></ul><ul><li>European Southern Observatory </li></ul><ul><li>Performance & Scalability </li></ul><ul><li>1000 mirrors, 1sec loop </li></ul><ul><li>Radar Systems </li></ul><ul><li>AWACS upgrade </li></ul><ul><li>Evolvability, Mainteinability, and supportability </li></ul>© 2009 Real-Time Innovations, Inc.
    29. 29. Architecture for the next-generation systems <ul><li>Existing technologies are reaching robustness/performance/scalability limits </li></ul><ul><li>RTI DDS brings a fundamental new architecture and approach </li></ul><ul><ul><li>Fully decentralized, peer-to-peer, “no bottlenecks” architecture </li></ul></ul><ul><ul><li>Superior Wire Protocol </li></ul></ul><ul><ul><li>Powerful data-centric model </li></ul></ul><ul><ul><li>Built-in Robustness and High-Availability </li></ul></ul><ul><ul><li>Standards-based, multi-platform </li></ul></ul>Single-lane traffic No prioritization Brokers as choke-points RTI Approach
    30. 30. Benefits of the DDS approach <ul><li>Simple & Powerful Data-Centric Pub-Sub Model </li></ul><ul><ul><li>Reduces Risk and Development/Integration Time </li></ul></ul><ul><ul><li>Enhances effective performance by delivering the right data to the right place with the right QoS </li></ul></ul><ul><ul><li>Standards-based: API and Protocol </li></ul></ul><ul><li>Unsurpassed Performance and Scalability </li></ul><ul><ul><li>Priority-aware no choke-points architecture </li></ul></ul><ul><li>Builds higher quality systems and lowers TCO </li></ul><ul><ul><li>Built-in high-value capabilities </li></ul></ul><ul><ul><li>Handles Availability & other </li></ul></ul><ul><ul><li>“ hard problems” </li></ul></ul><ul><ul><li>Easy to maintain and Evolve </li></ul></ul><ul><ul><li>Leverage multicore </li></ul></ul>© 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL Messaging & Caching Event Processing Database Bridge Persistence & Durability Recording Redundancy & Failover SQL
    31. 31. DDS Global Data Space And Programming Model
    32. 32. Data-Centric Pub-Sub Model © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL Persistence Service Recording Service Essentially a virtual, decentralized global data space © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL Source (key) Latitude Longitude Altitude UAV1 37.4 -122.0 500.0 UAV2 40.7 -74.0 250.0 UAV3 50.2 -0.7 2000.0
    33. 33. Data-Centric Model <ul><li>“ Global Data Space ” generalizes Subject-Based Addressing </li></ul><ul><ul><li>Data objects addressed by DomainId, Topic and Key </li></ul></ul><ul><ul><li>Domains provide a level of isolation </li></ul></ul><ul><ul><li>Topic groups homogeneous subjects (same data-type & meaning) </li></ul></ul><ul><ul><li>Key is a generalization of subject </li></ul></ul><ul><ul><ul><li>Key can be any set of fields, not limited to a “x.y.z …” formatted string </li></ul></ul></ul>Data Writer Data Writer Data Writer Data Writer Data Reader Data Reader Data Reader Data Reader Data Writer example Data Object
    34. 34. Data-Centric Model <ul><li>“ Global Data Space ” generalizes Subject-Based Addressing </li></ul><ul><ul><li>Data objects addressed by DomainId, Topic and Key </li></ul></ul><ul><ul><li>Domains provide a level of isolation </li></ul></ul><ul><ul><li>Topic groups homogeneous subjects (same data-type & meaning) </li></ul></ul><ul><ul><li>Key is a generalization of subject </li></ul></ul><ul><ul><ul><li>Key can be any set of fields, not limited to a “x.y.z …” formatted string </li></ul></ul></ul>Data Writer Data Writer Data Writer Data Writer Data Reader Data Reader Data Reader Data Reader Data Writer example Topic
    35. 35. Data-Centric Model <ul><li>“ Global Data Space ” generalizes Subject-Based Addressing </li></ul><ul><ul><li>Data objects addressed by DomainId, Topic and Key </li></ul></ul><ul><ul><li>Domains provide a level of isolation </li></ul></ul><ul><ul><li>Topic groups homogeneous subjects (same data-type & meaning) </li></ul></ul><ul><ul><li>Key is a generalization of subject </li></ul></ul><ul><ul><ul><li>Key can be any set of fields, not limited to a “x.y.z …” formatted string </li></ul></ul></ul>Data Writer Data Writer Data Writer Data Writer Data Reader Data Reader Data Reader Data Reader Data Writer example Key (subject)
    36. 36. Demo: Publish-Subscribe © 2009 Real-Time Innovations, Inc. ShapesDemo
    37. 37. DDS communications model <ul><li>Participants scope the global data space (domain) </li></ul><ul><li>Topics define the data-objects (collections of subjects) </li></ul><ul><li>Writers publish data on Topics </li></ul><ul><li>Readers subscribe to data on Topics </li></ul><ul><li>QoS Policies are used configure the system </li></ul><ul><li>Listeners are used to notify the application of events </li></ul>Data Reader “ Alarm” Domain Participant Data Writer “ Alarm” Domain Participant New subscriber! example Listener Offered QoS Listener Got new data Requested QoS
    38. 38. Demo: Real-Time Quality of Service <ul><li>Content filter </li></ul><ul><li>Time-based filter </li></ul><ul><li>History </li></ul><ul><li>Deadline </li></ul>© 2009 Real-Time Innovations, Inc. ShapesDemo Analyzer
    39. 39. Real-Time Quality of Service (QoS) © 2009 Real-Time Innovations, Inc. Volatility User QoS Delivery Presentation Redundancy Infrastructure Transport QoS Policy DURABILITY HISTORY READER DATA LIFECYCLE WRITER DATA LIFECYCLE LIFESPAN ENTITY FACTORY RESOURCE LIMITS RELIABILITY TIME BASED FILTER DEADLINE CONTENT FILTERS QoS Policy USER DATA TOPIC DATA GROUP DATA PARTITION PRESENTATION DESTINATION ORDER OWNERSHIP OWNERSHIP STRENGTH LIVELINESS LATENCY BUDGET TRANSPORT PRIORITY
    40. 40. Integrating components to generic middleware technology Copyright © 2010 RTI - All rights Reserved. . Data Model Custom Mapping Custom Integration Akin to implementing an OO design on a Procedural Language: Requires mapping inheritance, encapsulation, exceptions, … Comp Comp Comp Comp Middleware Artifacts
    41. 41. Integrating components to data-centric middleware technology Copyright © 2010 RTI - All rights Reserved. . Data Model Standard Mapping(*) Standard API No custom mappings / code necessary Direct support for data-centric actions: create, dispose, read/take Comp Comp Comp Comp DDS Global Data Space
    42. 42. Example: Message-Centric Legacy Define message-sets / handshakes Copyright © 2010 RTI - All rights Reserved. . Publish Subscribe Component or Use-case based Schema, Limited QoS) x=float(45.6) y=float(78.9) id=“AA123” 0x0000000641410102030042366666429DC “ My app knows this means dispose.” Nothing to base filters, xforms on Error checking dev  integration Self-describing data is verbose
    43. 43. Example: Modern Data-Centric Design Start with Data Model / Schemas / Meaning Copyright © 2010 RTI - All rights Reserved. . Publish Subscribe Data Schema x : float y : float id : string ( key ) New 45.6 78.9 “ AA123” Update 56.7 89.0 “ AA123” New 65.4 32.1 “ DL987” Dispose “ AA123” X Map this into XML; rows + cols Express content-based filters Propagate data efficiently
    44. 44. Example: Modern Data-Centric Design Attach QoS to Data Model <ul><li>Once infrastructure understands data items, can attach QoS contracts to them </li></ul><ul><li>“ Keep only the latest value” or “I need updates at this rate” make no sense unless per-item </li></ul><ul><ul><li>Flight AA123 updates shouldn’t overwrite DL987, even if AA123 is updated more frequently </li></ul></ul><ul><ul><li>Update rate for one track shouldn’t change just because another track appeared </li></ul></ul>Copyright © 2010 RTI - All rights Reserved. . Publish Subscribe Data Schema x : float y : float id : string ( key ) Quality of Service Deadline Time-Based Filter History
    45. 45. Domain and Domain Participants Domain 1 Domain 2 Domain 3 Added Func. Multiple Domain System Using Multiple domains for Scalability, Modularity & Isolation demo_domain_0 demo_domain_1 Node 1 - App 1 Pub/Sub Node 2 - App 1 Subscribe Node 4 - App 1 Pub/Sub Node 4 - App 2 Publish Node 3 - App 1 Pub/Sub Node 5 - App 1 Subscribe Node 5 - App 2 Pub/Sub Node 6 - App 1 Pub/Sub
    46. 46. <ul><li>DDS Theoretical Foundation: </li></ul><ul><li>Observer Pattern </li></ul><ul><li>Directed State Pattern </li></ul>
    47. 47. Observer Pattern: Use Cases <ul><li>How do I model state that changes over time? </li></ul><ul><li>How do I decouple data consumption from data production in time? </li></ul><ul><li>What if consumers need to come and go while the system is operational? </li></ul><ul><li>What if new functionality needs to be added to the system while it’s operational? </li></ul><ul><li>Examples </li></ul><ul><ul><li>Vehicles move from place to place </li></ul></ul><ul><ul><li>Bid/offer prices of an asset change </li></ul></ul><ul><ul><li>Sensors take new measurements periodically </li></ul></ul>
    48. 48. Observer Pattern: Introduction <ul><li>Roles </li></ul><ul><ul><li>Subject : Changes its state over time </li></ul></ul><ul><ul><ul><li>Has identity, abstract “address” </li></ul></ul></ul><ul><ul><li>Observer : Sees those changes when it cares to </li></ul></ul><ul><ul><ul><li>Known to infrastructure, anonymous to subjects </li></ul></ul></ul><ul><ul><ul><li>Notified when subject changes; may examine new state or not </li></ul></ul></ul><ul><li>Operations (CRUD) </li></ul><ul><ul><li>Create : create new subject exists with certain state </li></ul></ul><ul><ul><li>Read : access state(s) of subject(s) </li></ul></ul><ul><ul><li>Update : modify existing subject with new state </li></ul></ul><ul><ul><li>Delete : delete existing subject </li></ul></ul>
    49. 49. Observer Pattern: Introduction Observer ( Data Bus : Notify) Subject Update Delete Create Read
    50. 50. Observer Pattern: How To <ul><li>Premier pattern in DDS , central to data-centric design! </li></ul><ul><li>Roles </li></ul><ul><ul><li>Subject address = domain  topic  instance </li></ul></ul><ul><ul><li>Observer = DataReader </li></ul></ul><ul><li>Operations </li></ul><ul><ul><li>Create = write new instance </li></ul></ul><ul><ul><li>Read = read or take samples </li></ul></ul><ul><ul><li>Update = write existing instance </li></ul></ul><ul><ul><li>Delete = dispose instance </li></ul></ul>
    51. 51. Observer Pattern: How To <ul><li>Domain : world you’re talking about </li></ul><ul><ul><li>Best practice : use to isolate subsystems </li></ul></ul><ul><ul><li>Best practice : connect subsystems with Routing Service </li></ul></ul><ul><li>Topic : group of similar objects </li></ul><ul><ul><li>Similar structure (type) </li></ul></ul><ul><ul><ul><li>Best practice : model complete state, not diffs </li></ul></ul></ul><ul><ul><ul><li>Best practice : use DDS type system, not opaque bytes or custom encapsulations </li></ul></ul></ul><ul><ul><li>Similar way they change over time (QoS) </li></ul></ul><ul><ul><ul><li>Best practice : differentiate </li></ul></ul></ul><ul><ul><ul><ul><li>… how subject changes (writer QoS) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>… from what observer sees (reader QoS) </li></ul></ul></ul></ul><ul><li>Instance : individual object </li></ul><ul><ul><li>Best practice : declare key fields always </li></ul></ul><ul><ul><li>Best practice : key fields establish identity </li></ul></ul><ul><ul><ul><li>e.g. Social Security Number, Track ID, etc. </li></ul></ul></ul><ul><ul><li>Best practice : key fields allow resource management per object over time </li></ul></ul><ul><ul><ul><li>e.g. “keep last 5” </li></ul></ul></ul>Topic (e.g. bears in the park) Instance (e.g. Yogi the bear) Domain (e.g. Yellowstone Park)
    52. 52. Observer Pattern: How To <ul><li>Subject Change Contract : </li></ul><ul><ul><li>History keep last 10 </li></ul></ul><ul><ul><li>Deadline 1 sec </li></ul></ul><ul><li>Observer Requirements : </li></ul><ul><ul><li>History keep last 5 </li></ul></ul><ul><ul><li>Deadline 5 sec </li></ul></ul><ul><ul><li>Minimum separation 2 sec </li></ul></ul>Example Type : typedef long BearID; struct Bear { @Key BearID which_bear; long pos_x; long pos_y; float speed; bool is_growling; bool is_charging; };
    53. 53. Observer Pattern: How To <ul><li>Two variants : </li></ul><ul><li>Publish when it changes </li></ul><ul><ul><li>Uses network efficiently when changes are slow / sporadic </li></ul></ul><ul><ul><li>Examples: </li></ul></ul><ul><ul><ul><li>e.g. DDS discovery </li></ul></ul></ul><ul><ul><ul><li>e.g. Vehicles / devices that start and stop </li></ul></ul></ul><ul><ul><li>Best practice : use durability, reliability, keep-last history depth >= 1 </li></ul></ul><ul><li>Publish periodically </li></ul><ul><ul><li>Good for continuous feedback control algorithms </li></ul></ul><ul><ul><ul><li>e.g. Vehicles / devices while they’re moving </li></ul></ul></ul><ul><ul><ul><li>e.g. Stuff that tends to be connected to Matlab / Simulink, LabView </li></ul></ul></ul><ul><ul><li>Smoothes resource use when changes are regular </li></ul></ul><ul><ul><li>Examples: </li></ul></ul><ul><ul><ul><li>Sensors, actuators in a control loop </li></ul></ul></ul><ul><ul><li>Best practice : declare finite deadline </li></ul></ul><ul><ul><li>Best practice : consider time-based filter for slow readers and best efforts </li></ul></ul>
    54. 54. Observer Pattern: Benefits <ul><li>Supports dynamic system composition, evolution </li></ul><ul><ul><li>Subject states don’t depend on previous states: observers can come, go whenever without being out of sync </li></ul></ul><ul><ul><li>Uses of data unspecified: functions can be added, removed </li></ul></ul><ul><li>Maximizes parallelism, minimizes blocking </li></ul><ul><ul><li>Subject changes on its own time scale </li></ul></ul><ul><ul><li>Observer reads on its time scale </li></ul></ul><ul><ul><li>Data bus can overwrite, filter, down-sample as necessary </li></ul></ul><ul><li>Allows flexible deployment </li></ul><ul><ul><li>Subjects don’t care about identities, locations, number of observers </li></ul></ul><ul><ul><li>Observers don’t care about physical locations of subjects; just need abstract “address” </li></ul></ul>decoupled
    55. 55. Observer Pattern: Challenges <ul><li>Challenge: Large state may change in small ways </li></ul><ul><ul><li>Communicating new state may be unnecessarily expensive </li></ul></ul><ul><ul><li>If Observer cares what changed, may be expensive to determine </li></ul></ul><ul><li>Response: Don’t sweat it </li></ul><ul><ul><li>Are the data rates high enough that it matters? </li></ul></ul><ul><ul><li>How much network/memory/CPU/complexity will you pay to “fix” it? </li></ul></ul><ul><li>Response: Send diffs and reassemble </li></ul><ul><ul><li>Tools at your disposal: </li></ul></ul><ul><ul><ul><li>DDS-XTypes: optional fields (also in RTI [dynamic] sparse types) </li></ul></ul></ul><ul><ul><ul><li>Portable today: sequence<MyType, 1> </li></ul></ul></ul><ul><ul><ul><li>Multiple topics (with same key) </li></ul></ul></ul><ul><ul><ul><ul><li>Topic 1 : complete object state </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Topic 2 : cumulative changes to last state from Topic 1 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Use durability and explicit correlation IDs to merge 1 and 2 </li></ul></ul></ul></ul><ul><ul><li>Cost : increased logic, state in your applications </li></ul></ul>
    56. 56. Observer Pattern: See Also <ul><li>High-Volume patterns </li></ul><ul><li>Low-Latency patterns </li></ul><ul><li>[GoF] “Observer” </li></ul><ul><li>[EIP] “Publish-Subscribe Channel” </li></ul><ul><li>[EIP] “Datatype Channel” </li></ul><ul><li>[EIP] “Quality-of-Service Channel” </li></ul>
    57. 57. <ul><li>DDS Theoretical Foundation: </li></ul><ul><li>Observer Pattern </li></ul><ul><li>Directed State Pattern </li></ul>
    58. 58. Directed State Pattern: Use Cases <ul><li>How can one party request that another party do something? </li></ul><ul><li>How can a producer know that observers(s) have acted on its data? </li></ul><ul><li>How can a producer observe the “result” of a remote action? </li></ul><ul><li>Examples : </li></ul><ul><ul><li>Defense: Fire control </li></ul></ul><ul><ul><li>Financial Services: Order execution </li></ul></ul>
    59. 59. Directed State Pattern: Introduction <ul><li>Best practice : keep interactions data-centric </li></ul><ul><ul><li>Insight : Objectives are state and can be distributed </li></ul></ul><ul><ul><li>Insight : Desires and intentions are state and can be distributed </li></ul></ul><ul><li>Anti-pattern : point-to-point commands, replies </li></ul><ul><ul><li>Why not : Couple applications based on function and role </li></ul></ul><ul><ul><ul><li>Why not : Constrain future functional changes to system </li></ul></ul></ul><ul><ul><ul><li>Why not : Integration effort doesn’t scale as number of apps increases </li></ul></ul></ul>
    60. 60. Directed State Pattern: Introduction <ul><li>3 States : </li></ul><ul><ul><li>Current </li></ul></ul><ul><ul><li>Objective (“goal”)— Optional </li></ul></ul><ul><ul><li>Requested State </li></ul></ul><ul><li>2+ Roles (special case of Observer pattern): </li></ul><ul><ul><li>Effector </li></ul></ul><ul><ul><ul><li>aka Subject wrt Current State and Objective State </li></ul></ul></ul><ul><ul><ul><li>aka Observer wrt Requested Objective State </li></ul></ul></ul><ul><ul><li>Requester </li></ul></ul><ul><ul><ul><li>aka Subject wrt Requested Objective </li></ul></ul></ul><ul><ul><ul><li>aka Observer wrt Current State and Objective State </li></ul></ul></ul><ul><ul><li>( Observer —with respect to any state) </li></ul></ul>Current State Objective State change Requested State Effector executes this Requester changes this
    61. 61. Directed State Pattern: How To <ul><li>Reliability QoS policy : </li></ul><ul><ul><li>Current : Reliable or Best Effort </li></ul></ul><ul><ul><li>Requested : Reliable </li></ul></ul><ul><li>History QoS policy : </li></ul><ul><ul><li>Current, Objective : Keep Last n </li></ul></ul><ul><ul><li>Requested : Keep Last 1 </li></ul></ul>Requester Effector Data Bus if long-running if short-running request processed? feedback Current State write Objective State write Requested State read, filter Current State read, filter Objective State read, filter same type same or different types same key Requested State write
    62. 62. Directed State Pattern: How To <ul><li>Variant: Simple state, Fast transition </li></ul><ul><li>Current + Requested States; no Objective State </li></ul><ul><li>States use same type </li></ul><ul><li>Flow: </li></ul><ul><ul><li>Current = x </li></ul></ul><ul><ul><li>Requested  y </li></ul></ul><ul><ul><li>Current  y </li></ul></ul><ul><ul><li>Requested unregister </li></ul></ul><ul><li>Variant: Simple state, Slow transition </li></ul><ul><li>Current, Objective, Requested States </li></ul><ul><li>States use same type </li></ul><ul><li>Flow: </li></ul><ul><ul><li>Current, Objective = x </li></ul></ul><ul><ul><li>Requested  y </li></ul></ul><ul><ul><li>Objective  y </li></ul></ul><ul><ul><li>Current  x + 1  x + 2  … y </li></ul></ul><ul><ul><li>Requested unregister </li></ul></ul>
    63. 63. Directed State Pattern: How To <ul><li>Variant: Complex state, slow transition </li></ul><ul><li>Current, Objective, Requested States </li></ul><ul><li>Requested States use different type than Current </li></ul><ul><ul><li>Requests have identity and lifecycle: e.g. REQUESTED, PROCESSING, DONE </li></ul></ul><ul><ul><li>Objective indicates which request is in process </li></ul></ul><ul><li>Flow: </li></ul><ul><ul><li>Effector : Current, Objective = x </li></ul></ul><ul><ul><li>Requester : Requested  id = 1 , value = y , state = REQ </li></ul></ul><ul><ul><li>Effector : Objective  id = 1 , value = y , state = PROC </li></ul></ul><ul><ul><li>Requester : Requested unregister id = 1 </li></ul></ul><ul><ul><li>Requester : Requested  id = 2 , value = z , state = REQ </li></ul></ul><ul><ul><li>Effector : Current  x + 1  x + 2  … y </li></ul></ul><ul><ul><li>Effector : Objective  id = 1 , value = y , state = DONE </li></ul></ul><ul><ul><li>Effector : Objective  id = 2 , value = z , state = PROC </li></ul></ul><ul><ul><li>Requester : Requested unregister id = 2 </li></ul></ul><ul><ul><li>Effector : Current  y + 1  y + 2  … z </li></ul></ul><ul><ul><li>Effector : Objective  id = 2 , value = z , state = DONE </li></ul></ul>
    64. 64. Directed State Pattern: How To <ul><li>Example: Fire Missile </li></ul>Fire Control Fire Effector Launcher = 5 State = Idle Launcher #5 Firing Authorized Target Identified (Objectives  ) (Current  ) Request ID = 5 Launcher = 5 Action = Fire State = Requested 1 Request ID = 5 Launcher = 5 Action = Fire State = Processing 2 Request ID = 6 Launcher = 6 Action = Fire State = Requested 4 Request ID = 5 Launcher = 5 Action = Fire State = Done 7 Launcher = 5 State = Prep’ing 3 Launcher = 5 State = Firing 5 Launcher = 5 State = Idle 6
    65. 65. Directed State Pattern: Benefits <ul><li>Allows components to express intent and communicate about actions </li></ul><ul><li>Allows additional observing components to leverage request, objective data </li></ul><ul><li>Avoids tight application-to-application coupling of RPC-like (anti-)patterns </li></ul><ul><ul><li>More flexible asynchronous behavior </li></ul></ul><ul><ul><li>Stronger type-based, QoS-based governance </li></ul></ul><ul><ul><li>Easier insertion, removal of application components </li></ul></ul><ul><ul><li>Easier insertion, removal of infrastructure components </li></ul></ul>
    66. 66. Directed State Pattern: Challenges <ul><li>Beware degradation into RPC-style request-reply </li></ul><ul><ul><li>Code smell : blocking behavior, e.g. wait_for_acknowledgments </li></ul></ul><ul><ul><li>Code smell : point-to-point comm. back channels, e.g. payloads in app ACKs </li></ul></ul><ul><ul><li>Code smell : stateless interactions, e.g. many occurrences of keep-all reliability </li></ul></ul>
    67. 67. Directed State Pattern: Challenges <ul><li>Manage state ownership </li></ul><ul><ul><li>One party established state, another needs to change it—why not “pass” ownership on single topic? </li></ul></ul><ul><ul><ul><li>Why not : Embedding workflow of current  objective state transition into multiple components => maintenance pain </li></ul></ul></ul><ul><ul><ul><li>Why not : “Same” objects may be written by different parties but with different ownership, e.g. I own #1, you own #2, she owns #3, … </li></ul></ul></ul><ul><ul><ul><li>Why not : Moving ownership => hard to manage fault tolerance, e.g. persistence </li></ul></ul></ul><ul><ul><li>Best practice : always distinguish Current, (Requested) and Objective states </li></ul></ul><ul><ul><ul><li>Each should have a single owner </li></ul></ul></ul><ul><ul><ul><li>Fail-over should occur among redundant copies, not different functional units </li></ul></ul></ul>
    68. 68. Directed State Pattern: See Also <ul><li>Patterns </li></ul><ul><ul><li>Observer pattern </li></ul></ul><ul><li>Anti-Patterns </li></ul><ul><ul><li>[GoF] “Command” </li></ul></ul><ul><ul><li>[EIP] “Command Message” </li></ul></ul><ul><ul><li>[EIP] “Correlation Identifier” </li></ul></ul><ul><ul><li>[EIP] “Point-to-Point Channel” </li></ul></ul><ul><ul><li>[EIP] “Request-Reply” </li></ul></ul><ul><ul><li>[EIP] “Return Address” </li></ul></ul><ul><ul><li>[EIP] “Scatter-Gather” </li></ul></ul>
    69. 69. Part I Summary <ul><li>Real-World Systems are facing information volume, velocity, and mgmt. challenges </li></ul><ul><li>Common solution is integration around a Data Model </li></ul><ul><li>DDS is a family of OMG specifications that directly supports data-centric publish-subscribe communications </li></ul><ul><li>Use of RTI DDS results in reduced programming, decreased cost, and lowered risk </li></ul>Copyright © 2010 RTI - All rights Reserved. . Cost and Interoperability are the key drivers Resources: http://dds.omg.org http://community.rti.com http://www.rti.com/dds
    70. 70. Day 2: Exercises Gerardo Pardo-Castellote, Ph.D. Co-chair OMG DDS SIG CTO, Real-Time Innovations [email_address] <ul><li>http://www.rti.com </li></ul>
    71. 71. Part II <ul><li>Solving integration use-cases with RTI </li></ul><ul><ul><li>Recording, Persistence, Database Integration, Web Access </li></ul></ul><ul><li>Hello World with RTI DDS </li></ul><ul><li>Configuring QoS via XML </li></ul><ul><li>Development and Run-time Tools: </li></ul><ul><ul><li>Ping, Spy, Analyzer, Wireshark, Excel </li></ul></ul><ul><ul><li>Discovery and Builtin-Topics </li></ul></ul><ul><li>Protocol, Performance & Scalability. </li></ul><ul><li>Future directions and Standards </li></ul>
    72. 72. RTI DDS builds Higher quality, Lower TCO Systems DDS Global Data Space Messaging & Caching Event Processing Database Bridge Persistence & Durability Recording Redundancy & Failover SQL © 2010 Real-Time Innovations, Inc. <ul><li>Presence </li></ul><ul><li>Discovery </li></ul><ul><li>Historical Cache </li></ul><ul><li>Availability </li></ul><ul><li>Redundancy </li></ul><ul><li>Failover Mgmt </li></ul><ul><li>Durable Data </li></ul><ul><li>DDS QoS </li></ul><ul><li>Security Guard Hooks </li></ul><ul><ul><li>Pre-built components address many challenging use-cases </li></ul></ul><ul><li>Extended QoS (*) </li></ul><ul><li>Durable Writers (*) </li></ul><ul><li>Parallel Persistence (*) </li></ul><ul><li>Excel Visualization (*) </li></ul><ul><li>Monitoring (*) </li></ul><ul><li>Recording (*) </li></ul><ul><li>Database Connectivity (*) </li></ul><ul><li>Web Accessibility (*) </li></ul><ul><li>Transformation (*) </li></ul><ul><li>Event Processing (*) </li></ul><ul><li>Secure TLS Transport (*) </li></ul><ul><li>WAN Routing (*) </li></ul><ul><li>Configurability via XML (*) </li></ul><ul><li>Dynamic Data & Types (*) </li></ul><ul><li>Pluggable Transports, Pluggable Discovery(*) </li></ul>(*) Extended capabilities in RTI DDS Comp Comp Comp
    73. 73. Real-Time Recording Service <ul><li>Applications: </li></ul><ul><ul><li>Future analysis and debugging </li></ul></ul><ul><ul><li>Post-mortem </li></ul></ul><ul><ul><li>Compliance checking </li></ul></ul><ul><ul><li>Replay for testing and simulation purposes </li></ul></ul><ul><li>Record high-rate data arriving in real-time </li></ul><ul><li>Non-intrusive – multicast reception </li></ul>© 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL <ul><li>Demo: </li></ul><ul><li>Start RecordingService </li></ul><ul><li>Start ShapesDemo </li></ul><ul><li>See output files </li></ul><ul><li>Convert to: HTML XML </li></ul><ul><li>View Data: HTML XML </li></ul>sqlite stop_all start
    74. 74. Data Persistence <ul><li>A standalone service that persists data outside of the context of a DataWriter </li></ul>Permanent Storage Permanent Storage <ul><li>Can be configured for: </li></ul><ul><li>Redundancy </li></ul><ul><li>Load balancing </li></ul>© 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL <ul><li>Demo: </li></ul><ul><li>PersistenceService </li></ul><ul><li>ShapesDemo </li></ul><ul><li>Application failure </li></ul><ul><li>Application re-start </li></ul><ul><li>Persistence Svc failure </li></ul><ul><li>Application re-start </li></ul>start stop_all Data Writer Global Data Space Data Reader Persistence Service Persistence Service Data Reader Data Writer
    75. 75. Ownership and High Availability <ul><li>Owner determined per Topic and Key </li></ul><ul><li>Only writer with highest strength can publish a Key </li></ul><ul><li>Automatic failover when highest strength writer: </li></ul><ul><ul><li>Loses liveliness </li></ul></ul><ul><ul><li>Misses a deadline </li></ul></ul><ul><ul><li>Stops writing the subject </li></ul></ul><ul><li>Shared Ownership allows any writer to update any object </li></ul>Producer / Writer strength=10 Topic T1 K1 K2 Producer / Writer strength=5 Producer / Writer strength=1 K1 Primary K1 Backup K2 Primary K2 Backup © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL ShapesDemo
    76. 76. Relational Database Integration Relational Actions Topic T1 I1 I2 I3 I1 I2 I3 Table T1 Publish-Subscribe Action Write() Read() & Take() Dispose() Wait() & Listener UPDATE [ 2 , 3 ] & INSERT SELECT DELETE Event driven – The fastest way to observe database changes! © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 1. start mysql 2. start RTC ShapesDemo sql_gui stop_all sql_shell RTI Real-Time Connect
    77. 77. RTI Routing Service <ul><li>Selective, real-time data forwarding and transformation </li></ul><ul><li>Can Change Topic Name and Topic Schema </li></ul><ul><ul><li>Allows for custom transformations via “plugin” </li></ul></ul><ul><ul><li>Can filter/guard data </li></ul></ul><ul><li>QoS managed, can cache last-known value for data </li></ul><ul><li>Dynamically configured </li></ul><ul><li>Location independent deployment </li></ul>DDS Router GUARD XFORM Topic A Topic B ShapesDemo_0 start ShapesDemo_1 stop_all
    78. 78. Global Scalability: LAN to WAN… …without sacrificing Performance and Security DDS Routing Site A Site B Site C Site D WAN / Internet TCP/TLS/SSL Topics: Site Status Alarms Health Logs Sensor Data Proc Sensor Data Topics: Site Status Result Data Topics: Site Status Proc Sensor Data Result Data Alarms Topics: Site Status Sensor Data © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL DDS Routing DDS Routing DDS Routing
    79. 79. Web Accessibility <ul><li>Direct access to real-time data from Web-Based Applications </li></ul>Tactical Real-Time Data Web Enabled DDS Web Enabled DDS Web Enabled DDS GUARD Recorded Data Recorded Data Files Recorded Track Files 1. start replay & router 2. start view maps simulated tracks stop_all DDS Recording Service
    80. 80. COTS tools: Excel – Interacting with your data <ul><li>Display live RTI DDS Data in Excel </li></ul><ul><li>Perform real-time computations and charts </li></ul><ul><li>Publish RTI DDS data from Excel </li></ul>blank saved demo ShapesDemo
    81. 81. Part II <ul><li>Solving integration use-cases with RTI </li></ul><ul><ul><li>Recording, Persistence, Database Integration, Web Access </li></ul></ul><ul><li>Hello World with RTI DDS </li></ul><ul><li>Configuring QoS via XML </li></ul><ul><li>Development and Run-time Tools: </li></ul><ul><ul><li>Ping, Spy, Analyzer, Wireshark, Excel </li></ul></ul><ul><ul><li>Discovery and Builtin-Topics </li></ul></ul><ul><li>Protocol, Performance & Scalability. </li></ul><ul><li>Future directions and Standards </li></ul>
    82. 82. Hands-on Example (C++) Type Definition MyType.idl rtiddsgen MyType.h MyTypeSupport.c MyTypePublisher.cpp MyTypeSubscriber.cpp MyType.sln Publisher.exe Subscriber.exe <ul><li>Three minutes to a running app!! </li></ul><ul><li>Define your data </li></ul><ul><li>Create your project </li></ul><ul><li>Build </li></ul><ul><li>Run: publisher subscriber </li></ul>Aux: File Browser Console Delete Files rtiddsspy compiler
    83. 83. Exercise #1 - Hello World <ul><li>Define you data type: </li></ul><ul><li>Create a directory “HelloWorld” </li></ul><ul><li>Create a file called HelloWorld.idl and open it in VisualStudio </li></ul><ul><li>Add the following contents: </li></ul><ul><ul><li>const long MSG_LEN=256; </li></ul></ul><ul><ul><li>struct Hello { </li></ul></ul><ul><ul><li> string<MSG_LEN> user; //@key </li></ul></ul><ul><ul><ul><li>string<MSG_LEN> msg; </li></ul></ul></ul><ul><ul><li>}; </li></ul></ul>
    84. 84. Run rtiddsgen (for C++) <ul><li>rtiddsgen HelloWorld.idl -language C++ -example i86Win32VS2005 </li></ul><ul><li>-replace -ppDisable </li></ul><ul><li>rtiddsgen HelloWorld.idl -language Java -example i86Win32jdk </li></ul><ul><li>-replace -ppDisable </li></ul><ul><li>Look at the directory you should see: </li></ul><ul><ul><li>HelloWorld-vs2005.sln </li></ul></ul><ul><ul><li>And Several other files… </li></ul></ul><ul><li>Open the Solution File (type hello-vs2005.sln on the console) </li></ul><ul><ul><li>Look at HelloMsgPublisher.cxx </li></ul></ul><ul><ul><li>Look at HelloMsgSubscriber.cxx </li></ul></ul><ul><li>Build the Solution </li></ul>
    85. 85. Run rtiddsgen (for Java) <ul><li>rtiddsgen hello.idl -language Java -example i86Win32jdk </li></ul><ul><li>-replace -ppDisable </li></ul><ul><li>Look at the directory you should see: </li></ul><ul><ul><li>makefile_hello_i86Win32jdk </li></ul></ul><ul><ul><li>And Several other files… </li></ul></ul><ul><ul><ul><li>Look at HelloMsgPublisher.java </li></ul></ul></ul><ul><ul><ul><li>Look at HelloMsgSubscriber.java </li></ul></ul></ul><ul><li>You can use the makefile to build and the Java programs: </li></ul><ul><ul><li>gmake –f makefile_hello_i86Win32jdk </li></ul></ul>
    86. 86. Execute the program <ul><li>C++: </li></ul><ul><ul><li>On one window run: </li></ul></ul><ul><ul><ul><li>objsi86Win32VS2005HelloMsgPublisher.exe </li></ul></ul></ul><ul><ul><li>On another window run: </li></ul></ul><ul><ul><ul><li>objsi86Win32VS2005HelloMsgSubscriber.exe </li></ul></ul></ul><ul><li>Java </li></ul><ul><ul><li>On one window run: </li></ul></ul><ul><ul><ul><li>gmake –f makefile_hello_i86Win32jdk HelloMsgPublisher </li></ul></ul></ul><ul><ul><li>On another window run: </li></ul></ul><ul><ul><ul><li>gmake –f makefile_hello_i86Win32jdk HelloMsgSubscriber </li></ul></ul></ul><ul><li>You should see the subscribers getting an empty string… </li></ul>
    87. 87. Modify the program to produce something <ul><li>C++: Open HelloMsgPublisher.cxx in VisualStudio </li></ul><ul><li>Java: Open HelloMsgPublisher.java in your preferred tool </li></ul><ul><li>Look for the comment: </li></ul><ul><ul><li>/* Modify the data to be sent here * / </li></ul></ul><ul><li>Add the line: </li></ul><ul><li>strcpy_s(instance->user, MSG_LEN, “ Your name here &quot;); </li></ul><ul><li>strcpy_s(instance->msg, MSG_LEN, &quot;Hello world &quot;); </li></ul><ul><li>Kill the Publisher, Rebuild the publisher and run it again </li></ul>
    88. 88. Playing with rtiddsspy <ul><li>Run rtiddsspy while the other applications are running </li></ul><ul><li>Start and stop applications. What do you see in rtiddsspy </li></ul>
    89. 89. Example: Publication // Entities creation DomainParticipant participant = TheParticipantFactory->create_participant( domain_id, participant_qos, participant_listener); Publisher publisher = domain->create_publisher( publisher_qos, publisher_listener); Topic topic = domain->create_topic( “ MyTopic”, “Text”, topic_qos, topic_listener); DataWriter writer = publisher->create_datawriter( topic, writer_qos, writer_listener); TextDataWriter twriter = TextDataWriter::narrow(writer); TextStruct my_text; twriter->write(&my_track);
    90. 90. Example: Subscription // Entities creation Subscriber subscriber = domain->create_subscriber( subscriber_qos, subscriber_listener); Topic topic = domain->create_topic( “ Track”, “TrackStruct”, topic_qos, topic_listener); DataReader reader = subscriber->create_datareader( topic, reader_qos, reader_listener); // Use listener-based or wait-based access
    91. 91. How to Get Data? (Listener-Based) // Listener creation and attachment Listener listener = new MyListener(); reader->set_listener(listener); // Listener code MyListener::on_data_available( DataReader reader ) { TextSeq received_data; SampleInfoSeq sample_info; TextDataReader reader = TextDataReader::narrow(reader); treader->take( &received_data, &sample_info, …) // Use received_data printf(“Got: %sn”, received_data[0]->contents); }
    92. 92. How to Get Data? (WaitSet-Based) // Creation of condition and attachement Condition foo_condition = treader->create_readcondition(…); waitset->add_condition(foo_condition); // Wait ConditionSeq active_conditions; waitset->wait(&active_conditions, timeout); // Wait returns when there is data (or timeout) FooSeq received_data; SampleInfoSeq sample_info; treader->take_w_condition (&received_data, &sample_info, foo_condition); // Use received_data printf(“Got: %sn”, received_data[0]->contents);
    93. 93. Listeners, Conditions & WaitSets <ul><li>Middleware must notify user application of relevant events: </li></ul><ul><ul><li>Arrival of data </li></ul></ul><ul><ul><li>But also: </li></ul></ul><ul><ul><ul><li>QoS violations </li></ul></ul></ul><ul><ul><ul><li>Discovery of relevant entities </li></ul></ul></ul><ul><ul><li>These events may be detected asynchronously by the middleware </li></ul></ul><ul><ul><ul><li>… Same issue arises with POSIX signals </li></ul></ul></ul><ul><li>DDS allows the application to choice: </li></ul><ul><ul><li>Either to get notified asynchronously using a Listener </li></ul></ul><ul><ul><li>Or to wait synchronously using a WaitSet </li></ul></ul><ul><li>Both approaches are unified using STATUS changes </li></ul>
    94. 94. Status Changes <ul><li>DDS defines </li></ul><ul><li>A set of enumerated STATUS </li></ul><ul><li>The statuses relevant to each kind of DDS Entity </li></ul><ul><li>DDS entities maintain a value for each STATUS </li></ul>struct LivelinessChangedStatus { long active_count; long inactive_count; long active_count_change; long inactive_count_change; } STATUS Entity INCONSISTENT_TOPIC Topic DATA_ON_READERS Subscriber LIVELINESS_CHANGED DataReader REQUESTED_DEADLINE_MISSED DataReader RUQESTED_INCOMPATIBLE_QOS DataReader DATA_AVAILABLE DataReader SAMPLE_LOST DataReader SUBSCRIPTION_MATCH DataReader LIVELINESS_LOST DataWriter OFFERED_INCOMPATIBLE_QOS DataWriter PUBLICATION_MATCH DataWriter
    95. 95. Listeners, Conditions and Statuses <ul><li>A DDS Entity is associated with: </li></ul><ul><ul><li>A listener of the proper kind (if attached) </li></ul></ul><ul><ul><li>A StatusCondition (if activated) </li></ul></ul><ul><li>The Listener for an Entity has a separate operation for each of the relevant statuses </li></ul>STATUS Entity Listener operation INCONSISTENT_TOPIC Topic on_inconsistent_topic DATA_ON_READERS Subscriber on_data_on_readers LIVELINESS_CHANGED DataReader on_liveliness_changed REQUESTED_DEADLINE_MISSED DataReader on_requested_deadline_missed RUQESTED_INCOMPATIBLE_QOS DataReader on_requested_incompatible_qos DATA_AVAILABLE DataReader on_data_available SAMPLE_LOST DataReader on_sample_lost SUBSCRIPTION_MATCH DataReader on_subscription_match LIVELINESS_LOST DataWriter on_liveliness_lost OFFERED_INCOMPATIBLE_QOS DataWriter on_offered_incompatible_qos PUBLICATION_MATCH DataWriter on_publication_match
    96. 96. Listeners & Condition duality <ul><li>A StatusCondition can be selectively activated to respond to any subset of the statuses </li></ul><ul><li>An application can wait changes in sets of StatusConditions using a WaitSet </li></ul><ul><li>Each time the value of a STATUS changes DDS </li></ul><ul><ul><li>Calls the corresponding Listener operation </li></ul></ul><ul><ul><li>Wakes up any threads waiting on a related status change </li></ul></ul>Asynchronous notification via Listener operation Synchronous notification via activation/wakeup of conditions/waitsets DDS Entity Status Change
    97. 97. Exercise #2 – Shapes Publisher <ul><li>Create a new directory Shapes </li></ul><ul><li>In the Directory create a file called ShapeType.idl </li></ul><ul><li>Edit the file to have the following content: </li></ul><ul><ul><li>const long COLOR_LEN=64; </li></ul></ul><ul><ul><li>struct ShapeType { </li></ul></ul><ul><ul><li>string<COLOR_LEN>color; //@key </li></ul></ul><ul><ul><li>long x; </li></ul></ul><ul><ul><li>long y; </li></ul></ul><ul><ul><li>long shapesize; </li></ul></ul><ul><ul><li>}; </li></ul></ul><ul><li>Run: </li></ul><ul><li>rtiddsgen ShapeType.idl -language C++ -example i86Win32VS2005 –replace -ppNotRun </li></ul>
    98. 98. rtiddsgen Details <ul><li>rtiddsgen [-d <outdir>] [-language <C|C++|Java|C++/CLI|C#>] </li></ul><ul><li>[-namespace] [-package <packagePrefix>] </li></ul><ul><li>[-example <arch>] [-replace] [-debug] </li></ul><ul><li>[-corba [client header file]] [-optimization <level of optimization>] </li></ul><ul><li>[-stringSize <Unbounded strings size>] </li></ul><ul><li>[-sequenceSize <Unbounded sequences size>] </li></ul><ul><li>[-notypecode] [-ppDisable] [-ppPath <path to the preprocessor>] </li></ul><ul><li>[-ppOption <option>] [-D <name>[=<value>]] </li></ul><ul><li>[-U <name>] [-I <directory>] [-noCopyable] [-use42eAlignment] </li></ul><ul><li>[-help] [-version] [-convertToIdl | -convertToXml | -convertToXsd | </li></ul><ul><li> -convertToWsdl] </li></ul><ul><li>[[-inputIdl] <IDLInputFile.idl> | [-inputXml] <XMLInputFile.xml> | [-inputXsd] <XSDInputFile.xsd> | [-inputWsdl] <WSDLInputFile.wsdl>] </li></ul><ul><li>DefinitionFile can be IDL, XSD and XML file </li></ul><ul><li>-example generates example pub/sub apps and makefiles for compilation. </li></ul><ul><li>-replace replaces everything that’s generated. Use if the data type definition has changed. Always use with caution if you’ve made modifications. </li></ul>
    99. 99. IDL vs. XML: IDL Example struct MemberStruct{ short sData; } typedef MemberStructType; //@top-level false
    100. 100. IDL vs. XML: XML Example <? xml version =&quot;1.0“ encoding =&quot;UTF-8&quot;?> < types xmlns:xsi =&quot;http://www.w3.org/2001/XMLSchema-instance&quot; xsi:noNamespaceSchemaLocation =&quot;../rti_dds_topic_types.xsd&quot;> < struct name =&quot;MemberStruct&quot; topLevel =&quot;false&quot;> < member name =&quot;sData“ type =&quot;short&quot;/> </ struct > < typedef name =&quot;MemberStructType&quot; type =&quot;nonBasic“ nonBasicTypeName =&quot;MemberStruct“ topLevel =&quot;false&quot;/> </ types >
    101. 101. IDL vs. XSD: XSD Example <? xml version =&quot;1.0&quot; encoding =&quot;UTF-8&quot;?> < xsd:schema xmlns:xsd =&quot;http://www.w3.org/2001/XMLSchema&quot; xmlns:dds =&quot;http://www.omg.org/dds&quot; xmlns:tns =&quot;http://www.omg.org/IDL-Mapped/&quot; targetNamespace =&quot;http://www.omg.org/IDL-Mapped/&quot;> < xsd:import namespace =&quot;http://www.omg.org/dds&quot; schemaLocation =&quot;rti_dds_topic_types_common.xsd&quot;/> < xsd:complexType name =&quot;MemberStruct&quot;> < xsd:sequence > < xsd:element name =&quot;sData&quot; minOccurs =&quot;1&quot; maxOccurs =&quot;1&quot; type =&quot;xsd:short&quot;/> </ xsd:sequence > </ xsd:complexType > <!-- @topLevel false --> < xsd:complexType name =&quot;MemberStructType&quot;> < xsd:complexContent > < xsd:restriction base =&quot;tns:MemberStruct&quot;> < xsd:sequence > < xsd:element name =&quot;sData&quot; type =&quot;xsd:short&quot; minOccurs =&quot;1&quot; maxOccurs =&quot;1&quot;/> </ xsd:sequence > </ xsd:restriction > </ xsd:complexContent > </ xsd:complexType > <!-- @topLevel false --> </ xsd:schema >
    102. 102. Part II <ul><li>Solving integration use-cases with RTI </li></ul><ul><ul><li>Recording, Persistence, Database Integration, Web Access </li></ul></ul><ul><li>Hello World with RTI DDS </li></ul><ul><li>Development and Run-time Tools: </li></ul><ul><ul><li>Ping, Spy, Analyzer, Wireshark, Excel </li></ul></ul><ul><ul><li>Discovery and Builtin-Topics </li></ul></ul><ul><ul><li>Monitoring </li></ul></ul><ul><li>Configuring QoS via XML </li></ul><ul><li>Protocol, Performance & Scalability. </li></ul><ul><li>Future directions and Standards </li></ul>
    103. 103. Tools provide insight into a distributed system <ul><li>RTI Analyzer </li></ul><ul><ul><li>Understand connections and data flow </li></ul></ul><ul><ul><li>Tune QoS properties without changing code </li></ul></ul><ul><li>RTI Excel Addin </li></ul><ul><ul><li>View Data </li></ul></ul><ul><ul><li>Inject Data </li></ul></ul><ul><ul><li>Compute / Analyze RT Data </li></ul></ul><ul><li>RTI Protocol Analyzer </li></ul><ul><ul><li>Sniff the wire and analyze traffic </li></ul></ul><ul><li>RTI Monitoring </li></ul><ul><ul><li>Visualize and monitor system operation </li></ul></ul><ul><ul><li>Detect system problems, bottlenecks, etc. </li></ul></ul>
    104. 104. Let’s try the Tools!
    105. 105. XML-Defined Qos Profiles: Overview <ul><li>QoS can be specified in XML file(s) and is automatically loaded when the application starts </li></ul><ul><li>XML – specified QoS will be automatically used when DDS entities are created </li></ul><ul><ul><li>Globally for all new entities </li></ul></ul><ul><ul><li>Selectively, depending on the Topic associated with the entity </li></ul></ul><ul><li>XML –specified QoS can be accessed via new API calls: </li></ul><ul><ul><li>create_<ENTITY>_with_profile() </li></ul></ul><ul><ul><li>get_< ENTITY >_qos_from_profile() </li></ul></ul><ul><ul><li>set_default_< ENTITY >_qos_with_profile() </li></ul></ul>
    106. 106. Benefits of using QoS profiles <ul><li>Simplifies application programming. </li></ul><ul><ul><li>Editing XML is simpler than editing/compiling source code </li></ul></ul><ul><ul><li>Same settigs can be shared between applications/languages </li></ul></ul><ul><li>Accelerates development </li></ul><ul><ul><li>Changing QoS does not require re-compilation & re-deployment </li></ul></ul><ul><li>Increases flexibility and maintainability </li></ul><ul><ul><li>Can change QoS applications to meet changing system needs without requiring re-deployment </li></ul></ul><ul><li>Improves code reuse </li></ul><ul><ul><li>Separates configuration/deployment from application logic </li></ul></ul><ul><ul><li>Same code can be re-deployed on different hardware/networks/systems just changing the QoS file </li></ul></ul><ul><li>Enhances portability </li></ul><ul><ul><li>Application code is not dependent on vendor QoS extensions </li></ul></ul>
    107. 107. More on using QoS profiles <ul><li>Could increase brittleness </li></ul><ul><ul><li>QoS files can change between runs. What was working no longer works the same way… </li></ul></ul><ul><li>Portability </li></ul><ul><ul><li>XML profile format now standardized as part of “DDS for light-weight CCM” specification </li></ul></ul><ul><ul><ul><li>Will be folded in the next revision of the standard </li></ul></ul></ul><ul><ul><li>File-based use of QoS profiles is always portable </li></ul></ul><ul><ul><li>Direct calls to the C/C++/Java are not part of the standard yet </li></ul></ul><ul><li>Makes deployment dependent on accessing a file-system… </li></ul><ul><ul><li>XML can be compiled into the code, but that negates some of the benefits </li></ul></ul>
    108. 108. What is a QoS Profile? <ul><li>Set of related Entity-QoS, usually one per entity kind, identified by a name. </li></ul><ul><ul><li>Provides all the information needed to fully configure the QoS for any DDS Entity ( DomainParticipant, Publisher, DataWriter, …) </li></ul></ul><ul><li>QoS policies and attributes can be set explicitly, in the profile or otherwise are inherited from it’s “base profile” </li></ul><ul><ul><li>Profiles that do not specify a “base profile” automatically inherit from the DDS-Specified Default QoS. </li></ul></ul><ul><li>May contain multiple definitions of a EntityQos (e.g. DataWriterQos) </li></ul><ul><ul><li>Can be disambiguated by Topic related to the Entity </li></ul></ul><ul><ul><li>If ambiguous the last definition prevails </li></ul></ul><ul><li>Defined inside Qos libraries </li></ul>
    109. 109. What is inside an XML QoS file? <ul><li>A collection of QoS profiles organized inside QoS Libraries </li></ul>Example <ul><li>Field names are the same as in C++/Java source code. Eg.: </li></ul><ul><ul><li>datawriter_qos.reliability.max_blocking_time.sec </li></ul></ul>
    110. 110. How are XML Qos files specified? <ul><li>NDDSHOME directory (first to be loaded) </li></ul><ul><ul><li>$NDDSHOME/resource/qos_profiles_XXX/xml/NDDS_QOS_PROFILES.xml </li></ul></ul><ul><ul><li>Automatically loaded if present unless: </li></ul></ul><ul><ul><li>DDS_ProfileQosPolicy::ignore_resource_profile == DDS_BOOLEAN_TRUE </li></ul></ul><ul><li>NDDS_QOS_PROFILES Environment variable </li></ul><ul><ul><li>NDDS_QOS_PROFILES= [<url1> | <url2>]; … </li></ul></ul><ul><ul><li><file_url> ::= file://usr/local/default_dds.xml </li></ul></ul><ul><ul><li><string_url> ::= str://<dds><qos_library>…</qos_library></dds> </li></ul></ul><ul><ul><li>Automatically loaded if defined unless: </li></ul></ul><ul><ul><li>DDS_ProfileQosPolicy::ignore_environment_profile == DDS_BOOLEAN_TRUE </li></ul></ul><ul><li>Application directory </li></ul><ul><ul><li>USER_QOS_PROFILES.xml </li></ul></ul><ul><ul><li>Automatically loaded if present unless: </li></ul></ul><ul><ul><li>DDS_ProfileQosPolicy::ignore_user_profile == DDS_BOOLEAN_TRUE </li></ul></ul><ul><li>DDS_ProfileQosPolicy::url_profile </li></ul><ul><ul><li>Automatically loaded if defined </li></ul></ul><ul><li>DDS_ProfileQosPolicy::string_profile </li></ul><ul><ul><li>Automatically loaded if defined </li></ul></ul>Duplicate Qos Libraries and/or Qos Profiles within a library are not allowed
    111. 111. Side note: URL Format <ul><li>URLs are required for either specifying QoS profile location, or for specifying a profile string </li></ul><ul><ul><li>File URL : </li></ul></ul><ul><ul><ul><li>Example: file:// usr/local/default_dds.xml </li></ul></ul></ul><ul><ul><li>String URL </li></ul></ul><ul><ul><ul><li>Can be set in code, or as part of the environment variable </li></ul></ul></ul><ul><ul><ul><li>Must contain whole XML document </li></ul></ul></ul><ul><ul><ul><li>Example: str:// <dds><qos_library> . . . </qos_library><dds> </li></ul></ul></ul><ul><li>URL groups </li></ul><ul><ul><li>Redundant files in URL groups provide fault-tolerance </li></ul></ul><ul><ul><li>Only the first one found will be loaded </li></ul></ul><ul><ul><li>Example: </li></ul></ul><ul><ul><li>[file://usr/local/default_dds.xml | file://usr/local/alternative_default_dds.xml] </li></ul></ul>
    112. 112. API access to XML QoS Policies <ul><li>If XML QoS policies are used, they are automatically loaded before creating the first participant </li></ul><ul><li>XML QoS policies can be explicitly reloaded and unloaded via API calls: DDSDomainParticipantFactory::reload_profiles( ) DDSDomainParticipantFactory::unload_profiles( ) </li></ul><ul><li>XML QoS can be explicitly applied to an entity by calling <ENTITY>::set_qos_with_profile(), </li></ul><ul><li>create_<ENTITY>_with_profile() </li></ul><ul><li>XML QoS can be retrieved by calling DDSDomainParticipantFactory::get<ENTITY>_qos_with_profile() </li></ul>
    113. 113. QoS Profile Inheritance <ul><li>A QoS profile can inherit the values of another QoS profile </li></ul><ul><li>Base profile set using the base_name tag </li></ul><ul><li>Base profile can be in another file and/or library </li></ul><ul><li>A Profile that does not specify a base_name implicitly inherits from the Default Profile </li></ul><ul><li>Example: </li></ul>< qos_profile name = &quot; ReliableKeepLast &quot; base_name = &quot; BasicQos &quot; > < datawriter_qos > < reliability > < kind > RELIABLE_RELIABILITY_QOS </ kind > < max_blocking_time > < sec > 1 </ sec > < nanosec > 0 </ nanosec > </ max_blocking_time > </ reliability > < history > < kind > KEEP_LAST_HISTORY_QOS </ kind > < depth > 10 </ depth > </ history > </ datawriter_qos > </ qos_profile >
    114. 114. Modifying the default QoS using XML files <ul><li>The “default” QoS used by DDS can be modified via an explicit call to: </li></ul><ul><ul><li>set_default_<ENTITY>_qos_with_profile() </li></ul></ul><ul><li>Alternatively a QoS profile can be marked such that it is used as the default QoS </li></ul><ul><ul><li>In combination with the “topic filter” (next slide) this can be used to surgically modify the QoS of a deployed application </li></ul></ul>< qos_library name = &quot; MyQosLib &quot; > < qos_profile name = &quot; ReliableKeepLast &quot; is_default_qos = &quot; true &quot; > < datawriter_qos > … </ datawriter_qos > < datareader_qos > … </ datareader_qos > </ qos_profile > </ qos_library >
    115. 115. Fine-tuning which QoS applies to an Entity via Topic filters <ul><li>Allows a QoS profile to affect only selected DataReaders or DataWriters </li></ul><ul><li>Affected DataReaders or DataWriters are specified via an expression in their Topic name </li></ul><ul><li>Single QoS file and profile can be used to configure the whole system </li></ul><ul><li>QoS of selected entities can be changed without application code changes </li></ul><ul><li>QoS can be inherited from another library. E.g.: </li></ul>< datawriter_qos topic_filter = &quot; Alarm* &quot; > … </ datawriter_qos >
    116. 116. XML QoS Notes <ul><li>If both XML and source code are used to configure an entity, the last one set will be used </li></ul><ul><li>If the entity already exists, set_qos_with_profile() can be called after reload() only of no immutable QoS have changed </li></ul><ul><li>DTD validation is done automatically. Specify the DTD in the XML file when you want to overwrite the DTD used by the middleware </li></ul><ul><li>XSD is available. Makes editing very easy when using a XML/XSD aware editor </li></ul><ul><li>More details on Chapter 15 of the Reference Manual </li></ul>
    117. 117. Configuration of QoS via XML-defined profiles <ul><li>Try it out using the shapes demo! </li></ul>
    118. 118. Part II <ul><li>Solving integration use-cases with RTI </li></ul><ul><ul><li>Recording, Persistence, Database Integration, Web Access </li></ul></ul><ul><li>Hello World with RTI DDS </li></ul><ul><li>Development and Run-time Tools: </li></ul><ul><ul><li>Ping, Spy, Analyzer, Wireshark, Excel </li></ul></ul><ul><ul><li>Discovery and Builtin-Topics </li></ul></ul><ul><li>Configuring QoS via XML </li></ul><ul><li>Protocol, Performance & Scalability. </li></ul><ul><li>Future directions and Standards </li></ul>
    119. 119. 20X Faster than JMS / Broker-based solutions Platform: Linux 2.6 on AMD Athlon, Dual core, 2.2 GHz RTI DDS is about 20X faster than JMS RTI DDS reliable multicast exhibits near perfect scalability ( 2KB messages ) © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
    120. 120. Extremely low latency and jitter © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL Reliable, ordered delivery over Gigabit Ethernet between 2.4 GHz Core 2 Quad processors running 32-bit Red Hat Enterprise Linux 5.0
    121. 121. Orders of magnitude more scalable than broker-based solutions <ul><li>Going from 1 to 888 subscribers of the same data has only a 10% impact on throughput </li></ul><ul><li>New topics can be added to a system without impacting the latency and throughput on other topics </li></ul><ul><li>Throughput with 8 topics is 8x the throughput with 1 topic </li></ul>http://www.rti.com/products/dds/benchmarks-cpp-linux.html © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
    122. 122. Realizing Performance & Scalability <ul><li>RTI DDS operates peer-to-peer, without brokers </li></ul><ul><li>RTI DDS uses RTPS, an Advanced Multi-Session protocol supporting Reliable Multicast </li></ul>RTI DDS Approach RTPS JMS AMQP ESBs … Others: Broker-based middleware © 2010 Real-Time Innovations, Inc.
    123. 123. Realizing Performance & Scalability <ul><li>RTI DDS completely isolates each process and communicates peer-to-peer </li></ul><ul><li>Other DDS implementations communicate vis shared memory with local daemon </li></ul>RTI DDS Approach RTPS SharedMem RTPS Others: Broker-based middleware © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
    124. 124. Advanced Scalability & Performance Techniques <ul><li>Latency and Priority Aware message batching </li></ul><ul><li>Content-Aware multi-channel reliable multicast </li></ul><ul><li>Enhanced Reliable Protocol </li></ul><ul><ul><li>Selective ACKs (SACKs) for Confirmed Reliability </li></ul></ul><ul><ul><li>NACK-only Reliable Protocol for Massive Scalability </li></ul></ul><ul><li>Smart caching integrated with the message protocol </li></ul><ul><li>Content-Filtering at the source </li></ul>© 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
    125. 125. Message Batching write() sender receiver write() sender Send queue Receive queue Send queue Receive queue Without batching each message is separately sent. For small messages protocol headers might be bigger than payload With batching messages are held a little and combined into larger batches maximizing throughout and minimizing CPU receiver Transparent: Receiver still sees individual messages © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
    126. 126. Reliability with Batching <ul><li>Reliability must work even when messages are batched </li></ul><ul><li>ACK or NACK of individual samples would negate some of the benefits of batching… </li></ul><ul><li>=> Protocol must be batch aware so that it can ACK/NACK complete batches! </li></ul>B3 B2 B1 B3 B2 B1 ACK(B3), NACK(B2) Repair B2 B3 B2 B1 write() sender receiver © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
    127. 127. Batching has a huge payoff! Intel Core2Duo Single-CPU Dual-Core 2.4GHz, 4MB cache 32-bit CentOS 5 (RHEL 5), 2GB memory, Intel E1000 NIC © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
    128. 128. Classic (TCP Style) Reliable Protocol No packet loss situation Company Confidential 01 02 03 04 01 02 03 04, HB 01 02 03 04 ACK 1-4 05 06 07 08 05 06 07 08, HB 05 06 07 08 ACK 1-8 © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL ShapesDemo
    129. 129. Classic (TCP Style) Reliable Protocol with some packet loss 01 02 03 04 01 02 03 04, HB 01 02 X ACK 1-2, NACK 3 05 06 07 08 05 06 07 08, HB 06 07 08 ACK 1-8 03 04 05 X X Packets 04 and 05 are received but the protocol drops them because a prior packet 03 is missing. This wastes valuable bandwidth © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
    130. 130. RTI DDS Reliability (Reader Cache + SACK) improves performance when packet loss occurs 01 02 03 04 01 02 03 04, HB 01 02 X 04 ACK 1-2, SACK 3 05 06 07 08 05 06 07 08, HB 05 06 07 08 ACK 1-8 03 Packets 04 and 05 are received and cached waiting for the repair of 03. No bandwidth is wasted. © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
    131. 131. RTI DDS NACK-only reliability eliminates ACK traffic if there no packet loss 01 02 03 04 01 02 03 04, HB 01 02 03 04 05 06 07 08 05 06 07 08, HB 05 06 07 08 No ACK traffic under normal operating conditions © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
    132. 132. RTI DDS NACK-only reliability greatly reduces traffic even with packet loss 01 02 03 04 01 02 03 04, HB 01 02 X 04 NACK 3 05 06 07 08 05 06 07 08, HB 05 06 07 08 03 Negative Acknowledgments sent only when some message is lost This approach is far more scalable when there are many subscribers © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
    133. 133. Part II <ul><li>Solving integration use-cases with RTI </li></ul><ul><ul><li>Recording, Persistence, Database Integration, Web Access </li></ul></ul><ul><li>Hello World with RTI DDS </li></ul><ul><li>Development and Run-time Tools: </li></ul><ul><ul><li>Ping, Spy, Analyzer, Wireshark, Excel </li></ul></ul><ul><ul><li>Discovery and Builtin-Topics </li></ul></ul><ul><li>Configuring QoS via XML </li></ul><ul><li>Protocol, Performance & Scalability. </li></ul><ul><li>Future directions and Standards </li></ul>
    134. 134. OMG Standards update <ul><li>DDS API - 2003 </li></ul><ul><li>DDS Wire Interoperability - 2006 </li></ul><ul><li>UML Profile for DDS - 2007 </li></ul><ul><li>DDS for lwCCM – 2008 </li></ul><ul><li>Extensible Topics for DDS – 2010 </li></ul><ul><li>ISO C++ API – 2010 </li></ul><ul><li>Java 5 API - 2010 </li></ul>Upcoming standards: + Web Enabled DDS (Mar 2011) + Secure DDS (Dec 2011)
    135. 135. Web-Protocol Access to DDS Global Data Space <ul><li>Stateless access of data via standard Web Technologies & Protocols </li></ul><ul><li>Not a bridge, broker, or message router </li></ul>Web App DDS RTPS Global Data Space App App App App Web Integration Service HTTP
    136. 136. DDS Security RFP <ul><li>A PIM and language PSMs for set of “plugins” that provide Authentication, Authorization, Data Tagging, Signing, and Encryption </li></ul><ul><li>A set of pre-defined plugins that can be used to define and enforce security policies. </li></ul><ul><li>Extensions to the DDS-RTPS Wire Protocol to support secure interoperability between DDS Systems. </li></ul>
    137. 137. Pub/Sub middleware (DDS) Application Secure OS Secure Transport DDS Application Secure DDS Global Data Credential Mgmt Authentication Access Control Data Tagging Encryption DDS Application DDS Application ? ? ? ?
    138. 138. App. Other DDS System Secure DDS middleware Authentication Plugin Access Control Plugin Data Encryption Plugin Data Tagging Plugin Crypto Module (e.g. TPM ) Secure Transport (e.g. TLS) CS application component certificates Kernel Policies Network Driver Network Encrypted Data TAGS Other DDS System Other DDS System App. App. Secure Kernel (e.g. SE Linux, MILS) ? Data cache Protocol Engine DDS Entities ? 1 2 3 4 5 6 7 8 9
    139. 139. Part III <ul><li>Application development cycle </li></ul><ul><li>Advanced Exercise </li></ul>
    140. 140. Exercise: How could “chat rooms” be implemented? <ul><li>Different Topics for each Chat room? </li></ul><ul><li>Map to Partitions? </li></ul><ul><li>Add field to the message and use content-filtered Topics? </li></ul><ul><li>Same as before and also make room part of the Key? </li></ul><ul><li>Others? </li></ul>Discuss pros and cons of each approach
    141. 141. Exercise: How could we implement Ground control stations that monitor UAVs <ul><li>Different Topics for each UAV? </li></ul><ul><ul><li>Or use Keys? </li></ul></ul><ul><li>Different Domains for each Ground Station? </li></ul><ul><ul><li>Or Partitions? </li></ul></ul><ul><li>How to control multiple UAVs from the same ground station? </li></ul><ul><li>How to switch the ground station that controls the UAV? </li></ul><ul><li>How to do failover between ground stations? </li></ul><ul><li>How to direct a message to one or all UAVs? </li></ul><ul><li>How to detect loss of connection to an UAV? </li></ul>Discuss pros and cons of each approach
    142. 142. Edit the chat_publisher.cxx <ul><li>Go to the line with comment: /* Main loop */ </li></ul><ul><ul><li>Add the line: </li></ul></ul><ul><ul><li>strcpy_s(instance->name, </li></ul></ul><ul><ul><ul><ul><ul><li>NAME_LEN, &quot;Gerardo Pardo&quot;); </li></ul></ul></ul></ul></ul><ul><ul><li>(Use your own name) </li></ul></ul><ul><li>Go to the line with comment: </li></ul><ul><ul><li>/* Modify the data to be sent here */ </li></ul></ul><ul><ul><li>Add the lines: </li></ul></ul><ul><ul><li>instance->age = count; </li></ul></ul><ul><ul><li>strcpy_s(instance-> msg , </li></ul></ul><ul><ul><ul><ul><ul><li>NAME_LEN, “ Como va todo? &quot;); </li></ul></ul></ul></ul></ul><ul><ul><li>(Use your age and personalized message) </li></ul></ul><ul><li>Rebuild and execute </li></ul>
    143. 143. Exercise #3 – Chat Room <ul><li>Create a new directory Chat </li></ul><ul><li>In the Directory create a file called chat.idl </li></ul><ul><li>Edit the file to have the following content: </li></ul><ul><ul><li>const long NAME_LEN=64; </li></ul></ul><ul><ul><li>const long MSG_LEN=256; </li></ul></ul><ul><ul><li>struct ChatMsg { </li></ul></ul><ul><ul><li>string<NAME_LEN>name; //@key </li></ul></ul><ul><ul><li>long age; </li></ul></ul><ul><ul><li> string<MSG_LEN> chatRoom; </li></ul></ul><ul><ul><li>string<MSG_LEN> msg; </li></ul></ul><ul><ul><li>}; </li></ul></ul><ul><li>Run: </li></ul><ul><li>rtiddsgen chat.idl -language C++ -example i86Win32VS2005 –replace -ppNotRun </li></ul>
    144. 144. Edit the chat_publisher.cxx <ul><li>Go to the line with comment: /* Main loop */ </li></ul><ul><ul><li>Add the line: </li></ul></ul><ul><ul><li>strcpy_s(instance->name, </li></ul></ul><ul><ul><ul><ul><ul><li>NAME_LEN, &quot;Gerardo Pardo&quot;); </li></ul></ul></ul></ul></ul><ul><ul><li>(Use your own name) </li></ul></ul><ul><li>Go to the line with comment: </li></ul><ul><ul><li>/* Modify the data to be sent here */ </li></ul></ul><ul><ul><li>Add the lines: </li></ul></ul><ul><ul><li>instance->age = count; </li></ul></ul><ul><ul><li>strcpy_s(instance-> msg , </li></ul></ul><ul><ul><ul><ul><ul><li>NAME_LEN, “ Como va todo? &quot;); </li></ul></ul></ul></ul></ul><ul><ul><li>(Use your age and personalized message) </li></ul></ul><ul><li>Rebuild and execute </li></ul>
    145. 145. Exercise #3 (continued) Use Qos <ul><li>Set RELIABILITY </li></ul><ul><li>Set HISTORY to KEEP_LAST or KEEP_ALL </li></ul><ul><ul><li>Test different depths </li></ul></ul><ul><li>Use Partitions </li></ul><ul><ul><li>Create several Partitions: </li></ul></ul><ul><ul><ul><li>E.g. by ChatRoomName </li></ul></ul></ul><ul><ul><li>Publish in your ChatRoom </li></ul></ul><ul><ul><li>Subscribe to one or more ChatRooms </li></ul></ul>
    146. 146. Exercise #4 (continued) Use content filters <ul><li>Edit the chat_subscriber.cxx </li></ul><ul><li>Add the lines: </li></ul><ul><li>DDSContentFilteredTopic *cftopic; </li></ul><ul><li>DDS_StringSeq filter_params; </li></ul><ul><li>filter_params.maximum(0); </li></ul><ul><li>cf T opic = participant- > </li></ul><ul><li>create_contentfilteredtopic( </li></ul><ul><li>&quot;Selected Chats&quot;, topic, </li></ul><ul><li>&quot;age > 4&quot;, filter_params); </li></ul><ul><li>Look of the call to create_datareader </li></ul><ul><ul><li>Replace “topic” with “cfTopic” in the paramater list. </li></ul></ul>
    147. 147. Exercise #3 (continued) Use Exclusive Ownership <ul><li>Set up in pairs edit the chat_publisher.cxx and use the same “name” for both of you </li></ul><ul><li>Re-run the publisher application you will see mixed messages. </li></ul><ul><li>Edit the chat_publisher.cxx </li></ul><ul><li>Before creating the data writer add the lines </li></ul><ul><ul><li>publisher->get_default_datawriter_qos(dwq); </li></ul></ul><ul><ul><li>d wq.ownership.kind = DDS_EXCLUSIVE_OWNERSHIP_QOS; </li></ul></ul><ul><ul><li>dwq.ownership_strength .value = 10; </li></ul></ul><ul><li>Replace DDS_DATAWRITER_QOS_DEFAULT with dwq In the create_datawriter() call </li></ul><ul><li>Edit the chat_subscriber.cxx </li></ul><ul><li>Before creating the data reader add the lines </li></ul><ul><ul><li>DDS_DataReaderQos drq; </li></ul></ul><ul><ul><li>subscriber->get_default_datareader_qos(drq); </li></ul></ul><ul><ul><li>drq.ownership = DDS_EXCLUSIVE_OWNERSHIP_QOS; </li></ul></ul><ul><li>Replace DDS_DATAWRITER_QOS_DEFAULT with drq in the create_datareader() call </li></ul>
    148. 148. Exercise <ul><li>Record the data </li></ul>
    149. 149. Exercise <ul><li>Use durable data with a persistence service </li></ul>
    150. 150. Exercise <ul><li>Isolate domains and transform data using the routing service </li></ul>
    151. 151. Summary <ul><li>Reduces software lifecycle costs </li></ul><ul><ul><li>Loose coupling </li></ul></ul><ul><ul><li>Replaces need for custom middleware in high ­ performance, real ­ time applications </li></ul></ul><ul><li>Reduces risk </li></ul><ul><ul><li>Standards-compliant API and wire protocol </li></ul></ul><ul><ul><li>Multiple implementations </li></ul></ul><ul><ul><li>Widely adopted </li></ul></ul><ul><li>Most widely proven and mature implementation </li></ul><ul><li>Highest performance </li></ul><ul><li>Industry-leading expertise and services capability </li></ul><ul><li>Free trial, research and IR&D licenses </li></ul><ul><li>Comprehensive platform support </li></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×