• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Management High-level overview of the OMG Data Distribution Service (DDS)
 

Management High-level overview of the OMG Data Distribution Service (DDS)

on

  • 982 views

This document provides a good management-lever introduction to the Data-Distribution Service (DDS) technology and capabilities. It was prepared by the OMG at the request of the US Navy in order to ...

This document provides a good management-lever introduction to the Data-Distribution Service (DDS) technology and capabilities. It was prepared by the OMG at the request of the US Navy in order to educate on the data-centric software architectural principles of DDS and how they can help meet its agility and cost-control requirements.

Statistics

Views

Total Views
982
Views on SlideShare
982
Embed Views
0

Actions

Likes
0
Downloads
17
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Management High-level overview of the OMG Data Distribution Service (DDS) Management High-level overview of the OMG Data Distribution Service (DDS) Document Transcript

    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.Data Distribution Service(DDS) BriefStandards-Based Data-Centric Messagingfrom the Object Management Group (OMG)1 Executive SummaryU.S. Navy SPAWAR-ATL Engineering Competency requested the Object Management Group(OMG) to deliver a paper describing the technical capabilities of its Data Distribution Service(DDS). This paper complements an earlier one prepared in collaboration with several DDSvendors for the Office of the Secretary of Defense (OSD), which describes DDS adoption inmilitary and commercial applications. That paper, “The Data Distribution Service: ReducingCost through Agile Integration,” is hosted online by the UAS Control Segment (UCS) program1.Navy decision makers are being asked to respond more quickly on the basis of increasingvolumes of information. This information is sourced from multiple systems of systems executingon heterogeneous platforms and networks. To face this challenge, the Navy needs to increase itsleverage from proven technology and increase the integration between existing systems. Navyleadership has embraced these requirements with mandates for Open Architecture integrationbased on open standards and off-the-shelf products. These principles help the Navy to align itstechnology roadmap with broader industry directions and to empower competitive markets thatreduce vendor lock-in and drive down costs.The OMG has long been a favored venue for the collaboration of Navy interests with industrythought leaders around the promulgation of relevant standards. Navy Surface Warfare Center(NSWC), Navy Undersea Warfare Center (NUWC), Boeing, Lockheed Martin, GeneralDynamics, Northrop Grumman, and other U.S. and allied organizations are all activeparticipants. DDS technology in particular has been rapidly and widely adopted by theseorganizations. This adoption has been driven by the ease and flexibility with which it can be usedto develop, maintain, and integrate complex systems while maintaining strong performance andgovernance. DDS is supported by a large vendor community and has been called out in U.S.DoD guidance from Navy Open Architecture, DISA, NESI, UCS, and other U.S. and alliedorganizations. This guidance has been born out in hundreds of defense and civilian programs,and DDS implementations exist at Technology Readiness Level (TRL) 9.This paper describes the software architectural principles that can help the Navy to meet itsagility and cost-control requirements. It further describes how DDS technology in particularsupports this architecture—not just hypothetically but in real-world systems—in unique andpowerful ways.1 See http://www.ucsarchitecture.org/downloads/DDS%20Exec%20Brief%20v20l-public.pdf. —1 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.Table of Contents1   Executive Summary.................................................................................................................1  2   Step 1: System Architecture ...................................................................................................2   2.1   Benefits.................................................................................................................................................... 4   2.2   Challenges  Facing  Traditional  Implementations ...................................................................... 5   2.3   An  Improved  Approach  to  Managing  Data-­Centricity.............................................................. 7  3   Step 2: Supporting the Architecture ......................................................................................8   3.1   Data-­Centric  Messaging ..................................................................................................................... 8   3.2   DDS............................................................................................................................................................ 9  4   Step 3: Instantiating the Architecture .................................................................................11   4.1   Topology ...............................................................................................................................................12   4.2   Disadvantaged  Networks ................................................................................................................13   4.3   Scalability .............................................................................................................................................14   4.4   Security..................................................................................................................................................15  5   Conclusion ..............................................................................................................................17  6   Appendix: Technology Comparison ....................................................................................17   6.1   Specification  Comparison................................................................................................................18   6.2   Vendor  Comparison ..........................................................................................................................19  2 Step 1: System ArchitectureIndustry has grappled for over a decadewith the problem of deploying andmaintaining groups of applications thaton the one hand need to integrate withone another, but at the same time needto remain decoupled, so that they canjoin and leave the networkdynamically, and so that they canevolve according to their independentlife cycles.The architecture they have followed isdata-centric. Data-centric architectureis often instantiated in so-called “n-layer” or “n-tier” enterprise systems.Stateful data is maintained byinfrastructure, and applications remain Figure 1—Schematic of a data-centric, n-layer architecture —2 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.stateless. Applications do not communicate directly with one another; instead their interactionsare mediated by the data and expressed in terms of changes to that data.This architecture is described as “data-centric” because it organizes applications and theirinteractions in terms of stateful data rather than in terms of operations to be performed. Itconforms to the following principles: 1. The structure, changes, and motion of stateful data must be well defined and discoverable, both for and by humans as well as automation. What do we mean by “state”? State consists of the information about the application, the system, and the external world that an application needs in order to interpret events correctly. For example, suppose there is an announcement, “the score is four to three”. What game is being played? Who are the players? Which one of them has four points and which three? The answers to these questions comprise the state that is necessary to understand the message. 2. State must be managed by the infrastructure, and applications must be stateless. (This is also a recognized SOA pattern called “State Repository”2.) 3. State must be accessed and manipulated by a set of uniform operations. What do we mean by “operations”? Operations express attempts to change the state. In a data-centric architecture, the operations are uniform3. These operations are often referred to by the acronym CRUD, for Create, Read, Update, and Delete, because most supporting technologies define parallels for these concepts4.Multiple technologies directly support this architecture, including relational databases, RESTfulweb services5, and OMG Data Distribution Service.Consider a hypothetical distributed game of chess. • A non-data-centric implementation might assume that all parties understand the initial layout of the game. Then players would send their moves to one another—“pawn 4 to c3” for example. Such an implementation further assumes that each recipient has out-of-band access to its own copy of the current state of the board so that it can change that state accordingly and that each player receives every message so that different copies don’t get out of synch with one another. • A data-centric implementation would present a common picture of the board to authorized parties and allow them to query and modify it—to not only say that pawn 4 should move to c3, but also to ask what is at c3 beforehand. This state is maintained by the infrastructure; applications do not need to maintain their own copies. And applications act of this state independently of which other applications may or may not be2 See http://soapatterns.org/state_repository.php for an introduction to this pattern.3 Computer science uses the term “polymorphism” to describe a situation in which a common interface may be usedto access different kinds of resources. Polymorphism helps software fit together like puzzle pieces: a component thatunderstands a particular interface can communicate with any other component that understands the same interface.Data-centric architecture takes polymorphism to its logical conclusion: all state shares a single common set ofoperations.4 In SQL, the uniform operations are INSERT, SELECT, UPDATE, and DELETE. In HTTP, they are POST, GET,PUT, and DELETE. In DDS, they are write, read, dispose, and unregister.5 For a brief introduction to the Representation State Transfer (REST) pattern, seehttp://en.wikipedia.org/wiki/Representational_State_Transfer. —3 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved. observing. (Note that a distributed infrastructure may communicate within itself using messages, but applications are written to a higher level of abstraction.)The following sections describe the benefits of a data-centric approach, the challenges faced intraditional implementations of the approach, and how data-centric messaging technologies likeDDS overcome those challenges.2.1 BenefitsThe benefits of data-centricity derive from the loose coupling between applications. They do notcommunicate directly; instead, one modifies a given data object, and another observes thechange.2.1.1 Reliability and ResiliencyApplications have the ability to obtain from the infrastructure the current state of the world inwhich they’re interested. Therefore, if an application fails and has to be restarted, it can recoververy quickly. In contrast, if the application is not stateless, a restart is expensive and risky.Message senders must store all messages that they sent during the failure and replay them uponreconnection, because if the recovering application misses even a single message, its state will beout of synch, and it will act on incorrect information. If message rates are high relative to therecovery time, storing these messages will become infeasible.For example, consider an intermittent network link, such as a satellite or line-of-sight radio.Applications separated by such a link and architected in a data-centric way will be able to resumecommunication by sending only the differences between the relevant pre-disconnection state andthe current post-reconnection state. This data volume is bounded in size and often much less thanthe sum of all messages that might have been exchanged in the mean time.2.1.2 Integration ComplexityIt is best, when integrating multiple 90  elements (applications or entiresubsystems), to avoid mediating 80   Lingua  every element to every other. Such a 70   Franca  design requires (n * (n – 1)) 60   Pattern  integrations per n elements—the 50  complexity, effort, and cost of the 40  integration increase with the square Point-­‐ 30  of the number of elements. Instead, to-­‐employ the well-known Lingua 20   Point  Franca architectural pattern: design a 10   Integns  normalized model, and integrate each 0  element with that model. Each 1   2   3   4   5   6   7   8   9  element need only be integrated once, Figure 2—Relative complexity of point-to-point integration vs.and the complexity of the integration applying the Lingua Franca Pattern, in the worst casetherefore increases linearly with thenumber of elements. —4 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.This pattern applies to state as well as to behavior. A set of point-to-point integrations in whichneither state nor operations is normalized will consequently scale even worse than n2. Thisadditional complexity can be reclaimed by normalizing the programming interface and themessage set with an ESB; complexity growth returns to n2. A data-centric architecture takes thenext step: operations are uniform, and state is normalized. The growth in the complexity istherefore linear.2.1.3 System Evolution CostBecause applications have no awareness of one another, they can be added to or removed fromthe system with much lower impact. Changes are limited in scope—replacing one componentwith another does not require that all other components be updated as well.Consider again the chess example above. What if I want to add a new application to the game—perhaps to provide move recommendations, a turn timer, a GUI display, or other capability? Anyof these can be built based on the state of the board that I already have, and no other applicationthat uses that state needs to know or care that it’s being used for one more thing. A stateless ESBcannot provide the same benefit, because it provides applications with no ability to query thecurrent state of the board—it deals only with stateless messages, not with stateful data.2.1.4 Acquisition FlexibilityA standards-based data-centric system provides interoperability not only at the level of messageson the network but also at the level of an operational picture. This higher level of interoperabilitydecreases lock-in not only to middleware vendors but to integrators as well, because theintegration is fundamentally governed. The information about which information is to beexchanged under which conditions is captured in explicit configuration, not buried in applicationcode, and is accessible to any authorized vendor using industry-standard tools.2.2 Challenges Facing Traditional ImplementationsBefore the advent of data-centric messaging, data-centric designs were primarily based onproprietary and/or web-based protocols connecting “client” applications to relational databases.Such implementations remain valid for many systems, but they also face significant challenges—challenges that tempt some applications to abandon the architecture. This section describes someof those challenges and the unfortunate result.2.2.1 Challenges: Scalability, Reliability, Latency, and ManagementChallenge #1: Vulnerabilities of shared infrastructure. Shared infrastructure, such asdatabases and servers, can become a performance bottleneck and single point of failure.Challenge #2: Synchronization of federated state. All applications may not have access to asingle common state repository. In such cases, it’s necessary to maintain copies of the relevantstate in multiple locations and to synchronize them independently of the applications. This is adifficult task that not all teams are equipped to tackle.Challenge #3: Data access latency. Messaging between the application that wants a piece ofstate and the repository that has it can be slow. Response times may be acceptable in cases wherenothing changes faster than a human can process it—a person with a web browser is a good —5 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.example. But for machine-to-machine interactions, or machine-to-human interactions, thislatency proves much too high.2.2.2 Typical Result: Brittle, Unmanageable SystemsUnfortunately, too often the challenges above lead architects to abandon data-centricity for adhoc approaches.Figure 3—Tangled communication resulting from the application of messaging technology without agoverning architecture • Rather than allowing their actions to be mediated by the data, applications send messages directly to one another. They may use abstractions like messages queues, but these patterns remain fundamentally point-to-point. • Rather than relying on the state repository to manage their data, every application maintains its own state.In effect, system-level state management is neglected. Does our experience lead us to believethat when we don’t design something, it will nevertheless work well? The result instead issystems that are brittle and difficult to manage.Why brittle? • Applications are coupled to one another, so they can’t come and go, or evolve over time, independently. Implications: Decreased operational flexibility and increased costs for maintenance and integration. • State is coupled to individual applications, not maintained independently, so new applications can’t reuse the state that’s already there, and existing applications can’t recover their state if they restart or relocate on the network. Implications: Decreased reliability and resiliency and increased cost to develop and integrate incremental functionality. —6 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.Why unmanageable? • Data structure is ad hoc, so it’s impossible to detect whether a piece of information is malformed until a particular application tries to access it. By then, it’s too late to avoid and hard to debug. Implication: Error detection will occur later in the process, when it’s more expensive to fix and has a bigger impact on schedules. • Data movement around the network is ad hoc, so as each application maintains its own view of state, these views can get out of synch. Applications act on incorrect or obsolete information and can’t respond in a timely way to new information. Implication: Decreased reliability. • Data change is ad hoc, so making sure that the right parties see the right changes in an acceptable amount of time is the responsibility of every application—or else everything has to be sent to a single central party on the network, and that one has to know everything and never fail. Implications: Increased upfront cost due to duplicated application-development effort, increased maintenance costs due to inter-application coupling, and decreased reliability if a single point of failure is introduced.2.3 An Improved Approach to Managing Data-CentricityThe challenges described above can be solved in a more scalable way while retaining the data-centric architecture. This preferred approach relies on data-centric messaging, which is describedin section 3.1 below.Figure 4—Data-centric messaging improves scalability of data-centric architectureChallenge #1: Vulnerabilities of shared infrastructure. Federate state management to whereit’s needed. Each portion of the network has independent access to exactly the state it needs atthat moment and no more. This is the logical conclusion of server federation: the more broadlyyou federate, the smaller the burden is on any one party; each can remain lightweight. And thereare no longer any single points that can take out a whole network. —7 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.Challenge #2: Synchronization of federated state. This burden should not be on theapplications; it should be on a best-in-class infrastructure. Point-to-point messaging betweenapplications is replaced by data-centric publish-subscribe messaging within that infrastructure.Instead of full consistency, it seeks eventual consistency. That guarantee is easier to maintainunder challenging network conditions, and the implementation can be orders of magnitude faster.Challenge #3: Data access latency. Because we’re employing a solution based on distributedstate with eventual consistency, we can treat large-scale, long-term persistence as a separateconcern from application access. We can eliminate most databases and instead place lightweightin-memory caches very close to each application—on the same node, or even within the sameprocess—to maximally reduce this latency. Meanwhile, we can place high-availabilitypersistence stores on appropriately provisioned nodes elsewhere on the network.3 Step 2: Supporting the ArchitectureTo support this architecture, we need technology that can govern data contents, as a databasecan, as well as governing communication flows within complex networks, as messaging busescan. And it must go beyond conventional message buses—it must tie messages back to theunderlying data objects and formally describe how those objects will be synchronized across thenetwork as they change.3.1 Data-Centric MessagingData-centric messaging is the application of messaging to the distribution, access, andmanipulation of stateful data. Data-centric messaging supports data-centricity for data in motion,just as a relational database does for data at rest. The vendor community has been supportingsuch technology, called data-centric messaging, for over ten years. This section describes thetechnology generally.As described in section 2.3 above, an architecturethat employs data-centric messaging offerssignificant benefits over one instantiated based onsolely on the basis of databases or solely on thebasis of non-data-centric messaging. • Reduced integration and maintenance costs, as with any data-centric technology. • Improved performance. Where a web service connected to a database might deliver a dozen data updates per second, an Figuredata. database stores data. A data bus moves 5—A efficient data-centric messaging bus can deliver tens or even hundreds of thousands. • Improved scalability. Where centralized infrastructure might support a few connected applications, a decentralized data bus can support hundreds or thousands—on less hardware. • Improved reliability and resiliency. Single points of failure have been eliminated. • Improved manageability. The infrastructure enforces explicit contracts on the structure of data, how it moves across the network, and how it changes over time. And if and when —8 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved. these contracts are violated, it prevents incorrect communication, takes mitigating action, and notifies the relevant applications.Between 2001 and 2002, several of these vendors came together at the OMG to begin work onstandardizing data-centric messaging technology. The result, the Data Distribution Service(DDS), is the subject of the next section.3.2 DDSThe Data Distribution Service is the standard for data-centric messaging. Adopted by the OMGin 20036, DDS now comprises a comprehensive suite of layered specifications. In particular, it isthe only widely deployed standards-based middleware technology to define code portability aswell as on-the-network interoperability. • DDS itself, which defines the behavior of the bus itself as well as programming interfaces in several languages. Thirty-six companies voted to adopt the original specification, including Ericsson, Fujitsu, IBM, Lockheed Martin, MITRE, Mercury Computer Systems, Nokia, Objective Interface Systems, Oracle, PrismTech, RTI, Rockwell Collins, and THALES. The Navy Surface Warfare Center (NSWC) played a significant supporting role. Today, approximately ten vendors support the specification. • A network protocol, called Real-Time Publish-Subscribe (DDS-RTPS), which provides interoperability among DDS implementations. This specification became available through the OMG in 2008 at version 2.0. (It was based on an earlier specification, RTPS 1.0, which was standardized through the IEC in 2004.) The current version, 2.1, became available in January 2009. Most vendors now support this protocol, and interoperability has been publicly demonstrated at a number of OMG-hosted events. • Integration with UML models to bridge the gap from design to implementation. This UML profile was adopted in 2008. • Enhancements to the type system to address system evolution and more flexible data views. This specification was adopted in 2010 and is currently in the process of finalization and implementation with the involvement of multiple vendors. Founded in 1989, OMG is now the • Improvements to the C++ and Java largest and longest-standing not-for- programming interfaces to enhance profit, open-membership consortium portability, performance, and ease of use. developing and maintains computer These specifications were adopted in 2010 industry specifications with more than and are currently in the process of 470 member companies. It is finalization and implementation with the continuously evolving to remain current involvement of multiple vendors. while retaining a position of thought • etc. leadership.DDS continues to define one of the most active All OMG specifications are freelycommunities within the OMG. In addition to available to the public fromongoing direct collaboration among member www.omg.org.organizations, the OMG hosts quarterly in-person6 OMG issued an RFP for the definition of a data-centric publish-subscribe messaging bus in late 2001. Initialproposals were received from several vendors in 2002. The first version of the specification was preliminarilyadopted in 2003 and finalized in 2004. The current version, 1.2, became available in January 2007. —9 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.technical meetings. OMG also hosts an annual workshop on time-critical systems that in recentyears has become increasingly focused on DDS technology. And the ecosystem continues togrow, with new vendors joining the community and specifications for security enhancements andweb connectivity in progress.3.2.1 AdoptionDDS has been adopted and/or mandated by many military and civilian organizations.DDS plays a major role in naval combat systems in the U.S. and worldwide. It has been designedinto the Aegis, SSDS, and DDG 1000 programs and is deployed by allied navies, including thoseof Germany, Japan, the Netherland, and over a dozen more. DDS has been adopted by thefollowing organizations: • U.S. Navy—Open Architecture, FORCEnet • Defense Information Systems Agency (DISA)—Mandated standard within the DoD Information Technology Standards and Profile Registry (DISR) • U.S. Air Force, Navy—Net-centric Enterprise Solutions for Interoperability (NESI) • U.S. Army, OSD—UAS Control Segment (UCS) • U.S. intelligence community • UK Ministry of Defence—Generic Vehicle Architecture, an interoperable open architecture for unmanned vehiclesDDS is also used commercially in a number of industries, including communication,transportation, financial services, SCADA, industrial automation, agriculture, power generation,air traffic control, mobile asset tracking, and medicine. A number of universities worldwide areusing DDS in active research projects, including MIT, Carnegie Mellon University, andVanderbilt University in the U.S. and ETH Zurich, Technical University of Munich, and KoreaAdvanced Institute of Science and Technology internationally.The following sections describe some of the capabilities of DDS. These are not capabilitiesspecific to a particular vendor; they are specified within the DDS standard. —10 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.3.2.2 Managed Data DistributionNon-data-centric messaging technologies just provide Standards-Based Governanceways to send messages from A to B. Architects must Data structuredevelop their own idioms on top. Data value historyDDS is different. Like HTTP, DDS uses the technique ofmessaging to support system architecture. Because the Rate of data value changedata model is clear and explicit rather than implicit in Data ordering and coherencystatic code, DDS can define, propagate, and govern dataflows more comprehensively and more efficiently. DDS Lifespan/TTLprovides: Network partitions • Formal data design and integration to avoid lock- Resource utilization in to vendors and integrators • Strong enforcement of data structure and quality Priority of service to make propagation more efficient and Reliability catch errors sooner • Comprehensive status monitoring to detect and Durability, persistence, and high mitigate potential problems at run time availability • Flexible security with a comprehensive road map Fault tolerance and fail-over3.2.3 Flexible Deployment Filters based on contents and timeDDS is the only widely deployed messaging technology Publication/subscription matchingto scale from embedded systems to data centers to globalnetworks. Connection health • DDS implementations support both peer-to-peer and brokered message-flow topologies, which can be combined as needed for local, wide-area, and disadvantaged network environments. See section 4.1, “Topology”, below for more information. • DDS is interoperable across multiple programming languages, real-time and non-real- time constraints, and enterprise and embedded platforms.DDS is compatible with enterprise environments. In addition to support for higher-levellanguages like Java, it has an API that is similar to other messaging technologies7. Vendors alsoprovide a variety of connectors to other standards-based messaging and storage technologies,including JMS, databases, and web services.4 Step 3: Instantiating the ArchitectureThis section applies the architectural principles described above, and the technologies thatsupport them, to describe the construction of flexible, performant, and affordable systems. Itfocuses on three areas: topology, scalability, and security.7 OMG is the standards body responsible for both the DDS and CORBA specifications. However, these twotechnologies work differently and do not depend on one another. —11 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.4.1 TopologyPeer-to-peer communication is a fundamental building block of any network communication.Other topologies—such as brokeredcommunication—are constructed from it. Forexample, a network of data producer communicatingwith a data consumer by way of a broker consists ofthree peers, one in between the other two. It sohappens that the middle peer is typically provided by Figure 6—Peer-to-peer communication is athe messaging vendor and provides services to the fundamental building blockother two peers.4.1.1 Composing Brokered NetworksDDS specifications are defined peer-to-peer8 in order to provide implementers with maximumflexibility. Most vendors take advantage of this and support peer-to-peer communication as anoption within their products. However, other DDS vendors support only brokered configurations,while some support peer-to-peer communication but also ship message brokers, so that users cancompose their systems however is most appropriate.Figure 7—A brokered network is composed of multiple peer-to-peer connections. However, whether that isreflected in a given messaging implementation varies.Most vendors of traditional non-data-centric messaging technologies support only brokeredconfigurations.4.1.2 Composing Local and Global NetworksDifferent networks have different characteristics and requirements. • Local networks support deterministic, low-latency communication. They can often take advantage of efficient IP multicast. Applications running here may also be more trusted. • Wide-area networks have higher latencies and may or may not support multicast. They may route different transport protocols (e.g. TCP but not UDP). Applications connected across such networks may be less trusted than those running on a LAN. • Disadvantaged wireless networks have significantly different reliability and performance characteristics. Applications running here may be the least trusted, given that wireless connections may be easier to intercept than wired connections. • Any one of these physical networks, or a logical “subnet” within it, may represent an independent security domain or subsystem.8 This situation is not unlike that of other messaging technologies. For example, the non-data-centric AdvancedMessage Queuing Protocol (AMQP) is also specified peer-to-peer. —12 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.Connecting these heterogeneous environments requires mediation—a broker to filter andtransform the data, cleanse it to meet IA requirements, and ultimately forward it to the other side.However, brokers may not be needed within the LAN, because it is a more controlledenvironment. These opportunities and constraints lead us to design networks such as is depictedin the following figure.Figure 8—A composite network taking advantage of peer-to-peer communication on the LAN and brokeredcommunication across the WANSuch networks can take advantage of peer-to-peer performance and resilience on the LAN whilemediating data and enforcing security policies at subsystem or network boundaries with datarouters. Persistence and other services can be deployed and relocated as appropriate.4.2 Disadvantaged NetworksIt is never acceptable for applications to act upon obsolete information. When networks aredisconnected, intermittent, or limited in their bandwidth (DIL), this challenge is even moresignificant. Messaging technologies have the opportunity to either mitigate or exacerbate it. Thefollowing are a few of the factors to consider: • Data compactness: The more limited the network’s bandwidth, the more important it is that the messaging layer does not bloat its payloads with an inefficient data representation. Larger payloads also take longer to send, increasing the chance that a network drop will hit in the middle, preventing successful transmission. System designers sometimes rely on XML to provide data transparency; unfortunately, XML can be bulky. DDS does support XML payloads but also provides similar benefits using a very compact binary data representation. • Protocol responsiveness: The protocol must recover from losses and disconnections quickly, and while the network is connected, it must use it efficiently. TCP—and protocols layered on top of it—suffers in this area. While it can provide excellent performance when connectivity is good, it can be slow to respond to changing network conditions. And its head-of-line blocking behavior and global timeouts can cause multiple message streams to halt delivery for an extended period if any one of them suffers a transitory loss of synchronization. —13 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved. The DDS-RTPS protocol can be layered on top of TCP. However, most typically it runs on top of UDP, where it provides reliability and independent quality-of-service control on a per-stream basis, avoiding cross-stream interference and extended blocking. • Bounded resource use: A typical durable messaging technology operating over an intermittent link must store every message that was sent while the link was down and replay those messages when the link is restored. If the link goes down for an extended period, the resources needed to store these messages can grow in an unbounded fashion. And upon reconnection, replaying those messages will take an increasing amount of time. At some point, the relationship between the data rate, the available bandwidth, and the likelihood of network disconnection will reach a tipping point, at which it will be impossible to replay the messages cached from the previous disconnection before the next disconnection occurs. At this point, the network, while connected, will be continually full, but receiving applications will be permanently behind. A data-centric message design eliminates dependencies between messages, allowing durable implementations to cache safely only a bounded number of messages rather than all that were ever sent, reducing both local resource use as well as network bandwidth requirements. DDS can express such a design directly using standard QoS policies, and the DDS-RTPS/UDP protocol stack can support these policies all the way down to the network level. • Graceful degradation: In some cases, if it’s not possible to deliver every message, it’s best to deliver none of them. In other cases, graceful degradation is more desirable: deliver as much data as possible within the reliability and timeliness requirements of the applications involved, but allow other messages to be dropped in the interest of allowing those applications to continue processing the most up-to-date information. Paradoxically, a middleware that expects to be able to deliver everything over a network that can’t fulfill that expectation will often end up delivering very little—and at great expense, as it continually floods the network with negative acknowledgements and resends of messages that were previously dropped by the network. A data-centric message design enables graceful degradation by eliminating dependencies among messages in the same stream. DDS provides standard QoS policies and a flexible protocol that allow this design to be realized in a portable and interoperable way. These policies allow administrators to specify the strict reliability guarantees some message streams require. But they also allow more relaxed contracts when and where appropriate, including dropping unacknowledged messages that have been superseded by a certain number of subsequent messages, down-sampling rapid-fire streams, and so on.4.3 ScalabilityDDS networks such as that shown in Figure 8 above enable scalability in each portion of theoverall system.Peers are lightweight. Each application participating in the local network needs to keep only anin-memory cache of the recent updates to the data it is publishing or has subscribed to. It doesnot need a traditional database or persistent storage. Furthermore, a single UDP socket cancommunicate with an arbitrary number of remote peers, so IP resources are kept to a minimum. —14 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.These efficiencies allow DDS implementations to run in embedded systems in addition toenterprise-class workstations and servers.Peer-to-peer networks are reliable and efficient. Peer-to-peer communication avoids artificialbottlenecks and single points of failure. The DDS on-the-network format in particular is designedto be highly compact, and multicast communication is supported (though not required).Thousands of applications can communicate in a single network, exchanging hundreds ofthousands of data updates per second per producer-consumer pair, or many millions in aggregate.(These same properties make DDS well suited for disadvantaged, limited-bandwidth, and/orintermittent links.)Wide-area networks require flexibility. Unlike TCP-based protocols, DDS-RTPS offers per-channel quality-of-service control and avoids head-of-line blocking. These characteristicsimprove performance and make the infrastructure’s performance more predictable, even overchallenging links. When connecting local and wide-area nodes, a broker can forward data andshape traffic appropriately for each side.4.4 SecuritySecure messaging requires a comprehensive approach. • Implementations must run on secure platforms to prevent errant code from exploiting the network. • Applications must communicate over secure transports to prevent unauthorized parties from snooping their data. • Data must remain confidential even when passing through brokers, persistence services, and other infrastructure components. • Data must be properly attributed such that recipients can understand from whence it came. • Middleware must enforce system-level access-control policies to limit the production and consumption of data to authorized parties.OMG has published a complete security roadmap for DDS. This document describes where thespecification is currently and where it is going.4.4.1 DDS Security—In ProductionStandard interception APIs for policy enforcement. The DDS API notifies applications whenremote peers attempt to communicate with them—to join the same network, publish data to theapplication, or subscribe to data from the application. These notifications carry with themmetadata about the remote application, publication, or subscription—including, if desired,security credentials. Applications then have the opportunity to reject communication withunauthorized peers. —15 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.Figure 9—DDS access controlVendor-supplied secure transport. OMG is currently working with vendors on thespecification of an interoperable secure transport for DDS (see below). In the mean time, securetransport support based on the IETF-standard TLS9 protocol (over TCP, or DTLS over UDP) isavailable from the vendor community.Deep packet inspection. In DDS, data types are discoverable and data formats are standardized,allowing data updates to be introspected at run time by the infrastructure. This can be donewithout or without the use of XML—there is no need to give up the compactness or performanceof binary data.Secure operating-system support. DDS implementations run on secure enterprise platformssuch as SE Linux as well as secure partitioned operating systems such as VxWorks MILS.4.4.2 DDS Security—In ProgressOMG is currently working with vendors on a comprehensive security specification that will takeDDS to the forefront of middleware and messaging technologies. This specification will addressthe following scope: • Interoperable secure transport, such as TLS, for security in transit • Data-level tagging, signing, and encrypting for non-repudiation and confidentiality, even as data updates traverse brokers or are persisted to disk • Authentication and authorization at the domain (network) and topic level to enforce system access-control policies • A richer set of pluggable service-provider interfaces to allow users to integrate security- enforcement mechanisms across multiple platforms and technologies in a system— without locking themselves into a vendor-proprietary stackOMG issued the RFP for this specification late last year and is currently processing initialproposals. An adopted specification is expected late this year or early next year.9 Transport-Layer Security (TLS) is the current name of the specification previously known as Secure Socket Layer(SSL). —16 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.Figure 10—Overview of the in-progress DDS security specification (source: OMG DDS Security RFP)5 ConclusionInteroperability based on open standards fosters the growth of a competitive marketplace toempower innovation while driving down costs. A monolithic “common” infrastructure by itselfcan achieve neither of those ends. The customer community, the vendor community, andindependent standards bodies like OMG must work together.This is what OMG, the Navy, its integrators, and its supporting vendors have done around DDStechnology. DDS applies long-proven architectural principles in new ways to enable rapiddevelopment and insertion of new capability, lower-risk system integration and evolution, andmore reliable operations. At the same time, multiple vendors actively compete for Navybusiness, lowering acquisition and life-cycle costs.6 Appendix: Technology ComparisonThe following tables compare specifications and vendor implementations of severaltechnologies. —17 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved.6.1 Specification Comparison DDS WS-N AMQPGoverning Body OMG W3C AMQP Working Independent standards body; Independent Group standards body; Industry consortium; Open membership under published rules Open membership Ad hoc membership under published rulesParticipation 12+ (DDS), 12+ (C4I, A&D specs Unknown 12+; atop DDS); Weekly conference Quarterly in-person calls (minutes available) meetings (minutes available to OMG members)Vendors About 6+ About 3 About 4Primary Defense (prod’n) Defense (prod’n) Finance (prod’n)Adoption Communication (prod’n) Defense (dev’t) SCADA/Industrial (prod’n) Transportation (unknown) Transportation (prod’n) Finance (prod’n)Integration Data-Centric Message- Message-CentricArchitecture CentricPortable API DDS 1.2 (Java, C++, C); WSDL-based None; JMS 1.1 (Java; vendor-specific) JMS 1.1 (Java; vendor- specific)Data/Message Formal Formal Informal or formalDefinition W3C XSD or OMG IDL W3C XSD AMQP-specific language —18 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved. DDS WS-N AMQPInteroperable Real-Time Publish- SOAP 1.1, 1.2 AMQP 1-0r0Protocol Subscribe 2.1 Transport: Pre-v1 release candidate; Transport: UDP ucast & mcast; HTTP/TCP Transport: TCP Transport: TCP (in progress + vendor-specific)Throughput 10Ks–100Ks msgs/s10 10s msgs/s 100s–1Ks msgs/s11(one-to-one)Security AuthN/AuthZ interception Secure Secure transport; pts; transport; AuthN/AuthZ Improved AuthN/AuthZ (in Data signing, (vendor-specific) progress); encryption Secure transport (in progress + vendor-specific); Data tagging, signing, encryption (in progress)6.2 Vendor Comparison IBM PrismTech Red Hat NCES (R3) (OpenSplice) RTI (MRG-M) (JUM)API JMS 1.1 DDS 1.2 (Java, DDS 1.2 (Java, Vendor- WSDL- (Java) C, C++, .NET) C, C++, .NET, specific (Java, based Ada); Python, C++, .NET); JMS 1.1 (Java) JMS 1.1 (Java)10 Source: http://www.rti.com/products/dds/benchmarks-cpp-linux.html#MSGRATE. This data is presented for one-to-one connections.11 Source: “Reference Architecture Red Hat Enterprise MRG Messaging” whitepaper linked fromhttp://www.redhat.com/mrg/messaging/. This data is presented in aggregate across 60 applications. The testmethodology describes how to derive one-to-one measurements from it. —19 of 20—
    • Copyright 2011, Object Management Group (OMG). All Rights Reserved. IBM PrismTech Red Hat NCES (R3) (OpenSplice) RTI (MRG-M) (JUM)Protocol Real-Time Real-Time Real-Time AMQP 0-10 WS-N Publish- Publish- Publish- Pre-v1 Subscribe Subscribe 2.1; Subscribe 2.1 RT- 2.1 Networking (vendor-specific)Enterprise Yes Yes Yes Yes NoSupportOptionLicense Comm’l Open Source; Comm’l Comm’l Comm’l Comm’l Free for eval, Based on open- IR&D; source Apache (free for eval) qpid Source avail for purchaseTopology Brokered Peer-to-peer; Peer-to-peer; Brokered Brokered Brokered BrokeredRedundancy Unknown Hot producer Hot Clustered Unknown fail-over producer brokers Per DDS fail-over Vendor-specific Ownership spec Per DDS Ownership specPersistence Broker Per-node Broker; Broker BrokerLocation daemon Standalone serviceData Caching None Yes Yes None None In-memory; In-memory; Persistent Persistent —20 of 20—