Scaling the Data Distribution Service to Global Networks
Upcoming SlideShare
Loading in...5
×
 

Scaling the Data Distribution Service to Global Networks

on

  • 1,824 views

This presentation explain how the OMG DDS Standard can be extended to address System of System deployments and scaling the applicability of DDS to Ultra-Large Scale Systems.

This presentation explain how the OMG DDS Standard can be extended to address System of System deployments and scaling the applicability of DDS to Ultra-Large Scale Systems.

Statistics

Views

Total Views
1,824
Views on SlideShare
1,784
Embed Views
40

Actions

Likes
0
Downloads
69
Comments
0

3 Embeds 40

http://www.prismtech.com 32
http://www.slideshare.net 7
http://www.apurva.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Scaling the Data Distribution Service to Global Networks Scaling the Data Distribution Service to Global Networks Presentation Transcript

    • Angelo Corsaro, Ph.D. PrismTech angelo.corsaro@prismtech.com Sara Tucci-Piergiovanni, Ph.D. University of Rome “La Sapienza” sara.tucci@dis.uniroma1.it OpenSplice DDS Delivering Performance, Openness, and Freedom Scaling the Data Distribution Service to Global Networks
    • State of the Art Scaling DDS to Global Networks Agenda Extensions to the Standard Concluding Remarks
    • Addressing Data Distribution Challenges DDS is standard designed to address the data-distribution challenges across The OMG DDS Standard a wide class of Defense and Aerospace Applications ‣ Introduced in 2004 to address the Data Distribution challenges faced by a wide class of Defense and Aerospace Applications Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Key requirement for the standard were its ability to deliver very high performance while seamlessly scaling from embedded to ultra- large-scale deployments ‣ Today recommended by key administration worldwide and widely adopted across several different application domains, such as, Automated Trading, Simulations, SCADA, Telemetry, etc. © 2009, PrismTech. All Rights Reserved
    • Addressing Data Distribution Challenges ‣ DDS is swiftly being adopted as the standard for providing System of Systems with ubiquitous and access to tactical data Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ DDS deployments on WAN and sizable MAN have also proven it fitness with Metropolitan/Wide Area Networks So... Why I am doing this presentation? © 2009, PrismTech. All Rights Reserved
    • Challenges DDS presents today some limitations with respect to addressing large scale geographical deployments, these are: Discovery ‣ Standard Discovery Protocol Requires Multicast ‣ Scalability Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Limited Control over Topics Visibility, e.g., choosing what to expose to the external world Communication ‣ Standard Communication Protocol Relies on UDP ‣ No Support for Compression ‣ Difficult NAT/Firewall traversal ‣ Sub-Optimal Data Distribution for WAN Scenario (limited exploitation of multicast) ‣ Scalability © 2009, PrismTech. All Rights Reserved
    • Challenges Publisher Publisher B B Subscriber Subscriber Proprietary Information - Distribution without Expressed Written Permission is Prohibited. m m A F A F Publisher J Publisher D C J X Z K E K Subscriber Y Subscriber DDS-1 Publisher Wan © 2009, PrismTech. All Rights Reserved
    • Challenges Publisher A Publisher B B Subscriber Subscriber Proprietary Information - Distribution without Expressed Written Permission is Prohibited. m m A F A F Publisher J Publisher D C J X Z K E K Subscriber Y Subscriber DDS-1 Publisher Wan © 2009, PrismTech. All Rights Reserved
    • State of the Art Scaling DDS to Global Networks Agenda Extensions to the Standard Concluding Remarks
    • Looking at Internet Apps. When looking at a solutions for extending DDS applicability to Internet Scale Systems, we can’t ignore architectures adopted by very successful Internet Applications. ‣ P2P application like KaZaa have shown to scale very well and to support millions of users while Proprietary Information - Distribution without Expressed Written Permission is Prohibited. effectively managing millions of subjects ‣ Application like Skype are connecting millions of people and managing in real-time they presence (discovery) © 2009, PrismTech. All Rights Reserved
    • Architecture for Internet Scale DDS ‣ DDS-Level Routers will be introduced that H 1,1,h establish a Peer-to-Peer overlay that N2,5 R N1,1 H 1,1,1 relays DDS data over the WAN SN R H 1,1,0 ‣ The Router provides a single access point for H 2,1,m Proprietary Information - Distribution without Expressed Written Permission is Prohibited. deciding what to expose to the external world SN H 1,0,k ‣ The Router can perform Topic Transformation, etc. SN ‣ A Network of Super-Nodes is used for R N2,0 H 1,0,1 N1,0 ensuring: H 2,0,i H 1,0,0 R R N2,4 ‣ Scalable (Global) Discovery R SN ‣ Subscription Management (when crossing the N2,1 boundaries of a System) SN H 2,1,n ‣ Support for establishing communication between Pub and Sub (perhaps via STUNT to circumvent N2,2 R R N2,3 H 2,1,j NAT/FIREWALL issues) H 2,1,k H 2,1,h © 2009, PrismTech. All Rights Reserved
    • DDS Router ‣ DDS Router communicates with the local DDS Domain by Multicast Ni R ‣ DDS Router are configured with the set of topics to be exposed along with potential transformations ‣ Router2Router communication is Proprietary Information - Distribution without Expressed Written Permission is Prohibited. R N carried on by RTPS over TCP (since k TCP has been tuned over years to work fine in WAN ‣ If required the Router takes care of compressing Data ‣ DDS Router Runs a “variation” or DDSI/RTPS ‣ Topic, Subscription and Publication are propagated to Super Nodes ‣ Information available into super nodes ensure that local reader/writer will be properly matched © 2009, PrismTech. All Rights Reserved
    • DDS Super-Nodes (SN) ‣ Super-Nodes store all the “discovery information” ‣ List of Topics available in the system R SN ‣ Associations between Topics and Router role (e.g. Pub or SN R Sub) Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ A distributed protocol is used to replicated data, SN eventually, on all replicas SN ‣ Router keep a list of well-known Super-Nodes to bootstrap discovery R R ‣ NOTE: Super-Nodes can also be used to support the implementation of STUNT-like protocols to facilitate NAT/Firewall trespassing © 2009, PrismTech. All Rights Reserved
    • How it Works {P(A),S(B),P(C)} {S(B), S(D),P(E)} DDS R1 SN1 SN4 R4 DDS Proprietary Information - Distribution without Expressed Written Permission is Prohibited. SN2 SN3 {S(A),P(B),P(C)} R2 R3 {S(A),P(D),S(E)} DDS DDS © 2009, PrismTech. All Rights Reserved
    • How it Works {P(A),S(B),P(C)} R1-{P(A),S(B),P(C)} {S(B), S(D),P(E)} DDS R1 SN1 SN4 R4 DDS R4-{S(B), S(D),P(E)} Proprietary Information - Distribution without Expressed Written Permission is Prohibited. SN2 SN3 R3-{S(A),P(D),S(E)} R2-{S(A),P(B),P(C)} {S(A),P(B),P(C)} R2 R3 {S(A),P(D),S(E)} DDS DDS © 2009, PrismTech. All Rights Reserved
    • How it Works {P(A),S(B),P(C)} {S(B), S(D),P(E)} DDS R1 SN1 SN4 R4 DDS R1-{P(A),S(B),P(C)} R2-{S(A),P(B),P(C)} R3-{S(A),P(D),P(E)} R4-{S(B), S(D),S(E)} Proprietary Information - Distribution without Expressed Written Permission is Prohibited. SN2 SN3 {S(A),P(B),P(C)} R2 R3 {S(A),P(D),S(E)} DDS DDS © 2009, PrismTech. All Rights Reserved
    • How it Works {P(A),S(B),P(C)} {S(B), S(D),P(E)} DDS R1 SN1 SN4 R4 DDS {P(A)} R1-{P(A),S(B),P(C)} R2-{S(A),P(B),P(C)} {P(A)} R3-{S(A),P(D),P(E)} {P(E)} R4-{S(B), S(D),S(E)} Proprietary Information - Distribution without Expressed Written Permission is Prohibited. SN2 SN3 {P(B)} {P(D)} {S(A),P(B),P(C)} {P(B)} R3 {S(A),P(D),S(E)} R2 DDS DDS © 2009, PrismTech. All Rights Reserved
    • How it Works {P(A),S(B),P(C)} {S(B), S(D),P(E)} DDS A R1 SN1 SN4 R4 DDS {P(A)} R1-{P(A),S(B),P(C)} R2-{S(A),P(B),P(C)} {P(A)} R3-{S(A),P(D),P(E)} {P(E)} R4-{S(B), S(D),S(E)} Proprietary Information - Distribution without Expressed Written Permission is Prohibited. SN2 SN3 {P(B)} {P(D)} {S(A),P(B),P(C)} {P(B)} R3 {S(A),P(D),S(E)} R2 A A A DDS DDS © 2009, PrismTech. All Rights Reserved
    • State of the Art Scaling DDS to Global Networks Agenda Extensions to the Standard Concluding Remarks
    • Extension to the DDS Standard ‣ What described in the previous slides can easily be implemented in an interoperable manner, exploiting the pre-designed extension points present in the DDS Standard Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Areas of extension include: ‣Discovery ‣DDSI/RTPS © 2009, PrismTech. All Rights Reserved
    • Discovery Extension 1‣ A new Discovery Protocol should be R added that allows to discover 1 SN SN relevant information (Topics, R 2 Subscriptions, Publications) via the Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Super-Nodes SN SN 2‣ A new protocol to be used between Super-Nodes should also be R standardized! R © 2009, PrismTech. All Rights Reserved
    • DDSI/RTPS Protocol xTensions DDSI/RTPS Should be extended DDS R1 SN1 SN4 R4 DDS to include: SN2 SN3 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Support for TCP/IP ‣Offloading reliability from DDSI to TCP R2 R3 whenever possible DDS DDS ‣ Compression © 2009, PrismTech. All Rights Reserved
    • State of the Art Scaling DDS to Global Networks Agenda Extensions to the Standard Concluding Remarks
    • Concluding Remarks ‣ The DDS Standard includes today some known limitation when trying to address Ultra-Large- Scale Systems ‣ These limitation mostly impact its scalability and efficiency Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ The good news is that exploiting available extension points in the standard it is possible to extend the DDS Architectural Style to scale to Ultra-Large-Scale Systems ‣ The Router/Super-Nodes approach builds over years of experience in the domain of Internet Scale applications, such as, KaZaa, Skype, etc. ‣ Thus, PrismTech will work toward update of the DDS Spec to include these extension points. © 2009, PrismTech. All Rights Reserved