0
OpenSplice DDS
                                       Delivering Performance, Openness, and Freedom




                  ...
OpenSplice DDS
Delivering Performance, Openness, and Freedom




                               Challenges
Financial Firms
Trends
‣ Direct connection to exchanges is increasingly chosen
  over data aggregators to reduce trading l...
Exchanges
Trends
‣ More and more firms connecting directly
‣ Increasing number of sources providing the same quotes
‣ Comp...
OPRA Predictions
                              OPRA Market Data Rates Forecasts                                           ...
Trading with Pareto
                                                                        ‣ Usually 20% of the trades
  ...
OpenSplice DDS
Delivering Performance, Openness, and Freedom




             The Data Distribution Service
             f...
Addressing Data Distribution Challenges
                                                          DDS is standard designed...
The OMG Data Distribution Service (DDS)

DDS v1.2 API Standard
‣ Language Independent, OS and HW architecture             ...
Scope of Standardization
      AMQP                                              OMG DDS
                                 ...
Scope of Standardization
      AMQP                                              OMG DDS
                                 ...
Scope of Standardization
      AMQP                                              OMG DDS
                                 ...
Scope of Standardization
      AMQP                                              OMG DDS
                                 ...
Architectural Style


                      AMQP                                                                      OMG ...
Key Differences in Summary
              DDS                                                                         AMQP
...
OpenSplice DDS
Delivering Performance, Openness, and Freedom




                    DDS and Open Source
OpenSplice DDS
‣ Product reorganized into Editions
 ‣ Community Edition
                                                  ...
Community Edition
Features                                          Licensing
‣ OMG DDS v1.2 DCPS                         ...
Performance on Commodity HW
                   Throughput                                                              -./...
OpenSplice DDS Adoption
Defense
‣ Combat Management Systems
‣ Tactical Systems
‣ Drones (land/air)
‣ Communication Systems...
OpenSplice DDS Adoption
Transportation
‣ Air Traffic Management/Control
‣ Metropolitan Traffic Management
‣ Underground Me...
Some OpenSplice DDS Users




                                                    !"#$%&'()('*+'##,-+.+/($$,012"0




    ...
OpenSplice DDS
Delivering Performance, Openness, and Freedom




                       Understanding DDS
As Simple as it Gets
‣ DDS is based around the concept
 of a fully distributed Global Data
 Space (GDS)
                  ...
As Simple as it Gets
‣ DDS is based around the concept
 of a fully distributed Global Data
 Space (GDS)
‣ Publishers and S...
As Simple as it Gets
‣ DDS is based around the concept
 of a fully distributed Global Data
                               ...
As Simple as it Gets
‣ DDS is based around the concept
 of a fully distributed Global Data
                               ...
As Simple as it Gets
‣ DDS is based around the concept
 of a fully distributed Global Data
                               ...
As Simple as it Gets
‣ DDS is based around the concept
  of a fully distributed Global Data
                              ...
As Simple as it Gets
‣ DDS is based around the concept
  of a fully distributed Global Data
                              ...
OpenSplice DDS
Delivering Performance, Openness, and Freedom




                           Defining Data
DDS Topics
                                                                                   {Circle, Square, Triangle}
T...
DDS Topics


                                                    Circle


struct ShapeType {
   long   x;
   long   y;
   ...
DDS Topics


                                                 Triangle



struct ShapeType {
   long   x;
   long   y;
   ...
DDS Topic Instances and Samples
Topic Instances
‣ Each key value identifies a unique Topic Instance,
‣ Topic’s instance li...
Topic/Instances/Samples Recap.
   Topics                                                         Instances




           ...
Content Filtering
                                                                               X0                   X1
 ...
Local Queries
‣ Subscribed Topics can be seen locally as “Tables”
‣ A subset of SQL92 can be used for performing queries o...
OpenSplice DDS
Delivering Performance, Openness, and Freedom




             Anatomy of a DDS App.
Anatomy of a DDS Application

                                                    Topic
                                  ...
Anatomy of a DDS Application

                                                                 Topic
                     ...
Anatomy of a DDS Application

                                                                   Topic
                   ...
OpenSplice DDS
Delivering Performance, Openness, and Freedom




       From Shapes to Market Data
The Market Data Topic
       Type Declaration                                                           Topic Instantiatio...
Filtering Market Data
                                                   Apple
               164.0                       ...
Writing Market Data
                    Creating a Writer
       // Create Topic with Default QoS
       dds::Topic<StockQ...
Writing Market Data
                                Creating a Writer
                   // Create Topic with Default QoS
...
Writing Market Data
                                Creating a Writer
                   // Create Topic with Default QoS
...
OpenSplice DDS
Delivering Performance, Openness, and Freedom



 Topic Used in next section.
                             ...
Reading Samples
                                                                                          ‣ Read iterates ...
Reading Samples
                                                                                          ‣ Read iterates ...
Reading Samples
                                                                                    ‣ Read iterates over t...
Taking Samples
                                                                                    ‣ Take iterates over th...
Taking Samples
                                                                            ‣ Take iterates over the
      ...
Taking Samples
                                                                         ‣ Take iterates over the
         ...
Reading Data
                    Creating a Reader
       // Create Topic with Default QoS
       dds::Topic<StockQuote> t...
Reading Data
                               Creating a Reader
                  // Create Topic with Default QoS
         ...
Reading Data
                               Creating a Reader
                  // Create Topic with Default QoS
         ...
Taking Data
                    Creating a Reader
       // Create Topic with Default QoS
       dds::Topic<StockQuote> to...
Taking Data
                               Creating a Reader
                  // Create Topic with Default QoS
          ...
Taking Data
                               Creating a Reader
                  // Create Topic with Default QoS
          ...
Sample, Instance and View States
                      History Depth = 2


                                              ‣...
Would you let me know...
DDS provides three main mechanism for exchanging information with the
application


‣ Polling. Th...
Reading with Waitsets
      // Functor handling new data arrival          1
      struct MarketDataHandler {
          voi...
Reading with Waitsets
      // Functor handling new data arrival          1
      struct MarketDataHandler {
          voi...
Reading with Waitsets
                   // Functor handling new data arrival          1
                   struct MarketD...
Reading with Waitsets
                   // Functor handling new data arrival          1
                   struct MarketD...
Reading with Listeners
      // Functor handling new data arrival          1
      class MarketDataListener {
      public...
Reading with Listeners
       // Functor handling new data arrival          1
       class MarketDataListener {
       pub...
OpenSplice DDS
Delivering Performance, Openness, and Freedom




                      Quality of Service
QoS Model
‣ QoS-Policies are used to control relevant
  properties of OpenSplice DDS entities,                            ...
QoS Policies
QoS Policy           Applicability   RxO   Modifiable                                                         ...
Mapping QoS                                                                           Which properties does QoS controls?
...
Mapping QoS                                                                                                  Which propert...
Mapping QoS                                                                                                               ...
OpenSplice DDS
Delivering Performance, Openness, and Freedom




                       Playing with QoS
History QoS
                                                                                                       History...
Transport Priority QoS

Make sure that important data gets the right priority                                  Shared     ...
Ownership QoS
Availability of data producers can be controlled via two QoS Policies
‣ OWNERSHIP
‣ OWNERSHIP STRENGTH
‣ Ins...
Ownership QoS
Availability of data producers can be controlled via two QoS Policies
‣ OWNERSHIP
‣ OWNERSHIP STRENGTH
‣ Ins...
Ownership QoS
Availability of data producers can be controlled via two QoS Policies
 
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
Upcoming SlideShare
Loading in...5
×

I need the data and I need it 1ms ago!

2,081

Published on

This presentation was given at SkillsMatter "Open Source in Finance and Exchange" on 15 Jan 2009 (http://bit.ly/8VfSes). The presentation provides an introduction to the key DDS concepts as well as its programming.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,081
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
137
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "I need the data and I need it 1ms ago!"

  1. 1. OpenSplice DDS Delivering Performance, Openness, and Freedom I Need Market Data Angelo Corsaro, Ph.D. Chief Technology Officer OMG DDS SIG Co-Chair angelo.corsaro@prismtech.com and I need it 1ms Ago!
  2. 2. OpenSplice DDS Delivering Performance, Openness, and Freedom Challenges
  3. 3. Financial Firms Trends ‣ Direct connection to exchanges is increasingly chosen over data aggregators to reduce trading latency ‣ Market data is growing steadily ‣ Milliseconds speedup over competitors can be worth millions of dollars Challenges ‣ Process several hundreds-of-thousands of messages, quickly approaching millions messages, per second ‣ Minimize latency and jitter, while maintaining the system stability and reliability Deployed middleware solutions were not designed to address these challenges! © 2009, PrismTech. All Rights Reserved
  4. 4. Exchanges Trends ‣ More and more firms connecting directly ‣ Increasing number of sources providing the same quotes ‣ Competitive advantage obtained by reducing latency on data feeds Challenges ‣ Ensuring fairness of data distribution ‣ Minimizing latency ‣ Scaling with the number of destinations Deployed middleware solutions were not designed to address these challenges! © 2009, PrismTech. All Rights Reserved
  5. 5. OPRA Predictions OPRA Market Data Rates Forecasts Trends 2000000 Msgs/sec 1807000 ‣ One order of magnitude growth on market data from 2005 to 2008 1500000 ‣ OPRA message rates predicted to reach ~2M 1000000 msgs/sec by early 2009 701000 ‣ Traffic bursts predicted to further increase by 20% 500000 359000 Challenges ‣ Can your messaging 190000 Jan 6, 2009 Jan 8, 2008 88000 Jan 9, 2007 0 May 16, 2006 infrastructure keep-up Jan 13, 2005 with this trend? © 2009, PrismTech. All Rights Reserved
  6. 6. Trading with Pareto ‣ Usually 20% of the trades provides 80% of the gains Msgs/sec ‣ Under overload condition how can you ensure you keep trading at least 20% of your target? Vilfredo Pareto, 1848-1923 ‣ How can you ensure that you keep trading the the Average right 20% of data? System Message Rate Incoming Message Rate Time © 2009, PrismTech. All Rights Reserved
  7. 7. OpenSplice DDS Delivering Performance, Openness, and Freedom The Data Distribution Service for Real-Time Systems (DDS)
  8. 8. Addressing Data Distribution Challenges DDS is standard designed to address the data-distribution challenges across The OMG DDS Standard a wide class of Defense and Aerospace Applications ‣ Introduced in 2004 to address the Data Distribution challenges faced by a wide class of Defense and Aerospace Applications ‣ Key requirement for the standard were its ability to deliver very high performance while seamlessly scaling from embedded to ultra- large-scale deployments ‣ Today recommended by key administration worldwide and widely adopted across several different application domains, such as, Automated Trading, Simulations, SCADA, Telemetry, etc. © 2009, PrismTech. All Rights Reserved
  9. 9. The OMG Data Distribution Service (DDS) DDS v1.2 API Standard ‣ Language Independent, OS and HW architecture Application independent Object/Relational Mapping ‣ DCPS. Standard API for Data-Centric, Topic- Data Local Reconstruction Layer (DLRL) Based, Real-Time Publish/Subscribe Content ‣ Ownership Durability DLRL. Standard API for creating Object Views out Subscription of collection of Topics Minimum Profile Data Centric Publish/Subscribe (DCPS) DDSI/RTPS v2.1 Wire Protocol Standard ‣ Standard wire protocol allowing interoperability Real-Time Publish/Subscribe Protocol DDS Interoperability Wire Protocol between different implementations of the DDS standard UDP/IP ‣ Interoperability demonstrated among key DDS vendors in March 2009 © 2009, PrismTech. All Rights Reserved
  10. 10. Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Data Local Reconstruction Layer (DLRL) DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Profile Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved
  11. 11. Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Data Local Reconstruction Layer (DLRL) DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Profile Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Wire Protocol Standard Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved
  12. 12. Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Data Local Reconstruction Layer (DLRL) DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Profile Pub/Sub Standard API Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Wire Protocol Standard Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved
  13. 13. Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Object Relational Mapping Data Local Reconstruction Layer (DLRL) Standard API DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Profile Pub/Sub Standard API Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Wire Protocol Standard Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved
  14. 14. Architectural Style AMQP OMG DDS Subscriber Subscriber Publisher Publisher Broker Broker Publisher Broker Subscriber Publisher Broker Subscriber Broker Broker Publisher Publisher Subscriber Subscriber © 2009, PrismTech. All Rights Reserved
  15. 15. Key Differences in Summary DDS AMQP ‣ Peer-to-Peer Communication ‣ Brokered Communication Architecture Architecture ‣ Wire Protocol + API Standard ‣ Wire Protocol Standard ‣ Interoperability across DDS implementations ‣ Interoperability across AMQP implementations ‣ Application Portability across DDS Implementation ‣ Content-Based Filtering/Quering ‣ Header Based Routing ‣ Designed bottom-up for maximum ‣ Not geared for High-Performance performance and determinism © 2009, PrismTech. All Rights Reserved
  16. 16. OpenSplice DDS Delivering Performance, Openness, and Freedom DDS and Open Source
  17. 17. OpenSplice DDS ‣ Product reorganized into Editions ‣ Community Edition Enterprise Ed. ‣ Compact Edition ‣ Professional Edition ‣ Enterprise Edition Professional Ed. Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Product Editions provide a a growing set of Enterprise Ed. Compact Ed. functionality to address the needs of increasingly more sophisticated users ‣ The Community Edition is Open Source Community Ed. ‣ Compact, Enterprise and Professional Edition are available only through Commercial Subscriptions © 2009, PrismTech. All Rights Reserved
  18. 18. Community Edition Features Licensing ‣ OMG DDS v1.2 DCPS ‣ LGPLv3 ‣ Minimum Profile ‣ Content Subscription Profile Application ‣ Durability Profile Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Ownership Profile Content DDS v1.2 Ownership Durability Subscription ‣ Networking Minimum Profile ‣ DDSI v2.1 Implementation Real-Time Pub/Sub (DCPS) ‣ Real-Time Networking Implementation DDSI v2.1 Interoperable Wire Protocol Real-Time Networking (DDSI) Networking Technology UDP/IP Open Source LGPL Licensed Edition -- The Best Way to Get Started with DDS © 2009, PrismTech. All Rights Reserved
  19. 19. Performance on Commodity HW Throughput -./0/1/23' 456/' Latency #+(+%++' &&++%++' $+(+%++' "*(%)+' ")(%&!' ""&%,&'&+++%++' Inter-Node Latency !"#$%&#' ‣ 60 usec "!+%++' )"#%*)' "++%++' )$,%$*' !+(+%++' )++%++' Inter-Core Read-Latency &'$($)$*+% Proprietary Information - Distribution without Expressed Written Permission is Prohibited. *!#%,&' ‣ 2 usec !"#$% ,+(+%++' *++%++' ()(!%(*' #++%++' (+(+%++' &")&%+)' Inter-Core Latency ‣ <10 usec $&"%,,' $++%++' &!(,%!)' &+(+%++' ),,%$#' !++%++' ,*+%&)' !,*%))' ,($%!+' (("%!"' &&)%*$' $"%"$' ,+%($' ,++%++' HW: (+%++' )' &#' ,(' #!' &()' ($#' $&(' &+(!' (+!)' !+"#' ‣ Dell blade-server !*$$,(*%-./*% ‣ Dual-core, Dual-CPU, AMD Opteron 2.4 Ghz Test Scenario OS ‣ Linux 2.6.21-1.3194.fc7 ‣ Single Threaded Application (multi-threaded networking service) Network ‣ 8192 bit message batches ‣ Gigabit Ethernet cards ‣ Dell PowerConnect 5324 switch © 2009, PrismTech. All Rights Reserved
  20. 20. OpenSplice DDS Adoption Defense ‣ Combat Management Systems ‣ Tactical Systems ‣ Drones (land/air) ‣ Communication Systems Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Radar Processing ‣ Submarine Systems ‣ Future Combat Systems SCADA/Utilities ‣ Industrial Automation ‣ Power Grids (Smart Grids) ‣ Complex Telemetry (e.g. NASA Rocket Launch System) © 2009, PrismTech. All Rights Reserved
  21. 21. OpenSplice DDS Adoption Transportation ‣ Air Traffic Management/Control ‣ Metropolitan Traffic Management ‣ Underground Metropolitan Transportation Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Financial Services ‣ Automated Trading ‣ Risk Management ‣ Option Trading © 2009, PrismTech. All Rights Reserved
  22. 22. Some OpenSplice DDS Users !"#$%&'()('*+'##,-+.+/($$,012"0 Experti réalisée par Bernard M © 2009, PrismTech. All Rights Reserved
  23. 23. OpenSplice DDS Delivering Performance, Openness, and Freedom Understanding DDS
  24. 24. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Space (GDS) Brokers Global Data Space DDS © 2009, PrismTech. All Rights Reserved
  25. 25. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Global Data Space DDS © 2009, PrismTech. All Rights Reserved
  26. 26. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber Publisher Global Data Space Publisher Subscriber DDS © 2009, PrismTech. All Rights Reserved
  27. 27. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber ‣ Publishers and Subscribers Publisher Global Data Space express their intent to produce/ consume specific type of data, e.g., Topics Publisher Subscriber DDS © 2009, PrismTech. All Rights Reserved
  28. 28. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber ‣ Publishers and Subscribers Publisher Global Data Space express their intent to produce/ consume specific type of data, e.g., Topics Publisher Subscriber DDS © 2009, PrismTech. All Rights Reserved
  29. 29. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber ‣ Publishers and Subscribers Publisher Global Data Space express their intent to produce/ consume specific type of data, e.g., Topics Subscriber ‣ Data flows from Publisher to Publisher Subscribers DDS © 2009, PrismTech. All Rights Reserved
  30. 30. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber ‣ Publishers and Subscribers Publisher Global Data Space express their intent to produce/ consume specific type of data, e.g., Topics Subscriber ‣ Data flows from Publisher to Publisher Subscribers DDS © 2009, PrismTech. All Rights Reserved
  31. 31. OpenSplice DDS Delivering Performance, Openness, and Freedom Defining Data
  32. 32. DDS Topics {Circle, Square, Triangle} Topic ‣ Unit of information exchanged between Publisher and Subscribers. ‣ An association between a unique name, a type and a QoS setting {ShapeType} {...} Topic Type. struct ShapeType { ‣ Type describing the data associated with one long x; or more Topics long y; ‣ A Topic type can have a key represented by an long shapesize; string color; arbitrary number of attributes }; ‣ Expressed in IDL (or XML) #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
  33. 33. DDS Topics Circle struct ShapeType { long x; long y; long shapesize; Topic QoS string color; }; #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
  34. 34. DDS Topics Triangle struct ShapeType { long x; long y; long shapesize; Topic QoS string color; }; #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
  35. 35. DDS Topic Instances and Samples Topic Instances ‣ Each key value identifies a unique Topic Instance, ‣ Topic’s instance lifetime can be explicitly managed in DDS Topic Samples ‣ The values assumed by a Topic Instance over time are referred as Instance Sample struct ShapeType { long x; long y; long shapesize; string color; }; #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
  36. 36. Topic/Instances/Samples Recap. Topics Instances Samples ti tj tnow time © 2009, PrismTech. All Rights Reserved
  37. 37. Content Filtering X0 X1 X0 <= X <= X1 ‣ DDS allows to specify content- filtered Topics for which a subset of SQL92 is used to express the filter condition Y0 ‣ Content filters can be applied on the entire content of the Topic Type Y0 <= Y <= Y1 ‣ Content filters are applied by DDS Y1 each time a new sample is produced/delivered © 2009, PrismTech. All Rights Reserved
  38. 38. Local Queries ‣ Subscribed Topics can be seen locally as “Tables” ‣ A subset of SQL92 can be used for performing queries on X0 multiple topics as well as natural joins ‣ Queries are Circle Topic performed under Y0 color x y shapesize Y0 user control and red 57 62 50 provide a result that blue 90 85 50 Y0 <= Y <= Y1 depends on the yellow 30 25 50 current snapshot of SELECT * FROM ShapeType s WHERE s.x > 25 AND s.y < 55 Y1 the system, e.g., samples currently color x y shapesize available yellow 30 25 50 © 2009, PrismTech. All Rights Reserved
  39. 39. OpenSplice DDS Delivering Performance, Openness, and Freedom Anatomy of a DDS App.
  40. 40. Anatomy of a DDS Application Topic TopicTopic Circle) (e.g. Samples Samples Instances Samples Instances Instances 1 21 62 1 21 62 1 22 62 1 22 62 1 23 63 1 23 63 DataReader DataReader blue 21 62 80 blue 23 66 80 blue 27 70 80 2 20 61 2 19 60 DataReader DataReader 2 20 61 2 19 60 orange 21 62 80 orange 21 62 80 DataWriter DataWriter DataReader DataReader struct ShapeType { 3 25 70 3 25 71 25 3 25 74 3 26 77 3 26 77 DataWriter DataWriter long x; 3 25 70 3 25 71 25 3 25 74 long y; green 97 75 50 green 94 71 50 long shapesize; string color; }; #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
  41. 41. Anatomy of a DDS Application Topic TopicTopic Circle) (e.g. Samples Samples Instances Samples Instances Instances 1 21 62 1 21 62 1 22 62 1 22 62 1 23 63 1 23 63 DataReader DataReader blue 21 62 80 blue 23 66 80 blue 27 70 80 2 20 61 2 19 60 DataReader DataReader 2 20 61 2 19 60 orange 21 62 80 orange 21 62 80 DataWriter DataWriter DataReader DataReader struct ShapeType { 3 25 70 3 25 71 25 3 25 74 3 26 77 3 26 77 DataWriter DataWriter long x; 3 25 70 3 25 71 25 3 25 74 long y; green 97 75 50 green 94 71 50 long shapesize; string color; Arrows }; #pragma keylist ShapeType color show structural relationship s, not data- flows Partition © 2009, PrismTech. All Rights Reserved
  42. 42. Anatomy of a DDS Application Topic TopicTopic Circle) (e.g. Samples Samples Instances Samples Instances Instances 1 21 62 1 21 62 1 22 62 1 22 62 1 23 63 1 23 63 DataReader DataReader blue 21 62 80 blue 23 66 80 blue 27 70 80 2 20 61 2 19 60 DataReader DataReader 2 20 61 2 19 60 orange 21 62 80 orange 21 62 80 DataWriter DataWriter DataReader DataReader struct ShapeType { 3 25 70 3 25 71 25 3 25 74 3 26 77 3 26 77 DataWriter DataWriter long x; 3 25 70 3 25 71 25 3 25 74 long y; green 97 75 50 green 94 71 50 long shapesize; string color; Arrows }; #pragma keylist ShapeType color show structural relationship s, not data- Subscriber Publisher flows Partition Domain Participant Domain © 2009, PrismTech. All Rights Reserved
  43. 43. OpenSplice DDS Delivering Performance, Openness, and Freedom From Shapes to Market Data
  44. 44. The Market Data Topic Type Declaration Topic Instantiation struct StockQuote { char<4> symbol; // Create Topic with Default QoS char<4> exchange; dds::Topic<StockQuote> topic(“stock_quote”); char<64> name; float quote; // Create Content Filtered Topic dds::ContentFilteredTopic cfTopic = }; topic.create_content_filtered(expr, #pragma keylist StockQuote symbol params); symbol name exchange quote AAPL Apple Inc. NASD 165.37 GOOG Google Inc. NASD 663.97 MSFT Microsoft Corp. NASD 33.73 © 2009, PrismTech. All Rights Reserved
  45. 45. Filtering Market Data Apple 164.0 Microsoft 40 161.5 30 MSFT > $35 159.0 20 AAPL < $157 |156.5 AAPL > $162 10 154.0 Untitled 1 Untitled 20 Untitled 39 Untitled 58 Untitled 77 Untitled 96 0 AAPL MSFT ‣ DDS continuous queries can be used for filtering the stream of data produced for a specific stock symbol. © 2009, PrismTech. All Rights Reserved
  46. 46. Writing Market Data Creating a Writer // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Writer with Default Qos dds::Writer<StockQuote> writer(topic); © 2009, PrismTech. All Rights Reserved
  47. 47. Writing Market Data Creating a Writer // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Writer with Default Qos dds::Writer<StockQuote> writer(topic); Writing Data StockQuote quote = { “AAPL”, “NASD”, 163.37, “Apple Inc.” }; writer.write(quote); © 2009, PrismTech. All Rights Reserved
  48. 48. Writing Market Data Creating a Writer // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Writer with Default Qos dds::Writer<StockQuote> writer(topic); Writing Data Fastest Way of Writing Data StockQuote quote = { StockQuote quote = { “AAPL”, “NASD”, “AAPL”, “NASD”, 163.37, “Apple Inc.” 163.37, “Apple Inc.” }; }; dds::InstanceWriter iwriter = writer.write(quote); writer.register_instance(quote); iwriter.write(quote); © 2009, PrismTech. All Rights Reserved
  49. 49. OpenSplice DDS Delivering Performance, Openness, and Freedom Topic Used in next section. Read vs. Take struct Counter { int cID; int count; }; #pragma keylist Counter cID
  50. 50. Reading Samples ‣ Read iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Samples are not removed 2 1 2 2 2 3 from the local cache as DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 4 3 5 result of a read 3 1 3 2 3 3 ‣ Read samples can be read Topic again, by accessing the Samples Read Samples not Read cache with the proper options (more later) DataReader Cache © 2009, PrismTech. All Rights Reserved
  51. 51. Reading Samples ‣ Read iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Samples are not removed 2 1 2 2 2 3 from the local cache as DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 3 2 3 3 3 4 3 5 result of a read ‣ Read samples can be read Topic again, by accessing the Samples Read Samples not Read cache with the proper options (more later) DataReader Cache © 2009, PrismTech. All Rights Reserved
  52. 52. Reading Samples ‣ Read iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Samples are not removed 2 1 2 2 2 3 from the local cache as DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 3 2 3 3 3 4 3 5 result of a read ‣ Read samples can be read Topic again, by accessing the Samples Read Samples not Read cache with the proper options (more later) DataReader Cache © 2009, PrismTech. All Rights Reserved
  53. 53. Taking Samples ‣ Take iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Taken Samples are 2 1 2 2 2 3 removed from the local DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 4 3 5 cache as result of a take 3 1 3 2 3 3 ‣ Topic Samples not Taken DataReader Cache © 2009, PrismTech. All Rights Reserved
  54. 54. Taking Samples ‣ Take iterates over the available sample instances 1 2 1 3 1 4 ‣ Taken Samples are 2 2 2 3 removed from the local DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 2 3 3 3 4 3 5 cache as result of a take Topic Samples not Taken DataReader Cache © 2009, PrismTech. All Rights Reserved
  55. 55. Taking Samples ‣ Take iterates over the available sample instances 1 3 1 4 ‣ Taken Samples are 2 3 removed from the local DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 3 3 4 3 5 cache as result of a take Topic Samples not Taken DataReader Cache © 2009, PrismTech. All Rights Reserved
  56. 56. Reading Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); © 2009, PrismTech. All Rights Reserved
  57. 57. Reading Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); Reading Data StockQuoteSeq data; dds::SampleInfoSeq info; reader.read(data, info); © 2009, PrismTech. All Rights Reserved
  58. 58. Reading Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); Reading a Specific Instance Reading Data StockQuote quote; quote.symbol = “APPL”; StockQuoteSeq data; StockQuoteSeq data; dds::SampleInfoSeq info; dds::SampleInfoSeq info; reader.read(data, info); dds::InstanceReader ireader = reader.instance(quote); ireader.read(data, info); © 2009, PrismTech. All Rights Reserved
  59. 59. Taking Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); © 2009, PrismTech. All Rights Reserved
  60. 60. Taking Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); Taking Data StockQuoteSeq data; dds::SampleInfoSeq info; reader.take(data, info); © 2009, PrismTech. All Rights Reserved
  61. 61. Taking Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); Taking a Specific Instance Taking Data StockQuote quote; quote.symbol = “APPL”; StockQuoteSeq data; StockQuoteSeq data; dds::SampleInfoSeq info; dds::SampleInfoSeq info; reader.take(data, info); dds::InstanceReader ireader = reader.instance(quote); ireader.take(data, info); © 2009, PrismTech. All Rights Reserved
  62. 62. Sample, Instance and View States History Depth = 2 ‣ Along with data samples, DataReaders are provided 1 1 with state information allowing to detect relevant 2 2 transitions in the life-cycle of data as well as data 3 writers DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. SampleInfo ‣ Sample State (READ | NOT_READ): Determines wether a sample has already been read by this 1 1 1 2 DataReader or not. 2 2 2 3 ‣ Instance State (ALIVE, NOT_ALIVE, DISPOSED). 3 1 Determines wether (1) writer exist for the specific Samples instance, or (2) no matched writers are currently Topic available, or (3) the instance has been disposed DataReader Cache ‣ View State (NEW, NOT_NEW). Determines wether this is the first sample of a new (or re-born) instance © 2009, PrismTech. All Rights Reserved
  63. 63. Would you let me know... DDS provides three main mechanism for exchanging information with the application ‣ Polling. The application polls from time to time for new data or status changes. Proprietary Information - Distribution without Expressed Written Permission is Prohibited. The interval might depend on the kind of applications as well as data ‣ WaitSets. The application registers a WaitSet with DDS and waits (i.e. is suspended) until one of the specified events has happened. ‣ Listeners. The application registers a listener with a specific DDS entity to be notified when relevant events occur, such as state changes or © 2009, PrismTech. All Rights Reserved
  64. 64. Reading with Waitsets // Functor handling new data arrival 1 struct MarketDataHandler { void operator()(MarketDataReader& reader); }; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. © 2009, PrismTech. All Rights Reserved
  65. 65. Reading with Waitsets // Functor handling new data arrival 1 struct MarketDataHandler { void operator()(MarketDataReader& reader); }; // Create & Register Condition 2 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. MarketDataHandler functor; dds::ReadCondition cond = reader.create_read_condition(functor); dds::WaitSet ws; ws.attach(cond); © 2009, PrismTech. All Rights Reserved
  66. 66. Reading with Waitsets // Functor handling new data arrival 1 struct MarketDataHandler { void operator()(MarketDataReader& reader); }; // Create & Register Condition 2 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. MarketDataHandler functor; dds::ReadCondition cond = reader.create_read_condition(functor); dds::WaitSet ws; ws.attach(cond); // Wait & Handle 3’ dds::ConditionVector cvec = ws.wait(); for (int i = 0; i < cvec.size(); ++i) cvec[i].execute(); © 2009, PrismTech. All Rights Reserved
  67. 67. Reading with Waitsets // Functor handling new data arrival 1 struct MarketDataHandler { void operator()(MarketDataReader& reader); }; // Create & Register Condition 2 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. MarketDataHandler functor; dds::ReadCondition cond = reader.create_read_condition(functor); dds::WaitSet ws; ws.attach(cond); // Wait & Handle 3’ // Dispatch 3” dds::ConditionVector cvec = ws.wait(); ws.dispatch(); for (int i = 0; i < cvec.size(); ++i) cvec[i].execute(); © 2009, PrismTech. All Rights Reserved
  68. 68. Reading with Listeners // Functor handling new data arrival 1 class MarketDataListener { public: void handle_data(MarketDataReader& reader); }; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. © 2009, PrismTech. All Rights Reserved
  69. 69. Reading with Listeners // Functor handling new data arrival 1 class MarketDataListener { public: void handle_data(MarketDataReader& reader); }; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 2 // Functor handling new data arrival MarketDataListener mdl; dds::connection cnt = reader.on_data_available_signal_connect( std::bind(&MarketDataListener::handle_data, &mdl, _1)); © 2009, PrismTech. All Rights Reserved
  70. 70. OpenSplice DDS Delivering Performance, Openness, and Freedom Quality of Service
  71. 71. QoS Model ‣ QoS-Policies are used to control relevant properties of OpenSplice DDS entities, Type Matching QoS matching such as: QoS QoS QoS QoS QoS QoS QoS ‣ Temporal Properties ‣ Priority Topic Name Publisher Subscriber ‣ Durability ... DataWriter writes Type reads DataReader ... ... ‣ Availability DomainParticipant writes Type reads DataReader DomainParticipant DataWriter ‣ ... Name ‣ Some QoS-Policies are matched based on Topic QoS QoS QoS a Request vs. Offered Model thus QoS- enforcement ‣ Publications and Subscriptions match only if the declared vs. requested QoS are compatible ‣ e.g., it is not possible to match a publisher which delivers data unreliably with a subscriber which requires reliability © 2009, PrismTech. All Rights Reserved
  72. 72. QoS Policies QoS Policy Applicability RxO Modifiable Type Matching QoS matching DURABILITY T, DR, DW Y N Data Availability QoS QoS QoS QoS QoS QoS QoS DURABILITY SERVICE T, DW N N Topic Name Publisher Subscriber LIFESPAN T, DW - Y ... DataWriter writes Type reads DataReader ... T, DR, DW N N ... HISTORY DomainParticipant DataWriter writes Type reads DataReader DomainParticipant PRESENTATION P, S Y N Data Delivery Name Topic RELIABILITY T, DR, DW Y N PARTITION P, S N Y QoS QoS QoS DESTINATION ORDER T, DR, DW Y N OWNERSHIP T, DR, DW Y N ‣ Rich set of QoS allow to configure OWNERSHIP DW - Y STRENGTH several different aspects of data DEADLINE T, DR, DW Y Y Data Timeliness availability, delivery and timeliness LATENCY BUDGET T, DR, DW Y Y TRANSPORT PRIORITY T, DW - Y ‣ QoS can be used to control and optimize network as well as TIME BASED FILTER DR - Y Resources computing resource RESOURCE LIMITS T, DR, DW N N USER_DATA DP, DR, DW N Y Configuration TOPIC_DATA T N Y GROUP_DATA P, S N Y © 2009, PrismTech. All Rights Reserved
  73. 73. Mapping QoS Which properties does QoS controls? TimeBasedFilter Deadline Data Throughput Latency LatencyBudget TransportPriority Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Control over Latency/Throughput tradeoff Control over data latency Control over data priority © 2009, PrismTech. All Rights Reserved
  74. 74. Mapping QoS Which properties does QoS controls? TimeBasedFilter Deadline History Data Data Throughput Lifespan Durability Latency Availability Ownership LatencyBudget TransportPriority Ownership Strength Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Control over Latency/Throughput tradeoff Control over data queueing Control over data latency Control over data persistency Control over data priority Control over data sources hot-swap © 2009, PrismTech. All Rights Reserved
  75. 75. Mapping QoS Which properties does QoS controls? TimeBasedFilter Deadline History Data Data Throughput Lifespan Durability Latency Availability Ownership LatencyBudget TransportPriority Ownership Strength Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Control over Latency/Throughput tradeoff Control over data queueing Control over data latency Control over data persistency Control over data priority Control over data sources hot-swap Reliability OpenSplice DDS provides programmatic Destination QoS-driven support for configuring the most Presentation Data Delivery Order important properties of data distribution! Control over data distribution reliability Control over data ordering Control over presentation © 2009, PrismTech. All Rights Reserved
  76. 76. OpenSplice DDS Delivering Performance, Openness, and Freedom Playing with QoS
  77. 77. History QoS History Depth = 5 1 1 1 2 1 3 1 4 1 5 DataReader 2 1 2 2 2 3 2 4 2 5 3 1 3 2 3 3 3 4 3 5 1 // Create Reader QoS Topic Proprietary Information - Distribution without Expressed Written Permission is Prohibited. dds::DataReaderQos drqos; drqos.set_keep_last(N); // Create Reader ‣ The History QoS Controls the number of dds::DataReader<MarketData> reader(topic, drqos); samples-per-instance that will be stored by the middleware on behalf of a Reader 2 ‣ Keep Last K. The History QoS can be set so // Create Reader QoS to always have the latest K samples dds::DataReaderQos drqos; drqos.set_keep_all(); ‣ Keep All. The History QoS can be set so // Create Reader keep all samples produced by the writer and dds::DataReader<MarketData> reader(topic, drqos); not yet taken, until resource limits are not reached © 2009, PrismTech. All Rights Reserved
  78. 78. Transport Priority QoS Make sure that important data gets the right priority Shared Single Copy per Node Shared Pack Across Topics/Applications Memory Optimal Unmarshaling Memory 1 // Create Writer QoS dds::DataWriterQos dwqos; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. dwqos.set_priority(MID_PRIO); OpenSplice DDS OpenSplice DDS // Create Writer Binding Pre-emptive Network Scheduler Binding dds::DataWriter<MarketData> writer(topic, dwqos); Networking Priority Scheduler Networking Data Urgency Traffic Pacing // Write Data writer.write(data); Network Channels // Change Priority Priority Bands dwqos.set_priority(HIGH_PRIO); Traffic Shaping writer.writer(data) © 2009, PrismTech. All Rights Reserved
  79. 79. Ownership QoS Availability of data producers can be controlled via two QoS Policies ‣ OWNERSHIP ‣ OWNERSHIP STRENGTH ‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote symbol: "MSFT" StockQuote StockQuote name: "Microsoft Corp." StockQuote symbol: "GOOG" symbol: "GOOG" exchange: "NASD""GOOG" symbol: name: "Google Inc." quote: 33.73 name: "Google Inc." name: "Google Inc." exchange: "NASD" exchange: "NASD" exchange: "NASD" StockQuote StockQuote StockQuote quote: 663.97 W1 quote: 663.97"AAPL" symbol: "AAPL" quote: 663.97 symbol: "AAPL" symbol: name: "Apple Inc." name: "Apple Inc." name: "Apple Inc." exchange: "NASD" exchange: "NASD" exchange: "NASD" R1 quote: 165.37 quote: 165.37 quote: 165.37 STRENGTH=3 StockQuote symbol: "AAPL" StockQuote Inc." name: "Apple symbol: "GOOG""NASD" exchange: name: quote: 165.37 "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W1’ R2 STRENGTH=2 StockQuote symbol: "AAPL" StockQuote Inc." name: "Apple symbol: "GOOG""NASD" exchange: name: quote: 165.37 "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W2’’ R3 STRENGTH=1 © 2009, PrismTech. All Rights Reserved
  80. 80. Ownership QoS Availability of data producers can be controlled via two QoS Policies ‣ OWNERSHIP ‣ OWNERSHIP STRENGTH ‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote symbol: "MSFT" StockQuote StockQuote name: "Microsoft Corp." StockQuote symbol: "GOOG" symbol: "GOOG" exchange: "NASD""GOOG" symbol: name: "Google Inc." quote: 33.73 name: "Google Inc." name: "Google Inc." exchange: "NASD" exchange: "NASD" exchange: "NASD" quote: 663.97 W1 quote: 663.97 quote: 663.97 StockQuote symbol: "AAPL" R1 STRENGTH=3 name: "Apple Inc." exchange: "NASD" quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W1’ R2 StockQuote symbol: "AAPL" name: "Apple Inc." exchange: "NASD" STRENGTH=2 quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W2’’ StockQuote symbol: "AAPL" name: "Apple Inc." exchange: "NASD" quote: 165.37 R3 STRENGTH=1 © 2009, PrismTech. All Rights Reserved
  81. 81. Ownership QoS Availability of data producers can be controlled via two QoS Policies  
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×