Your SlideShare is downloading. ×
0
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
I need the data and I need it 1ms ago!
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

I need the data and I need it 1ms ago!

2,078

Published on

This presentation was given at SkillsMatter "Open Source in Finance and Exchange" on 15 Jan 2009 (http://bit.ly/8VfSes). The presentation provides an introduction to the key DDS concepts as well as …

This presentation was given at SkillsMatter "Open Source in Finance and Exchange" on 15 Jan 2009 (http://bit.ly/8VfSes). The presentation provides an introduction to the key DDS concepts as well as its programming.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,078
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
137
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. OpenSplice DDS Delivering Performance, Openness, and Freedom I Need Market Data Angelo Corsaro, Ph.D. Chief Technology Officer OMG DDS SIG Co-Chair angelo.corsaro@prismtech.com and I need it 1ms Ago!
  • 2. OpenSplice DDS Delivering Performance, Openness, and Freedom Challenges
  • 3. Financial Firms Trends ‣ Direct connection to exchanges is increasingly chosen over data aggregators to reduce trading latency ‣ Market data is growing steadily ‣ Milliseconds speedup over competitors can be worth millions of dollars Challenges ‣ Process several hundreds-of-thousands of messages, quickly approaching millions messages, per second ‣ Minimize latency and jitter, while maintaining the system stability and reliability Deployed middleware solutions were not designed to address these challenges! © 2009, PrismTech. All Rights Reserved
  • 4. Exchanges Trends ‣ More and more firms connecting directly ‣ Increasing number of sources providing the same quotes ‣ Competitive advantage obtained by reducing latency on data feeds Challenges ‣ Ensuring fairness of data distribution ‣ Minimizing latency ‣ Scaling with the number of destinations Deployed middleware solutions were not designed to address these challenges! © 2009, PrismTech. All Rights Reserved
  • 5. OPRA Predictions OPRA Market Data Rates Forecasts Trends 2000000 Msgs/sec 1807000 ‣ One order of magnitude growth on market data from 2005 to 2008 1500000 ‣ OPRA message rates predicted to reach ~2M 1000000 msgs/sec by early 2009 701000 ‣ Traffic bursts predicted to further increase by 20% 500000 359000 Challenges ‣ Can your messaging 190000 Jan 6, 2009 Jan 8, 2008 88000 Jan 9, 2007 0 May 16, 2006 infrastructure keep-up Jan 13, 2005 with this trend? © 2009, PrismTech. All Rights Reserved
  • 6. Trading with Pareto ‣ Usually 20% of the trades provides 80% of the gains Msgs/sec ‣ Under overload condition how can you ensure you keep trading at least 20% of your target? Vilfredo Pareto, 1848-1923 ‣ How can you ensure that you keep trading the the Average right 20% of data? System Message Rate Incoming Message Rate Time © 2009, PrismTech. All Rights Reserved
  • 7. OpenSplice DDS Delivering Performance, Openness, and Freedom The Data Distribution Service for Real-Time Systems (DDS)
  • 8. Addressing Data Distribution Challenges DDS is standard designed to address the data-distribution challenges across The OMG DDS Standard a wide class of Defense and Aerospace Applications ‣ Introduced in 2004 to address the Data Distribution challenges faced by a wide class of Defense and Aerospace Applications ‣ Key requirement for the standard were its ability to deliver very high performance while seamlessly scaling from embedded to ultra- large-scale deployments ‣ Today recommended by key administration worldwide and widely adopted across several different application domains, such as, Automated Trading, Simulations, SCADA, Telemetry, etc. © 2009, PrismTech. All Rights Reserved
  • 9. The OMG Data Distribution Service (DDS) DDS v1.2 API Standard ‣ Language Independent, OS and HW architecture Application independent Object/Relational Mapping ‣ DCPS. Standard API for Data-Centric, Topic- Data Local Reconstruction Layer (DLRL) Based, Real-Time Publish/Subscribe Content ‣ Ownership Durability DLRL. Standard API for creating Object Views out Subscription of collection of Topics Minimum Profile Data Centric Publish/Subscribe (DCPS) DDSI/RTPS v2.1 Wire Protocol Standard ‣ Standard wire protocol allowing interoperability Real-Time Publish/Subscribe Protocol DDS Interoperability Wire Protocol between different implementations of the DDS standard UDP/IP ‣ Interoperability demonstrated among key DDS vendors in March 2009 © 2009, PrismTech. All Rights Reserved
  • 10. Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Data Local Reconstruction Layer (DLRL) DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Profile Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved
  • 11. Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Data Local Reconstruction Layer (DLRL) DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Profile Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Wire Protocol Standard Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved
  • 12. Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Data Local Reconstruction Layer (DLRL) DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Profile Pub/Sub Standard API Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Wire Protocol Standard Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved
  • 13. Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Object Relational Mapping Data Local Reconstruction Layer (DLRL) Standard API DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Profile Pub/Sub Standard API Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Wire Protocol Standard Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved
  • 14. Architectural Style AMQP OMG DDS Subscriber Subscriber Publisher Publisher Broker Broker Publisher Broker Subscriber Publisher Broker Subscriber Broker Broker Publisher Publisher Subscriber Subscriber © 2009, PrismTech. All Rights Reserved
  • 15. Key Differences in Summary DDS AMQP ‣ Peer-to-Peer Communication ‣ Brokered Communication Architecture Architecture ‣ Wire Protocol + API Standard ‣ Wire Protocol Standard ‣ Interoperability across DDS implementations ‣ Interoperability across AMQP implementations ‣ Application Portability across DDS Implementation ‣ Content-Based Filtering/Quering ‣ Header Based Routing ‣ Designed bottom-up for maximum ‣ Not geared for High-Performance performance and determinism © 2009, PrismTech. All Rights Reserved
  • 16. OpenSplice DDS Delivering Performance, Openness, and Freedom DDS and Open Source
  • 17. OpenSplice DDS ‣ Product reorganized into Editions ‣ Community Edition Enterprise Ed. ‣ Compact Edition ‣ Professional Edition ‣ Enterprise Edition Professional Ed. Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Product Editions provide a a growing set of Enterprise Ed. Compact Ed. functionality to address the needs of increasingly more sophisticated users ‣ The Community Edition is Open Source Community Ed. ‣ Compact, Enterprise and Professional Edition are available only through Commercial Subscriptions © 2009, PrismTech. All Rights Reserved
  • 18. Community Edition Features Licensing ‣ OMG DDS v1.2 DCPS ‣ LGPLv3 ‣ Minimum Profile ‣ Content Subscription Profile Application ‣ Durability Profile Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Ownership Profile Content DDS v1.2 Ownership Durability Subscription ‣ Networking Minimum Profile ‣ DDSI v2.1 Implementation Real-Time Pub/Sub (DCPS) ‣ Real-Time Networking Implementation DDSI v2.1 Interoperable Wire Protocol Real-Time Networking (DDSI) Networking Technology UDP/IP Open Source LGPL Licensed Edition -- The Best Way to Get Started with DDS © 2009, PrismTech. All Rights Reserved
  • 19. Performance on Commodity HW Throughput -./0/1/23' 456/' Latency #+(+%++' &&++%++' $+(+%++' "*(%)+' ")(%&!' ""&%,&'&+++%++' Inter-Node Latency !"#$%&#' ‣ 60 usec "!+%++' )"#%*)' "++%++' )$,%$*' !+(+%++' )++%++' Inter-Core Read-Latency &'$($)$*+% Proprietary Information - Distribution without Expressed Written Permission is Prohibited. *!#%,&' ‣ 2 usec !"#$% ,+(+%++' *++%++' ()(!%(*' #++%++' (+(+%++' &")&%+)' Inter-Core Latency ‣ <10 usec $&"%,,' $++%++' &!(,%!)' &+(+%++' ),,%$#' !++%++' ,*+%&)' !,*%))' ,($%!+' (("%!"' &&)%*$' $"%"$' ,+%($' ,++%++' HW: (+%++' )' &#' ,(' #!' &()' ($#' $&(' &+(!' (+!)' !+"#' ‣ Dell blade-server !*$$,(*%-./*% ‣ Dual-core, Dual-CPU, AMD Opteron 2.4 Ghz Test Scenario OS ‣ Linux 2.6.21-1.3194.fc7 ‣ Single Threaded Application (multi-threaded networking service) Network ‣ 8192 bit message batches ‣ Gigabit Ethernet cards ‣ Dell PowerConnect 5324 switch © 2009, PrismTech. All Rights Reserved
  • 20. OpenSplice DDS Adoption Defense ‣ Combat Management Systems ‣ Tactical Systems ‣ Drones (land/air) ‣ Communication Systems Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Radar Processing ‣ Submarine Systems ‣ Future Combat Systems SCADA/Utilities ‣ Industrial Automation ‣ Power Grids (Smart Grids) ‣ Complex Telemetry (e.g. NASA Rocket Launch System) © 2009, PrismTech. All Rights Reserved
  • 21. OpenSplice DDS Adoption Transportation ‣ Air Traffic Management/Control ‣ Metropolitan Traffic Management ‣ Underground Metropolitan Transportation Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Financial Services ‣ Automated Trading ‣ Risk Management ‣ Option Trading © 2009, PrismTech. All Rights Reserved
  • 22. Some OpenSplice DDS Users !"#$%&'()('*+'##,-+.+/($$,012"0 Experti réalisée par Bernard M © 2009, PrismTech. All Rights Reserved
  • 23. OpenSplice DDS Delivering Performance, Openness, and Freedom Understanding DDS
  • 24. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Space (GDS) Brokers Global Data Space DDS © 2009, PrismTech. All Rights Reserved
  • 25. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Global Data Space DDS © 2009, PrismTech. All Rights Reserved
  • 26. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber Publisher Global Data Space Publisher Subscriber DDS © 2009, PrismTech. All Rights Reserved
  • 27. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber ‣ Publishers and Subscribers Publisher Global Data Space express their intent to produce/ consume specific type of data, e.g., Topics Publisher Subscriber DDS © 2009, PrismTech. All Rights Reserved
  • 28. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber ‣ Publishers and Subscribers Publisher Global Data Space express their intent to produce/ consume specific type of data, e.g., Topics Publisher Subscriber DDS © 2009, PrismTech. All Rights Reserved
  • 29. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber ‣ Publishers and Subscribers Publisher Global Data Space express their intent to produce/ consume specific type of data, e.g., Topics Subscriber ‣ Data flows from Publisher to Publisher Subscribers DDS © 2009, PrismTech. All Rights Reserved
  • 30. As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber ‣ Publishers and Subscribers Publisher Global Data Space express their intent to produce/ consume specific type of data, e.g., Topics Subscriber ‣ Data flows from Publisher to Publisher Subscribers DDS © 2009, PrismTech. All Rights Reserved
  • 31. OpenSplice DDS Delivering Performance, Openness, and Freedom Defining Data
  • 32. DDS Topics {Circle, Square, Triangle} Topic ‣ Unit of information exchanged between Publisher and Subscribers. ‣ An association between a unique name, a type and a QoS setting {ShapeType} {...} Topic Type. struct ShapeType { ‣ Type describing the data associated with one long x; or more Topics long y; ‣ A Topic type can have a key represented by an long shapesize; string color; arbitrary number of attributes }; ‣ Expressed in IDL (or XML) #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
  • 33. DDS Topics Circle struct ShapeType { long x; long y; long shapesize; Topic QoS string color; }; #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
  • 34. DDS Topics Triangle struct ShapeType { long x; long y; long shapesize; Topic QoS string color; }; #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
  • 35. DDS Topic Instances and Samples Topic Instances ‣ Each key value identifies a unique Topic Instance, ‣ Topic’s instance lifetime can be explicitly managed in DDS Topic Samples ‣ The values assumed by a Topic Instance over time are referred as Instance Sample struct ShapeType { long x; long y; long shapesize; string color; }; #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
  • 36. Topic/Instances/Samples Recap. Topics Instances Samples ti tj tnow time © 2009, PrismTech. All Rights Reserved
  • 37. Content Filtering X0 X1 X0 <= X <= X1 ‣ DDS allows to specify content- filtered Topics for which a subset of SQL92 is used to express the filter condition Y0 ‣ Content filters can be applied on the entire content of the Topic Type Y0 <= Y <= Y1 ‣ Content filters are applied by DDS Y1 each time a new sample is produced/delivered © 2009, PrismTech. All Rights Reserved
  • 38. Local Queries ‣ Subscribed Topics can be seen locally as “Tables” ‣ A subset of SQL92 can be used for performing queries on X0 multiple topics as well as natural joins ‣ Queries are Circle Topic performed under Y0 color x y shapesize Y0 user control and red 57 62 50 provide a result that blue 90 85 50 Y0 <= Y <= Y1 depends on the yellow 30 25 50 current snapshot of SELECT * FROM ShapeType s WHERE s.x > 25 AND s.y < 55 Y1 the system, e.g., samples currently color x y shapesize available yellow 30 25 50 © 2009, PrismTech. All Rights Reserved
  • 39. OpenSplice DDS Delivering Performance, Openness, and Freedom Anatomy of a DDS App.
  • 40. Anatomy of a DDS Application Topic TopicTopic Circle) (e.g. Samples Samples Instances Samples Instances Instances 1 21 62 1 21 62 1 22 62 1 22 62 1 23 63 1 23 63 DataReader DataReader blue 21 62 80 blue 23 66 80 blue 27 70 80 2 20 61 2 19 60 DataReader DataReader 2 20 61 2 19 60 orange 21 62 80 orange 21 62 80 DataWriter DataWriter DataReader DataReader struct ShapeType { 3 25 70 3 25 71 25 3 25 74 3 26 77 3 26 77 DataWriter DataWriter long x; 3 25 70 3 25 71 25 3 25 74 long y; green 97 75 50 green 94 71 50 long shapesize; string color; }; #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
  • 41. Anatomy of a DDS Application Topic TopicTopic Circle) (e.g. Samples Samples Instances Samples Instances Instances 1 21 62 1 21 62 1 22 62 1 22 62 1 23 63 1 23 63 DataReader DataReader blue 21 62 80 blue 23 66 80 blue 27 70 80 2 20 61 2 19 60 DataReader DataReader 2 20 61 2 19 60 orange 21 62 80 orange 21 62 80 DataWriter DataWriter DataReader DataReader struct ShapeType { 3 25 70 3 25 71 25 3 25 74 3 26 77 3 26 77 DataWriter DataWriter long x; 3 25 70 3 25 71 25 3 25 74 long y; green 97 75 50 green 94 71 50 long shapesize; string color; Arrows }; #pragma keylist ShapeType color show structural relationship s, not data- flows Partition © 2009, PrismTech. All Rights Reserved
  • 42. Anatomy of a DDS Application Topic TopicTopic Circle) (e.g. Samples Samples Instances Samples Instances Instances 1 21 62 1 21 62 1 22 62 1 22 62 1 23 63 1 23 63 DataReader DataReader blue 21 62 80 blue 23 66 80 blue 27 70 80 2 20 61 2 19 60 DataReader DataReader 2 20 61 2 19 60 orange 21 62 80 orange 21 62 80 DataWriter DataWriter DataReader DataReader struct ShapeType { 3 25 70 3 25 71 25 3 25 74 3 26 77 3 26 77 DataWriter DataWriter long x; 3 25 70 3 25 71 25 3 25 74 long y; green 97 75 50 green 94 71 50 long shapesize; string color; Arrows }; #pragma keylist ShapeType color show structural relationship s, not data- Subscriber Publisher flows Partition Domain Participant Domain © 2009, PrismTech. All Rights Reserved
  • 43. OpenSplice DDS Delivering Performance, Openness, and Freedom From Shapes to Market Data
  • 44. The Market Data Topic Type Declaration Topic Instantiation struct StockQuote { char<4> symbol; // Create Topic with Default QoS char<4> exchange; dds::Topic<StockQuote> topic(“stock_quote”); char<64> name; float quote; // Create Content Filtered Topic dds::ContentFilteredTopic cfTopic = }; topic.create_content_filtered(expr, #pragma keylist StockQuote symbol params); symbol name exchange quote AAPL Apple Inc. NASD 165.37 GOOG Google Inc. NASD 663.97 MSFT Microsoft Corp. NASD 33.73 © 2009, PrismTech. All Rights Reserved
  • 45. Filtering Market Data Apple 164.0 Microsoft 40 161.5 30 MSFT > $35 159.0 20 AAPL < $157 |156.5 AAPL > $162 10 154.0 Untitled 1 Untitled 20 Untitled 39 Untitled 58 Untitled 77 Untitled 96 0 AAPL MSFT ‣ DDS continuous queries can be used for filtering the stream of data produced for a specific stock symbol. © 2009, PrismTech. All Rights Reserved
  • 46. Writing Market Data Creating a Writer // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Writer with Default Qos dds::Writer<StockQuote> writer(topic); © 2009, PrismTech. All Rights Reserved
  • 47. Writing Market Data Creating a Writer // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Writer with Default Qos dds::Writer<StockQuote> writer(topic); Writing Data StockQuote quote = { “AAPL”, “NASD”, 163.37, “Apple Inc.” }; writer.write(quote); © 2009, PrismTech. All Rights Reserved
  • 48. Writing Market Data Creating a Writer // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Writer with Default Qos dds::Writer<StockQuote> writer(topic); Writing Data Fastest Way of Writing Data StockQuote quote = { StockQuote quote = { “AAPL”, “NASD”, “AAPL”, “NASD”, 163.37, “Apple Inc.” 163.37, “Apple Inc.” }; }; dds::InstanceWriter iwriter = writer.write(quote); writer.register_instance(quote); iwriter.write(quote); © 2009, PrismTech. All Rights Reserved
  • 49. OpenSplice DDS Delivering Performance, Openness, and Freedom Topic Used in next section. Read vs. Take struct Counter { int cID; int count; }; #pragma keylist Counter cID
  • 50. Reading Samples ‣ Read iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Samples are not removed 2 1 2 2 2 3 from the local cache as DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 4 3 5 result of a read 3 1 3 2 3 3 ‣ Read samples can be read Topic again, by accessing the Samples Read Samples not Read cache with the proper options (more later) DataReader Cache © 2009, PrismTech. All Rights Reserved
  • 51. Reading Samples ‣ Read iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Samples are not removed 2 1 2 2 2 3 from the local cache as DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 3 2 3 3 3 4 3 5 result of a read ‣ Read samples can be read Topic again, by accessing the Samples Read Samples not Read cache with the proper options (more later) DataReader Cache © 2009, PrismTech. All Rights Reserved
  • 52. Reading Samples ‣ Read iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Samples are not removed 2 1 2 2 2 3 from the local cache as DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 3 2 3 3 3 4 3 5 result of a read ‣ Read samples can be read Topic again, by accessing the Samples Read Samples not Read cache with the proper options (more later) DataReader Cache © 2009, PrismTech. All Rights Reserved
  • 53. Taking Samples ‣ Take iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Taken Samples are 2 1 2 2 2 3 removed from the local DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 4 3 5 cache as result of a take 3 1 3 2 3 3 ‣ Topic Samples not Taken DataReader Cache © 2009, PrismTech. All Rights Reserved
  • 54. Taking Samples ‣ Take iterates over the available sample instances 1 2 1 3 1 4 ‣ Taken Samples are 2 2 2 3 removed from the local DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 2 3 3 3 4 3 5 cache as result of a take Topic Samples not Taken DataReader Cache © 2009, PrismTech. All Rights Reserved
  • 55. Taking Samples ‣ Take iterates over the available sample instances 1 3 1 4 ‣ Taken Samples are 2 3 removed from the local DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 3 3 4 3 5 cache as result of a take Topic Samples not Taken DataReader Cache © 2009, PrismTech. All Rights Reserved
  • 56. Reading Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); © 2009, PrismTech. All Rights Reserved
  • 57. Reading Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); Reading Data StockQuoteSeq data; dds::SampleInfoSeq info; reader.read(data, info); © 2009, PrismTech. All Rights Reserved
  • 58. Reading Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); Reading a Specific Instance Reading Data StockQuote quote; quote.symbol = “APPL”; StockQuoteSeq data; StockQuoteSeq data; dds::SampleInfoSeq info; dds::SampleInfoSeq info; reader.read(data, info); dds::InstanceReader ireader = reader.instance(quote); ireader.read(data, info); © 2009, PrismTech. All Rights Reserved
  • 59. Taking Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); © 2009, PrismTech. All Rights Reserved
  • 60. Taking Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); Taking Data StockQuoteSeq data; dds::SampleInfoSeq info; reader.take(data, info); © 2009, PrismTech. All Rights Reserved
  • 61. Taking Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); Taking a Specific Instance Taking Data StockQuote quote; quote.symbol = “APPL”; StockQuoteSeq data; StockQuoteSeq data; dds::SampleInfoSeq info; dds::SampleInfoSeq info; reader.take(data, info); dds::InstanceReader ireader = reader.instance(quote); ireader.take(data, info); © 2009, PrismTech. All Rights Reserved
  • 62. Sample, Instance and View States History Depth = 2 ‣ Along with data samples, DataReaders are provided 1 1 with state information allowing to detect relevant 2 2 transitions in the life-cycle of data as well as data 3 writers DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. SampleInfo ‣ Sample State (READ | NOT_READ): Determines wether a sample has already been read by this 1 1 1 2 DataReader or not. 2 2 2 3 ‣ Instance State (ALIVE, NOT_ALIVE, DISPOSED). 3 1 Determines wether (1) writer exist for the specific Samples instance, or (2) no matched writers are currently Topic available, or (3) the instance has been disposed DataReader Cache ‣ View State (NEW, NOT_NEW). Determines wether this is the first sample of a new (or re-born) instance © 2009, PrismTech. All Rights Reserved
  • 63. Would you let me know... DDS provides three main mechanism for exchanging information with the application ‣ Polling. The application polls from time to time for new data or status changes. Proprietary Information - Distribution without Expressed Written Permission is Prohibited. The interval might depend on the kind of applications as well as data ‣ WaitSets. The application registers a WaitSet with DDS and waits (i.e. is suspended) until one of the specified events has happened. ‣ Listeners. The application registers a listener with a specific DDS entity to be notified when relevant events occur, such as state changes or © 2009, PrismTech. All Rights Reserved
  • 64. Reading with Waitsets // Functor handling new data arrival 1 struct MarketDataHandler { void operator()(MarketDataReader& reader); }; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. © 2009, PrismTech. All Rights Reserved
  • 65. Reading with Waitsets // Functor handling new data arrival 1 struct MarketDataHandler { void operator()(MarketDataReader& reader); }; // Create & Register Condition 2 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. MarketDataHandler functor; dds::ReadCondition cond = reader.create_read_condition(functor); dds::WaitSet ws; ws.attach(cond); © 2009, PrismTech. All Rights Reserved
  • 66. Reading with Waitsets // Functor handling new data arrival 1 struct MarketDataHandler { void operator()(MarketDataReader& reader); }; // Create & Register Condition 2 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. MarketDataHandler functor; dds::ReadCondition cond = reader.create_read_condition(functor); dds::WaitSet ws; ws.attach(cond); // Wait & Handle 3’ dds::ConditionVector cvec = ws.wait(); for (int i = 0; i < cvec.size(); ++i) cvec[i].execute(); © 2009, PrismTech. All Rights Reserved
  • 67. Reading with Waitsets // Functor handling new data arrival 1 struct MarketDataHandler { void operator()(MarketDataReader& reader); }; // Create & Register Condition 2 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. MarketDataHandler functor; dds::ReadCondition cond = reader.create_read_condition(functor); dds::WaitSet ws; ws.attach(cond); // Wait & Handle 3’ // Dispatch 3” dds::ConditionVector cvec = ws.wait(); ws.dispatch(); for (int i = 0; i < cvec.size(); ++i) cvec[i].execute(); © 2009, PrismTech. All Rights Reserved
  • 68. Reading with Listeners // Functor handling new data arrival 1 class MarketDataListener { public: void handle_data(MarketDataReader& reader); }; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. © 2009, PrismTech. All Rights Reserved
  • 69. Reading with Listeners // Functor handling new data arrival 1 class MarketDataListener { public: void handle_data(MarketDataReader& reader); }; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 2 // Functor handling new data arrival MarketDataListener mdl; dds::connection cnt = reader.on_data_available_signal_connect( std::bind(&MarketDataListener::handle_data, &mdl, _1)); © 2009, PrismTech. All Rights Reserved
  • 70. OpenSplice DDS Delivering Performance, Openness, and Freedom Quality of Service
  • 71. QoS Model ‣ QoS-Policies are used to control relevant properties of OpenSplice DDS entities, Type Matching QoS matching such as: QoS QoS QoS QoS QoS QoS QoS ‣ Temporal Properties ‣ Priority Topic Name Publisher Subscriber ‣ Durability ... DataWriter writes Type reads DataReader ... ... ‣ Availability DomainParticipant writes Type reads DataReader DomainParticipant DataWriter ‣ ... Name ‣ Some QoS-Policies are matched based on Topic QoS QoS QoS a Request vs. Offered Model thus QoS- enforcement ‣ Publications and Subscriptions match only if the declared vs. requested QoS are compatible ‣ e.g., it is not possible to match a publisher which delivers data unreliably with a subscriber which requires reliability © 2009, PrismTech. All Rights Reserved
  • 72. QoS Policies QoS Policy Applicability RxO Modifiable Type Matching QoS matching DURABILITY T, DR, DW Y N Data Availability QoS QoS QoS QoS QoS QoS QoS DURABILITY SERVICE T, DW N N Topic Name Publisher Subscriber LIFESPAN T, DW - Y ... DataWriter writes Type reads DataReader ... T, DR, DW N N ... HISTORY DomainParticipant DataWriter writes Type reads DataReader DomainParticipant PRESENTATION P, S Y N Data Delivery Name Topic RELIABILITY T, DR, DW Y N PARTITION P, S N Y QoS QoS QoS DESTINATION ORDER T, DR, DW Y N OWNERSHIP T, DR, DW Y N ‣ Rich set of QoS allow to configure OWNERSHIP DW - Y STRENGTH several different aspects of data DEADLINE T, DR, DW Y Y Data Timeliness availability, delivery and timeliness LATENCY BUDGET T, DR, DW Y Y TRANSPORT PRIORITY T, DW - Y ‣ QoS can be used to control and optimize network as well as TIME BASED FILTER DR - Y Resources computing resource RESOURCE LIMITS T, DR, DW N N USER_DATA DP, DR, DW N Y Configuration TOPIC_DATA T N Y GROUP_DATA P, S N Y © 2009, PrismTech. All Rights Reserved
  • 73. Mapping QoS Which properties does QoS controls? TimeBasedFilter Deadline Data Throughput Latency LatencyBudget TransportPriority Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Control over Latency/Throughput tradeoff Control over data latency Control over data priority © 2009, PrismTech. All Rights Reserved
  • 74. Mapping QoS Which properties does QoS controls? TimeBasedFilter Deadline History Data Data Throughput Lifespan Durability Latency Availability Ownership LatencyBudget TransportPriority Ownership Strength Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Control over Latency/Throughput tradeoff Control over data queueing Control over data latency Control over data persistency Control over data priority Control over data sources hot-swap © 2009, PrismTech. All Rights Reserved
  • 75. Mapping QoS Which properties does QoS controls? TimeBasedFilter Deadline History Data Data Throughput Lifespan Durability Latency Availability Ownership LatencyBudget TransportPriority Ownership Strength Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Control over Latency/Throughput tradeoff Control over data queueing Control over data latency Control over data persistency Control over data priority Control over data sources hot-swap Reliability OpenSplice DDS provides programmatic Destination QoS-driven support for configuring the most Presentation Data Delivery Order important properties of data distribution! Control over data distribution reliability Control over data ordering Control over presentation © 2009, PrismTech. All Rights Reserved
  • 76. OpenSplice DDS Delivering Performance, Openness, and Freedom Playing with QoS
  • 77. History QoS History Depth = 5 1 1 1 2 1 3 1 4 1 5 DataReader 2 1 2 2 2 3 2 4 2 5 3 1 3 2 3 3 3 4 3 5 1 // Create Reader QoS Topic Proprietary Information - Distribution without Expressed Written Permission is Prohibited. dds::DataReaderQos drqos; drqos.set_keep_last(N); // Create Reader ‣ The History QoS Controls the number of dds::DataReader<MarketData> reader(topic, drqos); samples-per-instance that will be stored by the middleware on behalf of a Reader 2 ‣ Keep Last K. The History QoS can be set so // Create Reader QoS to always have the latest K samples dds::DataReaderQos drqos; drqos.set_keep_all(); ‣ Keep All. The History QoS can be set so // Create Reader keep all samples produced by the writer and dds::DataReader<MarketData> reader(topic, drqos); not yet taken, until resource limits are not reached © 2009, PrismTech. All Rights Reserved
  • 78. Transport Priority QoS Make sure that important data gets the right priority Shared Single Copy per Node Shared Pack Across Topics/Applications Memory Optimal Unmarshaling Memory 1 // Create Writer QoS dds::DataWriterQos dwqos; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. dwqos.set_priority(MID_PRIO); OpenSplice DDS OpenSplice DDS // Create Writer Binding Pre-emptive Network Scheduler Binding dds::DataWriter<MarketData> writer(topic, dwqos); Networking Priority Scheduler Networking Data Urgency Traffic Pacing // Write Data writer.write(data); Network Channels // Change Priority Priority Bands dwqos.set_priority(HIGH_PRIO); Traffic Shaping writer.writer(data) © 2009, PrismTech. All Rights Reserved
  • 79. Ownership QoS Availability of data producers can be controlled via two QoS Policies ‣ OWNERSHIP ‣ OWNERSHIP STRENGTH ‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote symbol: "MSFT" StockQuote StockQuote name: "Microsoft Corp." StockQuote symbol: "GOOG" symbol: "GOOG" exchange: "NASD""GOOG" symbol: name: "Google Inc." quote: 33.73 name: "Google Inc." name: "Google Inc." exchange: "NASD" exchange: "NASD" exchange: "NASD" StockQuote StockQuote StockQuote quote: 663.97 W1 quote: 663.97"AAPL" symbol: "AAPL" quote: 663.97 symbol: "AAPL" symbol: name: "Apple Inc." name: "Apple Inc." name: "Apple Inc." exchange: "NASD" exchange: "NASD" exchange: "NASD" R1 quote: 165.37 quote: 165.37 quote: 165.37 STRENGTH=3 StockQuote symbol: "AAPL" StockQuote Inc." name: "Apple symbol: "GOOG""NASD" exchange: name: quote: 165.37 "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W1’ R2 STRENGTH=2 StockQuote symbol: "AAPL" StockQuote Inc." name: "Apple symbol: "GOOG""NASD" exchange: name: quote: 165.37 "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W2’’ R3 STRENGTH=1 © 2009, PrismTech. All Rights Reserved
  • 80. Ownership QoS Availability of data producers can be controlled via two QoS Policies ‣ OWNERSHIP ‣ OWNERSHIP STRENGTH ‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote symbol: "MSFT" StockQuote StockQuote name: "Microsoft Corp." StockQuote symbol: "GOOG" symbol: "GOOG" exchange: "NASD""GOOG" symbol: name: "Google Inc." quote: 33.73 name: "Google Inc." name: "Google Inc." exchange: "NASD" exchange: "NASD" exchange: "NASD" quote: 663.97 W1 quote: 663.97 quote: 663.97 StockQuote symbol: "AAPL" R1 STRENGTH=3 name: "Apple Inc." exchange: "NASD" quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W1’ R2 StockQuote symbol: "AAPL" name: "Apple Inc." exchange: "NASD" STRENGTH=2 quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W2’’ StockQuote symbol: "AAPL" name: "Apple Inc." exchange: "NASD" quote: 165.37 R3 STRENGTH=1 © 2009, PrismTech. All Rights Reserved
  • 81. Ownership QoS Availability of data producers can be controlled via two QoS Policies ‣ OWNERSHIP ‣ OWNERSHIP STRENGTH ‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote symbol: "MSFT" name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W1 StockQuote symbol: "AAPL" R1 STRENGTH=3 name: "Apple Inc." exchange: "NASD" quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote symbol: "MSFT" name: "Microsoft Corp." exchange: "NASD" quote: 33.73 StockQuote W1’ R2 symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote symbol: "AAPL" name: "Apple Inc." exchange: "NASD" STRENGTH=2 quote: 165.37 StockQuote symbol: "MSFT" name: "Microsoft Corp." exchange: "NASD" StockQuote quote: 33.73 symbol: "GOOG" R3 name: "Google Inc." W2’’ exchange: "NASD"StockQuote quote: 663.97symbol: "AAPL" name: "Apple Inc." exchange: "NASD" quote: 165.37 STRENGTH=1 © 2009, PrismTech. All Rights Reserved
  • 82. Ownership QoS Availability of data producers can be controlled via two QoS Policies ‣ OWNERSHIP ‣ OWNERSHIP STRENGTH ‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote R1 symbol: "AAPL" name: "Apple Inc." exchange: "NASD" quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote symbol: "AAPL" name: "Apple Inc." exchange: "NASD" R2 quote: 165.37 StockQuote StockQuote StockQuote StockQuote symbol: "MSFT" symbol: "MSFT" symbol: "MSFT" symbol: "GOOG" name: "Microsoft Corp." name: "Microsoft Corp." name: "Microsoft Corp." exchange: "NASD" exchange: "NASD" name: "Google Inc." exchange: "NASD" quote: 33.73 quote: 33.73 quote: 33.73 exchange: "NASD" quote: 663.97 StockQuote R3 symbol: "AAPL" W2’’ name: "Apple Inc." exchange: "NASD" quote: 165.37 STRENGTH=1 © 2009, PrismTech. All Rights Reserved
  • 83. Ownership QoS Availability of data producers can be controlled via two QoS Policies ‣ OWNERSHIP ‣ OWNERSHIP STRENGTH ‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote R1 symbol: "AAPL" name: "Apple Inc." exchange: "NASD" StockQuote quote: 165.37 symbol: "MSFT" name: "Microsoft Corp." exchange: "NASD" quote: 33.73 StockQuote symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote R2 symbol: "MSFT" name: "Microsoft Corp." StockQuote exchange: "NASD" quote: 33.73 symbol: "AAPL" name: "Apple Inc." exchange: "NASD" quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." StockQuote exchange: "NASD" symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 StockQuote R3 symbol: "AAPL" W2’’ name: "Apple Inc." exchange: "NASD" quote: 165.37 STRENGTH=1 © 2009, PrismTech. All Rights Reserved
  • 84. OpenSplice DDS Delivering Performance, Openness, and Freedom Online References
  • 85. http://www.opensplice.org/
  • 86. SIMD Project http://code.google.com/p/simd-cxx/ © 2009, PrismTech. All Rights Reserved Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
  • 87. OpenSplice DDS Delivering Performance, Openness, and Freedom Summing Up
  • 88. Concluding Remarks ‣ DDS is the ideal fit for addressing several of the data distribution and management challenges faced by financial firms, such as: OpenSplice DDS Delivering Performance, Openness, and Freedom ‣ Market Data Distribution ‣ High Performance Caching Enterprise Ed. ‣ Real-Time Business Intelligence Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ OpenSplice DDS is an Open Source Professional Ed. implementation of the DDS standard widely deployed and delivering great performance Compact Ed. ‣ OpenSplice DDS is Commercially Supported Open Source Software Community Ed. Embrace Performance, Openness and Freedom. Join the OpenSplice DDS Community at www.opensplice.org © 2009, PrismTech. All Rights Reserved
  • 89. Online Resources http://www.opensplice.com/ http://www.slideshare.net/angelo.corsaro emailto:opensplicedds@prismtech.com http://bit.ly/1Sreg http://twitter.com/acorsaro/ http://opensplice.blogspot.com http://www.youtube.com/OpenSpliceTube © 2009, PrismTech. All Rights Reserved

×