• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
I need the data and I need it 1ms ago!
 

I need the data and I need it 1ms ago!

on

  • 3,057 views

This presentation was given at SkillsMatter "Open Source in Finance and Exchange" on 15 Jan 2009 (http://bit.ly/8VfSes). The presentation provides an introduction to the key DDS concepts as well as ...

This presentation was given at SkillsMatter "Open Source in Finance and Exchange" on 15 Jan 2009 (http://bit.ly/8VfSes). The presentation provides an introduction to the key DDS concepts as well as its programming.

Statistics

Views

Total Views
3,057
Views on SlideShare
2,654
Embed Views
403

Actions

Likes
1
Downloads
133
Comments
0

9 Embeds 403

http://skillsmatter.com 287
http://www.enscada.com 69
http://www.prismtech.com 37
http://www.slideshare.net 4
http://mail.skillsmatter.com 2
http://translate.googleusercontent.com 1
http://192.168.56.101 1
http://smash 1
http://webmail.skillsmatter.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    I need the data and I need it 1ms ago! I need the data and I need it 1ms ago! Presentation Transcript

    • OpenSplice DDS Delivering Performance, Openness, and Freedom I Need Market Data Angelo Corsaro, Ph.D. Chief Technology Officer OMG DDS SIG Co-Chair angelo.corsaro@prismtech.com and I need it 1ms Ago!
    • OpenSplice DDS Delivering Performance, Openness, and Freedom Challenges
    • Financial Firms Trends ‣ Direct connection to exchanges is increasingly chosen over data aggregators to reduce trading latency ‣ Market data is growing steadily ‣ Milliseconds speedup over competitors can be worth millions of dollars Challenges ‣ Process several hundreds-of-thousands of messages, quickly approaching millions messages, per second ‣ Minimize latency and jitter, while maintaining the system stability and reliability Deployed middleware solutions were not designed to address these challenges! © 2009, PrismTech. All Rights Reserved
    • Exchanges Trends ‣ More and more firms connecting directly ‣ Increasing number of sources providing the same quotes ‣ Competitive advantage obtained by reducing latency on data feeds Challenges ‣ Ensuring fairness of data distribution ‣ Minimizing latency ‣ Scaling with the number of destinations Deployed middleware solutions were not designed to address these challenges! © 2009, PrismTech. All Rights Reserved
    • OPRA Predictions OPRA Market Data Rates Forecasts Trends 2000000 Msgs/sec 1807000 ‣ One order of magnitude growth on market data from 2005 to 2008 1500000 ‣ OPRA message rates predicted to reach ~2M 1000000 msgs/sec by early 2009 701000 ‣ Traffic bursts predicted to further increase by 20% 500000 359000 Challenges ‣ Can your messaging 190000 Jan 6, 2009 Jan 8, 2008 88000 Jan 9, 2007 0 May 16, 2006 infrastructure keep-up Jan 13, 2005 with this trend? © 2009, PrismTech. All Rights Reserved
    • Trading with Pareto ‣ Usually 20% of the trades provides 80% of the gains Msgs/sec ‣ Under overload condition how can you ensure you keep trading at least 20% of your target? Vilfredo Pareto, 1848-1923 ‣ How can you ensure that you keep trading the the Average right 20% of data? System Message Rate Incoming Message Rate Time © 2009, PrismTech. All Rights Reserved
    • OpenSplice DDS Delivering Performance, Openness, and Freedom The Data Distribution Service for Real-Time Systems (DDS)
    • Addressing Data Distribution Challenges DDS is standard designed to address the data-distribution challenges across The OMG DDS Standard a wide class of Defense and Aerospace Applications ‣ Introduced in 2004 to address the Data Distribution challenges faced by a wide class of Defense and Aerospace Applications ‣ Key requirement for the standard were its ability to deliver very high performance while seamlessly scaling from embedded to ultra- large-scale deployments ‣ Today recommended by key administration worldwide and widely adopted across several different application domains, such as, Automated Trading, Simulations, SCADA, Telemetry, etc. © 2009, PrismTech. All Rights Reserved
    • The OMG Data Distribution Service (DDS) DDS v1.2 API Standard ‣ Language Independent, OS and HW architecture Application independent Object/Relational Mapping ‣ DCPS. Standard API for Data-Centric, Topic- Data Local Reconstruction Layer (DLRL) Based, Real-Time Publish/Subscribe Content ‣ Ownership Durability DLRL. Standard API for creating Object Views out Subscription of collection of Topics Minimum Profile Data Centric Publish/Subscribe (DCPS) DDSI/RTPS v2.1 Wire Protocol Standard ‣ Standard wire protocol allowing interoperability Real-Time Publish/Subscribe Protocol DDS Interoperability Wire Protocol between different implementations of the DDS standard UDP/IP ‣ Interoperability demonstrated among key DDS vendors in March 2009 © 2009, PrismTech. All Rights Reserved
    • Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Data Local Reconstruction Layer (DLRL) DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Profile Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved
    • Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Data Local Reconstruction Layer (DLRL) DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Profile Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Wire Protocol Standard Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved
    • Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Data Local Reconstruction Layer (DLRL) DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Profile Pub/Sub Standard API Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Wire Protocol Standard Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved
    • Scope of Standardization AMQP OMG DDS Application Application Object/Relational Mapping Object Relational Mapping Data Local Reconstruction Layer (DLRL) Standard API DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Profile Pub/Sub Standard API Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol Wire Protocol Standard Interoperability Wire Protocol DDS Interoperability Wire Protocol (TCP, SCTP, UDP)/IP UDP/IP © 2009, PrismTech. All Rights Reserved
    • Architectural Style AMQP OMG DDS Subscriber Subscriber Publisher Publisher Broker Broker Publisher Broker Subscriber Publisher Broker Subscriber Broker Broker Publisher Publisher Subscriber Subscriber © 2009, PrismTech. All Rights Reserved
    • Key Differences in Summary DDS AMQP ‣ Peer-to-Peer Communication ‣ Brokered Communication Architecture Architecture ‣ Wire Protocol + API Standard ‣ Wire Protocol Standard ‣ Interoperability across DDS implementations ‣ Interoperability across AMQP implementations ‣ Application Portability across DDS Implementation ‣ Content-Based Filtering/Quering ‣ Header Based Routing ‣ Designed bottom-up for maximum ‣ Not geared for High-Performance performance and determinism © 2009, PrismTech. All Rights Reserved
    • OpenSplice DDS Delivering Performance, Openness, and Freedom DDS and Open Source
    • OpenSplice DDS ‣ Product reorganized into Editions ‣ Community Edition Enterprise Ed. ‣ Compact Edition ‣ Professional Edition ‣ Enterprise Edition Professional Ed. Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Product Editions provide a a growing set of Enterprise Ed. Compact Ed. functionality to address the needs of increasingly more sophisticated users ‣ The Community Edition is Open Source Community Ed. ‣ Compact, Enterprise and Professional Edition are available only through Commercial Subscriptions © 2009, PrismTech. All Rights Reserved
    • Community Edition Features Licensing ‣ OMG DDS v1.2 DCPS ‣ LGPLv3 ‣ Minimum Profile ‣ Content Subscription Profile Application ‣ Durability Profile Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Ownership Profile Content DDS v1.2 Ownership Durability Subscription ‣ Networking Minimum Profile ‣ DDSI v2.1 Implementation Real-Time Pub/Sub (DCPS) ‣ Real-Time Networking Implementation DDSI v2.1 Interoperable Wire Protocol Real-Time Networking (DDSI) Networking Technology UDP/IP Open Source LGPL Licensed Edition -- The Best Way to Get Started with DDS © 2009, PrismTech. All Rights Reserved
    • Performance on Commodity HW Throughput -./0/1/23' 456/' Latency #+(+%++' &&++%++' $+(+%++' "*(%)+' ")(%&!' ""&%,&'&+++%++' Inter-Node Latency !"#$%&#' ‣ 60 usec "!+%++' )"#%*)' "++%++' )$,%$*' !+(+%++' )++%++' Inter-Core Read-Latency &'$($)$*+% Proprietary Information - Distribution without Expressed Written Permission is Prohibited. *!#%,&' ‣ 2 usec !"#$% ,+(+%++' *++%++' ()(!%(*' #++%++' (+(+%++' &")&%+)' Inter-Core Latency ‣ <10 usec $&"%,,' $++%++' &!(,%!)' &+(+%++' ),,%$#' !++%++' ,*+%&)' !,*%))' ,($%!+' (("%!"' &&)%*$' $"%"$' ,+%($' ,++%++' HW: (+%++' )' &#' ,(' #!' &()' ($#' $&(' &+(!' (+!)' !+"#' ‣ Dell blade-server !*$$,(*%-./*% ‣ Dual-core, Dual-CPU, AMD Opteron 2.4 Ghz Test Scenario OS ‣ Linux 2.6.21-1.3194.fc7 ‣ Single Threaded Application (multi-threaded networking service) Network ‣ 8192 bit message batches ‣ Gigabit Ethernet cards ‣ Dell PowerConnect 5324 switch © 2009, PrismTech. All Rights Reserved
    • OpenSplice DDS Adoption Defense ‣ Combat Management Systems ‣ Tactical Systems ‣ Drones (land/air) ‣ Communication Systems Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Radar Processing ‣ Submarine Systems ‣ Future Combat Systems SCADA/Utilities ‣ Industrial Automation ‣ Power Grids (Smart Grids) ‣ Complex Telemetry (e.g. NASA Rocket Launch System) © 2009, PrismTech. All Rights Reserved
    • OpenSplice DDS Adoption Transportation ‣ Air Traffic Management/Control ‣ Metropolitan Traffic Management ‣ Underground Metropolitan Transportation Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Financial Services ‣ Automated Trading ‣ Risk Management ‣ Option Trading © 2009, PrismTech. All Rights Reserved
    • Some OpenSplice DDS Users !"#$%&'()('*+'##,-+.+/($$,012"0 Experti réalisée par Bernard M © 2009, PrismTech. All Rights Reserved
    • OpenSplice DDS Delivering Performance, Openness, and Freedom Understanding DDS
    • As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Space (GDS) Brokers Global Data Space DDS © 2009, PrismTech. All Rights Reserved
    • As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Global Data Space DDS © 2009, PrismTech. All Rights Reserved
    • As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber Publisher Global Data Space Publisher Subscriber DDS © 2009, PrismTech. All Rights Reserved
    • As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber ‣ Publishers and Subscribers Publisher Global Data Space express their intent to produce/ consume specific type of data, e.g., Topics Publisher Subscriber DDS © 2009, PrismTech. All Rights Reserved
    • As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber ‣ Publishers and Subscribers Publisher Global Data Space express their intent to produce/ consume specific type of data, e.g., Topics Publisher Subscriber DDS © 2009, PrismTech. All Rights Reserved
    • As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber ‣ Publishers and Subscribers Publisher Global Data Space express their intent to produce/ consume specific type of data, e.g., Topics Subscriber ‣ Data flows from Publisher to Publisher Subscribers DDS © 2009, PrismTech. All Rights Reserved
    • As Simple as it Gets ‣ DDS is based around the concept of a fully distributed Global Data Publisher Subscriber Space (GDS) ‣ Publishers and Subscribers can Brokers join and leave the GDS at any time Subscriber ‣ Publishers and Subscribers Publisher Global Data Space express their intent to produce/ consume specific type of data, e.g., Topics Subscriber ‣ Data flows from Publisher to Publisher Subscribers DDS © 2009, PrismTech. All Rights Reserved
    • OpenSplice DDS Delivering Performance, Openness, and Freedom Defining Data
    • DDS Topics {Circle, Square, Triangle} Topic ‣ Unit of information exchanged between Publisher and Subscribers. ‣ An association between a unique name, a type and a QoS setting {ShapeType} {...} Topic Type. struct ShapeType { ‣ Type describing the data associated with one long x; or more Topics long y; ‣ A Topic type can have a key represented by an long shapesize; string color; arbitrary number of attributes }; ‣ Expressed in IDL (or XML) #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
    • DDS Topics Circle struct ShapeType { long x; long y; long shapesize; Topic QoS string color; }; #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
    • DDS Topics Triangle struct ShapeType { long x; long y; long shapesize; Topic QoS string color; }; #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
    • DDS Topic Instances and Samples Topic Instances ‣ Each key value identifies a unique Topic Instance, ‣ Topic’s instance lifetime can be explicitly managed in DDS Topic Samples ‣ The values assumed by a Topic Instance over time are referred as Instance Sample struct ShapeType { long x; long y; long shapesize; string color; }; #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
    • Topic/Instances/Samples Recap. Topics Instances Samples ti tj tnow time © 2009, PrismTech. All Rights Reserved
    • Content Filtering X0 X1 X0 <= X <= X1 ‣ DDS allows to specify content- filtered Topics for which a subset of SQL92 is used to express the filter condition Y0 ‣ Content filters can be applied on the entire content of the Topic Type Y0 <= Y <= Y1 ‣ Content filters are applied by DDS Y1 each time a new sample is produced/delivered © 2009, PrismTech. All Rights Reserved
    • Local Queries ‣ Subscribed Topics can be seen locally as “Tables” ‣ A subset of SQL92 can be used for performing queries on X0 multiple topics as well as natural joins ‣ Queries are Circle Topic performed under Y0 color x y shapesize Y0 user control and red 57 62 50 provide a result that blue 90 85 50 Y0 <= Y <= Y1 depends on the yellow 30 25 50 current snapshot of SELECT * FROM ShapeType s WHERE s.x > 25 AND s.y < 55 Y1 the system, e.g., samples currently color x y shapesize available yellow 30 25 50 © 2009, PrismTech. All Rights Reserved
    • OpenSplice DDS Delivering Performance, Openness, and Freedom Anatomy of a DDS App.
    • Anatomy of a DDS Application Topic TopicTopic Circle) (e.g. Samples Samples Instances Samples Instances Instances 1 21 62 1 21 62 1 22 62 1 22 62 1 23 63 1 23 63 DataReader DataReader blue 21 62 80 blue 23 66 80 blue 27 70 80 2 20 61 2 19 60 DataReader DataReader 2 20 61 2 19 60 orange 21 62 80 orange 21 62 80 DataWriter DataWriter DataReader DataReader struct ShapeType { 3 25 70 3 25 71 25 3 25 74 3 26 77 3 26 77 DataWriter DataWriter long x; 3 25 70 3 25 71 25 3 25 74 long y; green 97 75 50 green 94 71 50 long shapesize; string color; }; #pragma keylist ShapeType color © 2009, PrismTech. All Rights Reserved
    • Anatomy of a DDS Application Topic TopicTopic Circle) (e.g. Samples Samples Instances Samples Instances Instances 1 21 62 1 21 62 1 22 62 1 22 62 1 23 63 1 23 63 DataReader DataReader blue 21 62 80 blue 23 66 80 blue 27 70 80 2 20 61 2 19 60 DataReader DataReader 2 20 61 2 19 60 orange 21 62 80 orange 21 62 80 DataWriter DataWriter DataReader DataReader struct ShapeType { 3 25 70 3 25 71 25 3 25 74 3 26 77 3 26 77 DataWriter DataWriter long x; 3 25 70 3 25 71 25 3 25 74 long y; green 97 75 50 green 94 71 50 long shapesize; string color; Arrows }; #pragma keylist ShapeType color show structural relationship s, not data- flows Partition © 2009, PrismTech. All Rights Reserved
    • Anatomy of a DDS Application Topic TopicTopic Circle) (e.g. Samples Samples Instances Samples Instances Instances 1 21 62 1 21 62 1 22 62 1 22 62 1 23 63 1 23 63 DataReader DataReader blue 21 62 80 blue 23 66 80 blue 27 70 80 2 20 61 2 19 60 DataReader DataReader 2 20 61 2 19 60 orange 21 62 80 orange 21 62 80 DataWriter DataWriter DataReader DataReader struct ShapeType { 3 25 70 3 25 71 25 3 25 74 3 26 77 3 26 77 DataWriter DataWriter long x; 3 25 70 3 25 71 25 3 25 74 long y; green 97 75 50 green 94 71 50 long shapesize; string color; Arrows }; #pragma keylist ShapeType color show structural relationship s, not data- Subscriber Publisher flows Partition Domain Participant Domain © 2009, PrismTech. All Rights Reserved
    • OpenSplice DDS Delivering Performance, Openness, and Freedom From Shapes to Market Data
    • The Market Data Topic Type Declaration Topic Instantiation struct StockQuote { char<4> symbol; // Create Topic with Default QoS char<4> exchange; dds::Topic<StockQuote> topic(“stock_quote”); char<64> name; float quote; // Create Content Filtered Topic dds::ContentFilteredTopic cfTopic = }; topic.create_content_filtered(expr, #pragma keylist StockQuote symbol params); symbol name exchange quote AAPL Apple Inc. NASD 165.37 GOOG Google Inc. NASD 663.97 MSFT Microsoft Corp. NASD 33.73 © 2009, PrismTech. All Rights Reserved
    • Filtering Market Data Apple 164.0 Microsoft 40 161.5 30 MSFT > $35 159.0 20 AAPL < $157 |156.5 AAPL > $162 10 154.0 Untitled 1 Untitled 20 Untitled 39 Untitled 58 Untitled 77 Untitled 96 0 AAPL MSFT ‣ DDS continuous queries can be used for filtering the stream of data produced for a specific stock symbol. © 2009, PrismTech. All Rights Reserved
    • Writing Market Data Creating a Writer // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Writer with Default Qos dds::Writer<StockQuote> writer(topic); © 2009, PrismTech. All Rights Reserved
    • Writing Market Data Creating a Writer // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Writer with Default Qos dds::Writer<StockQuote> writer(topic); Writing Data StockQuote quote = { “AAPL”, “NASD”, 163.37, “Apple Inc.” }; writer.write(quote); © 2009, PrismTech. All Rights Reserved
    • Writing Market Data Creating a Writer // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Writer with Default Qos dds::Writer<StockQuote> writer(topic); Writing Data Fastest Way of Writing Data StockQuote quote = { StockQuote quote = { “AAPL”, “NASD”, “AAPL”, “NASD”, 163.37, “Apple Inc.” 163.37, “Apple Inc.” }; }; dds::InstanceWriter iwriter = writer.write(quote); writer.register_instance(quote); iwriter.write(quote); © 2009, PrismTech. All Rights Reserved
    • OpenSplice DDS Delivering Performance, Openness, and Freedom Topic Used in next section. Read vs. Take struct Counter { int cID; int count; }; #pragma keylist Counter cID
    • Reading Samples ‣ Read iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Samples are not removed 2 1 2 2 2 3 from the local cache as DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 4 3 5 result of a read 3 1 3 2 3 3 ‣ Read samples can be read Topic again, by accessing the Samples Read Samples not Read cache with the proper options (more later) DataReader Cache © 2009, PrismTech. All Rights Reserved
    • Reading Samples ‣ Read iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Samples are not removed 2 1 2 2 2 3 from the local cache as DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 3 2 3 3 3 4 3 5 result of a read ‣ Read samples can be read Topic again, by accessing the Samples Read Samples not Read cache with the proper options (more later) DataReader Cache © 2009, PrismTech. All Rights Reserved
    • Reading Samples ‣ Read iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Samples are not removed 2 1 2 2 2 3 from the local cache as DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 1 3 2 3 3 3 4 3 5 result of a read ‣ Read samples can be read Topic again, by accessing the Samples Read Samples not Read cache with the proper options (more later) DataReader Cache © 2009, PrismTech. All Rights Reserved
    • Taking Samples ‣ Take iterates over the available sample instances 1 1 1 2 1 3 1 4 ‣ Taken Samples are 2 1 2 2 2 3 removed from the local DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 4 3 5 cache as result of a take 3 1 3 2 3 3 ‣ Topic Samples not Taken DataReader Cache © 2009, PrismTech. All Rights Reserved
    • Taking Samples ‣ Take iterates over the available sample instances 1 2 1 3 1 4 ‣ Taken Samples are 2 2 2 3 removed from the local DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 2 3 3 3 4 3 5 cache as result of a take Topic Samples not Taken DataReader Cache © 2009, PrismTech. All Rights Reserved
    • Taking Samples ‣ Take iterates over the available sample instances 1 3 1 4 ‣ Taken Samples are 2 3 removed from the local DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 3 3 3 4 3 5 cache as result of a take Topic Samples not Taken DataReader Cache © 2009, PrismTech. All Rights Reserved
    • Reading Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); © 2009, PrismTech. All Rights Reserved
    • Reading Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); Reading Data StockQuoteSeq data; dds::SampleInfoSeq info; reader.read(data, info); © 2009, PrismTech. All Rights Reserved
    • Reading Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); Reading a Specific Instance Reading Data StockQuote quote; quote.symbol = “APPL”; StockQuoteSeq data; StockQuoteSeq data; dds::SampleInfoSeq info; dds::SampleInfoSeq info; reader.read(data, info); dds::InstanceReader ireader = reader.instance(quote); ireader.read(data, info); © 2009, PrismTech. All Rights Reserved
    • Taking Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); © 2009, PrismTech. All Rights Reserved
    • Taking Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); Taking Data StockQuoteSeq data; dds::SampleInfoSeq info; reader.take(data, info); © 2009, PrismTech. All Rights Reserved
    • Taking Data Creating a Reader // Create Topic with Default QoS dds::Topic<StockQuote> topic(“stock_quote”); // Create a Reader with Default Qos dds::Reader<StockQuote> reader(topic); Taking a Specific Instance Taking Data StockQuote quote; quote.symbol = “APPL”; StockQuoteSeq data; StockQuoteSeq data; dds::SampleInfoSeq info; dds::SampleInfoSeq info; reader.take(data, info); dds::InstanceReader ireader = reader.instance(quote); ireader.take(data, info); © 2009, PrismTech. All Rights Reserved
    • Sample, Instance and View States History Depth = 2 ‣ Along with data samples, DataReaders are provided 1 1 with state information allowing to detect relevant 2 2 transitions in the life-cycle of data as well as data 3 writers DataReader Proprietary Information - Distribution without Expressed Written Permission is Prohibited. SampleInfo ‣ Sample State (READ | NOT_READ): Determines wether a sample has already been read by this 1 1 1 2 DataReader or not. 2 2 2 3 ‣ Instance State (ALIVE, NOT_ALIVE, DISPOSED). 3 1 Determines wether (1) writer exist for the specific Samples instance, or (2) no matched writers are currently Topic available, or (3) the instance has been disposed DataReader Cache ‣ View State (NEW, NOT_NEW). Determines wether this is the first sample of a new (or re-born) instance © 2009, PrismTech. All Rights Reserved
    • Would you let me know... DDS provides three main mechanism for exchanging information with the application ‣ Polling. The application polls from time to time for new data or status changes. Proprietary Information - Distribution without Expressed Written Permission is Prohibited. The interval might depend on the kind of applications as well as data ‣ WaitSets. The application registers a WaitSet with DDS and waits (i.e. is suspended) until one of the specified events has happened. ‣ Listeners. The application registers a listener with a specific DDS entity to be notified when relevant events occur, such as state changes or © 2009, PrismTech. All Rights Reserved
    • Reading with Waitsets // Functor handling new data arrival 1 struct MarketDataHandler { void operator()(MarketDataReader& reader); }; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. © 2009, PrismTech. All Rights Reserved
    • Reading with Waitsets // Functor handling new data arrival 1 struct MarketDataHandler { void operator()(MarketDataReader& reader); }; // Create & Register Condition 2 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. MarketDataHandler functor; dds::ReadCondition cond = reader.create_read_condition(functor); dds::WaitSet ws; ws.attach(cond); © 2009, PrismTech. All Rights Reserved
    • Reading with Waitsets // Functor handling new data arrival 1 struct MarketDataHandler { void operator()(MarketDataReader& reader); }; // Create & Register Condition 2 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. MarketDataHandler functor; dds::ReadCondition cond = reader.create_read_condition(functor); dds::WaitSet ws; ws.attach(cond); // Wait & Handle 3’ dds::ConditionVector cvec = ws.wait(); for (int i = 0; i < cvec.size(); ++i) cvec[i].execute(); © 2009, PrismTech. All Rights Reserved
    • Reading with Waitsets // Functor handling new data arrival 1 struct MarketDataHandler { void operator()(MarketDataReader& reader); }; // Create & Register Condition 2 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. MarketDataHandler functor; dds::ReadCondition cond = reader.create_read_condition(functor); dds::WaitSet ws; ws.attach(cond); // Wait & Handle 3’ // Dispatch 3” dds::ConditionVector cvec = ws.wait(); ws.dispatch(); for (int i = 0; i < cvec.size(); ++i) cvec[i].execute(); © 2009, PrismTech. All Rights Reserved
    • Reading with Listeners // Functor handling new data arrival 1 class MarketDataListener { public: void handle_data(MarketDataReader& reader); }; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. © 2009, PrismTech. All Rights Reserved
    • Reading with Listeners // Functor handling new data arrival 1 class MarketDataListener { public: void handle_data(MarketDataReader& reader); }; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. 2 // Functor handling new data arrival MarketDataListener mdl; dds::connection cnt = reader.on_data_available_signal_connect( std::bind(&MarketDataListener::handle_data, &mdl, _1)); © 2009, PrismTech. All Rights Reserved
    • OpenSplice DDS Delivering Performance, Openness, and Freedom Quality of Service
    • QoS Model ‣ QoS-Policies are used to control relevant properties of OpenSplice DDS entities, Type Matching QoS matching such as: QoS QoS QoS QoS QoS QoS QoS ‣ Temporal Properties ‣ Priority Topic Name Publisher Subscriber ‣ Durability ... DataWriter writes Type reads DataReader ... ... ‣ Availability DomainParticipant writes Type reads DataReader DomainParticipant DataWriter ‣ ... Name ‣ Some QoS-Policies are matched based on Topic QoS QoS QoS a Request vs. Offered Model thus QoS- enforcement ‣ Publications and Subscriptions match only if the declared vs. requested QoS are compatible ‣ e.g., it is not possible to match a publisher which delivers data unreliably with a subscriber which requires reliability © 2009, PrismTech. All Rights Reserved
    • QoS Policies QoS Policy Applicability RxO Modifiable Type Matching QoS matching DURABILITY T, DR, DW Y N Data Availability QoS QoS QoS QoS QoS QoS QoS DURABILITY SERVICE T, DW N N Topic Name Publisher Subscriber LIFESPAN T, DW - Y ... DataWriter writes Type reads DataReader ... T, DR, DW N N ... HISTORY DomainParticipant DataWriter writes Type reads DataReader DomainParticipant PRESENTATION P, S Y N Data Delivery Name Topic RELIABILITY T, DR, DW Y N PARTITION P, S N Y QoS QoS QoS DESTINATION ORDER T, DR, DW Y N OWNERSHIP T, DR, DW Y N ‣ Rich set of QoS allow to configure OWNERSHIP DW - Y STRENGTH several different aspects of data DEADLINE T, DR, DW Y Y Data Timeliness availability, delivery and timeliness LATENCY BUDGET T, DR, DW Y Y TRANSPORT PRIORITY T, DW - Y ‣ QoS can be used to control and optimize network as well as TIME BASED FILTER DR - Y Resources computing resource RESOURCE LIMITS T, DR, DW N N USER_DATA DP, DR, DW N Y Configuration TOPIC_DATA T N Y GROUP_DATA P, S N Y © 2009, PrismTech. All Rights Reserved
    • Mapping QoS Which properties does QoS controls? TimeBasedFilter Deadline Data Throughput Latency LatencyBudget TransportPriority Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Control over Latency/Throughput tradeoff Control over data latency Control over data priority © 2009, PrismTech. All Rights Reserved
    • Mapping QoS Which properties does QoS controls? TimeBasedFilter Deadline History Data Data Throughput Lifespan Durability Latency Availability Ownership LatencyBudget TransportPriority Ownership Strength Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Control over Latency/Throughput tradeoff Control over data queueing Control over data latency Control over data persistency Control over data priority Control over data sources hot-swap © 2009, PrismTech. All Rights Reserved
    • Mapping QoS Which properties does QoS controls? TimeBasedFilter Deadline History Data Data Throughput Lifespan Durability Latency Availability Ownership LatencyBudget TransportPriority Ownership Strength Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Control over Latency/Throughput tradeoff Control over data queueing Control over data latency Control over data persistency Control over data priority Control over data sources hot-swap Reliability OpenSplice DDS provides programmatic Destination QoS-driven support for configuring the most Presentation Data Delivery Order important properties of data distribution! Control over data distribution reliability Control over data ordering Control over presentation © 2009, PrismTech. All Rights Reserved
    • OpenSplice DDS Delivering Performance, Openness, and Freedom Playing with QoS
    • History QoS History Depth = 5 1 1 1 2 1 3 1 4 1 5 DataReader 2 1 2 2 2 3 2 4 2 5 3 1 3 2 3 3 3 4 3 5 1 // Create Reader QoS Topic Proprietary Information - Distribution without Expressed Written Permission is Prohibited. dds::DataReaderQos drqos; drqos.set_keep_last(N); // Create Reader ‣ The History QoS Controls the number of dds::DataReader<MarketData> reader(topic, drqos); samples-per-instance that will be stored by the middleware on behalf of a Reader 2 ‣ Keep Last K. The History QoS can be set so // Create Reader QoS to always have the latest K samples dds::DataReaderQos drqos; drqos.set_keep_all(); ‣ Keep All. The History QoS can be set so // Create Reader keep all samples produced by the writer and dds::DataReader<MarketData> reader(topic, drqos); not yet taken, until resource limits are not reached © 2009, PrismTech. All Rights Reserved
    • Transport Priority QoS Make sure that important data gets the right priority Shared Single Copy per Node Shared Pack Across Topics/Applications Memory Optimal Unmarshaling Memory 1 // Create Writer QoS dds::DataWriterQos dwqos; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. dwqos.set_priority(MID_PRIO); OpenSplice DDS OpenSplice DDS // Create Writer Binding Pre-emptive Network Scheduler Binding dds::DataWriter<MarketData> writer(topic, dwqos); Networking Priority Scheduler Networking Data Urgency Traffic Pacing // Write Data writer.write(data); Network Channels // Change Priority Priority Bands dwqos.set_priority(HIGH_PRIO); Traffic Shaping writer.writer(data) © 2009, PrismTech. All Rights Reserved
    • Ownership QoS Availability of data producers can be controlled via two QoS Policies ‣ OWNERSHIP ‣ OWNERSHIP STRENGTH ‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote symbol: "MSFT" StockQuote StockQuote name: "Microsoft Corp." StockQuote symbol: "GOOG" symbol: "GOOG" exchange: "NASD""GOOG" symbol: name: "Google Inc." quote: 33.73 name: "Google Inc." name: "Google Inc." exchange: "NASD" exchange: "NASD" exchange: "NASD" StockQuote StockQuote StockQuote quote: 663.97 W1 quote: 663.97"AAPL" symbol: "AAPL" quote: 663.97 symbol: "AAPL" symbol: name: "Apple Inc." name: "Apple Inc." name: "Apple Inc." exchange: "NASD" exchange: "NASD" exchange: "NASD" R1 quote: 165.37 quote: 165.37 quote: 165.37 STRENGTH=3 StockQuote symbol: "AAPL" StockQuote Inc." name: "Apple symbol: "GOOG""NASD" exchange: name: quote: 165.37 "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W1’ R2 STRENGTH=2 StockQuote symbol: "AAPL" StockQuote Inc." name: "Apple symbol: "GOOG""NASD" exchange: name: quote: 165.37 "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W2’’ R3 STRENGTH=1 © 2009, PrismTech. All Rights Reserved
    • Ownership QoS Availability of data producers can be controlled via two QoS Policies ‣ OWNERSHIP ‣ OWNERSHIP STRENGTH ‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote symbol: "MSFT" StockQuote StockQuote name: "Microsoft Corp." StockQuote symbol: "GOOG" symbol: "GOOG" exchange: "NASD""GOOG" symbol: name: "Google Inc." quote: 33.73 name: "Google Inc." name: "Google Inc." exchange: "NASD" exchange: "NASD" exchange: "NASD" quote: 663.97 W1 quote: 663.97 quote: 663.97 StockQuote symbol: "AAPL" R1 STRENGTH=3 name: "Apple Inc." exchange: "NASD" quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W1’ R2 StockQuote symbol: "AAPL" name: "Apple Inc." exchange: "NASD" STRENGTH=2 quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." StockQuote "NASD" exchange: symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W2’’ StockQuote symbol: "AAPL" name: "Apple Inc." exchange: "NASD" quote: 165.37 R3 STRENGTH=1 © 2009, PrismTech. All Rights Reserved
    • Ownership QoS Availability of data producers can be controlled via two QoS Policies ‣ OWNERSHIP ‣ OWNERSHIP STRENGTH ‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote symbol: "MSFT" name: "Microsoft Corp." exchange: "NASD" quote: 33.73 W1 StockQuote symbol: "AAPL" R1 STRENGTH=3 name: "Apple Inc." exchange: "NASD" quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote symbol: "MSFT" name: "Microsoft Corp." exchange: "NASD" quote: 33.73 StockQuote W1’ R2 symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote symbol: "AAPL" name: "Apple Inc." exchange: "NASD" STRENGTH=2 quote: 165.37 StockQuote symbol: "MSFT" name: "Microsoft Corp." exchange: "NASD" StockQuote quote: 33.73 symbol: "GOOG" R3 name: "Google Inc." W2’’ exchange: "NASD"StockQuote quote: 663.97symbol: "AAPL" name: "Apple Inc." exchange: "NASD" quote: 165.37 STRENGTH=1 © 2009, PrismTech. All Rights Reserved
    • Ownership QoS Availability of data producers can be controlled via two QoS Policies ‣ OWNERSHIP ‣ OWNERSHIP STRENGTH ‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote R1 symbol: "AAPL" name: "Apple Inc." exchange: "NASD" quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote symbol: "AAPL" name: "Apple Inc." exchange: "NASD" R2 quote: 165.37 StockQuote StockQuote StockQuote StockQuote symbol: "MSFT" symbol: "MSFT" symbol: "MSFT" symbol: "GOOG" name: "Microsoft Corp." name: "Microsoft Corp." name: "Microsoft Corp." exchange: "NASD" exchange: "NASD" name: "Google Inc." exchange: "NASD" quote: 33.73 quote: 33.73 quote: 33.73 exchange: "NASD" quote: 663.97 StockQuote R3 symbol: "AAPL" W2’’ name: "Apple Inc." exchange: "NASD" quote: 165.37 STRENGTH=1 © 2009, PrismTech. All Rights Reserved
    • Ownership QoS Availability of data producers can be controlled via two QoS Policies ‣ OWNERSHIP ‣ OWNERSHIP STRENGTH ‣ Instances of exclusively owned Topics can be modified (are owned) by the higher strength writer ‣ Writer strength is used to coordinate replicated writers Proprietary Information - Distribution without Expressed Written Permission is Prohibited. StockQuote R1 symbol: "AAPL" name: "Apple Inc." exchange: "NASD" StockQuote quote: 165.37 symbol: "MSFT" name: "Microsoft Corp." exchange: "NASD" quote: 33.73 StockQuote symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote symbol: "GOOG" name: "Google Inc." exchange: "NASD" quote: 663.97 StockQuote R2 symbol: "MSFT" name: "Microsoft Corp." StockQuote exchange: "NASD" quote: 33.73 symbol: "AAPL" name: "Apple Inc." exchange: "NASD" quote: 165.37 StockQuote symbol: "GOOG" name: "Google Inc." StockQuote exchange: "NASD" symbol: "MSFT" quote: 663.97 name: "Microsoft Corp." exchange: "NASD" quote: 33.73 StockQuote R3 symbol: "AAPL" W2’’ name: "Apple Inc." exchange: "NASD" quote: 165.37 STRENGTH=1 © 2009, PrismTech. All Rights Reserved
    • OpenSplice DDS Delivering Performance, Openness, and Freedom Online References
    • http://www.opensplice.org/
    • SIMD Project http://code.google.com/p/simd-cxx/ © 2009, PrismTech. All Rights Reserved Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
    • OpenSplice DDS Delivering Performance, Openness, and Freedom Summing Up
    • Concluding Remarks ‣ DDS is the ideal fit for addressing several of the data distribution and management challenges faced by financial firms, such as: OpenSplice DDS Delivering Performance, Openness, and Freedom ‣ Market Data Distribution ‣ High Performance Caching Enterprise Ed. ‣ Real-Time Business Intelligence Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ OpenSplice DDS is an Open Source Professional Ed. implementation of the DDS standard widely deployed and delivering great performance Compact Ed. ‣ OpenSplice DDS is Commercially Supported Open Source Software Community Ed. Embrace Performance, Openness and Freedom. Join the OpenSplice DDS Community at www.opensplice.org © 2009, PrismTech. All Rights Reserved
    • Online Resources http://www.opensplice.com/ http://www.slideshare.net/angelo.corsaro emailto:opensplicedds@prismtech.com http://bit.ly/1Sreg http://twitter.com/acorsaro/ http://opensplice.blogspot.com http://www.youtube.com/OpenSpliceTube © 2009, PrismTech. All Rights Reserved