OpenSplice DDS: The Open Source Middleware Accelerating Wall Street

OpenSplice DDS Delivering Performance, Openness, and Freedom Angelo Corsaro, Ph.D. The Open Source Messaging Product Strategy & Marketing Manager OMG RTESS and DDS SIG Co-Chair angelo.corsaro@prismtech.com Accelerating Wall Street

Speaker Bio Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Angelo CORSARO, Ph.D. Product Strategy & Marketing Manager OMG RTESS and DDS SIG Co-Chair PrismTech F +33 1 69 01 53 54 | M +33 6 43 30 75 65 | T http://twitter.com/acorsaro Dr. Angelo Corsaro is the OpenSplice DDS Product Strategy & Marketing Manager at PrismTech, a market leader in the field of high performance middleware. His responsibilities include product strategy, competitive positioning, technology evolution, and technology evangelism. He is also responsible for strategic standardization at the Object Management Group (OMG), where is co-chair of the Data Distribution Service (DDS) Special Interest Group, and the Real-Time Embedded and Specialized Services Task Force. Angelo received a Ph.D. and a M.S. in Computer Science from the Washington University in St. Louis, and a Laurea Magna cum Laude in Computer Engineering from the University of Catania, Italy. © 2009, PrismTech. All Rights Reserved

Agenda ‣ The Big News ‣ What is OpenSplice DDS ‣ DDS vs AMQP Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Why OpenSplice DDS? ‣ Concluding Remarks © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom April 20th, 2009 OpenSplice|DDS Goes Live! :: http://www.opensplice.org

http://www.opensplice.org/

OpenSplice DDS Delivering Performance, Openness, and Freedom High Performance Pub/Sub

High Performance Pub/Sub The right data, at the right place, at the right time ‣ Fully distributed, Peer-to-Peer -- All the Time. Communication ‣ No Single Point of Failure Subscriber Publisher ‣ No Single Point of Bottleneck Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Multicast-enabled Brokers Subscriber ‣ High performance and highly Publisher scalable ‣ High availability Publisher ‣hot-swap Subscriber ‣hot-hot architecture © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Data-Centric Pub/Sub

Data-Centric Pub/Sub ‣ Data-Centric Features are built-in and don’t rely on an external DBMS ‣ Providing thus performance, scalability, and availability ‣ Distributed Relational Data Model Subscriber DBMS Publisher ‣ Local Queries B ‣ Continuous Queries / Content Based Proprietary Information - Distribution without Expressed Written Permission is Prohibited. m Subscriptions A F Subscriber ‣ Windows Publisher J D C ‣ Object/Relational Mapping K E ‣ Support for a subset of SQL-92 Publisher Subscriber Perfect Blend of Data-Centric and Real-Time Publish/Subscribe Technologies © 2009, PrismTech. All Rights Reserved

Data-Centric Pub/Sub Subscriber Publisher B Proprietary Information - Distribution without Expressed Written Permission is Prohibited. m A F Subscriber Publisher J D C K E Publisher Subscriber © 2009, PrismTech. All Rights Reserved

Data-Centric Pub/Sub Subscriber Publisher J B D D C K E A B B A Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Subscriber m D C Publisher A F J J A F D C K K A F E Subscriber Publisher J D D K E E © 2009, PrismTech. All Rights Reserved

Topics and Data-Centric Pub/Sub ‣ Topics. Unit of information exchanged between Topic Publisher and Subscribers. ‣ Data Types. Type associated to a Topic must be a structured type expressed in IDL struct TempSensor { int tID; Topic Type ‣ Topic Instances. Key values in a datatype float temp; Proprietary Information - Distribution without Expressed Written Permission is Prohibited. float humidity; uniquely identify a Topic Instance (like rows in }; #pragma keylist TempSensor tID table) TempSensor ‣ Content Awareness. SQL Expressions can be tID temp humidity used to do content-aware subscriptions, 1 21 62 Instances queries, joins, and correlate topic instances 2 27 78 3 25.5 72.3 y idit ity SELECT * FROM TempSensor t y idit mid Samples um hu um h WHERE t.temp > 25 h 62 63 65 p p tem p tem tem 21 22 tID 23 tID tID temp humidity tID 1 1 2 27 78 1 3 25.5 72.3 Time t1 t3 t2 © 2009, PrismTech. All Rights Reserved

Distributed Relational Information Modeling ‣ Topic Keys can be used to identify instances as well as relationships ‣ Relationships can be navigated by relying on a subset of SQL 92 ‣ One-to-many relationships can be captured using foreign keys ‣ Many-to-many relationships need to be modeled using a topics Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Keys can be represented by an arbitrary number of Topic fields TempSensor Room Floor tID: int rID: int fID: int temp: float width: float level: int humidity: float height: float open: int rID: int length: float fID: int © 2009, PrismTech. All Rights Reserved

Object/Relational Mapping TempSensor temp: float humidity: float getTemp() ‣ Arbitrary object * Room Floor getHumidity() 1 width: float level: int height: float open: int reconstructions 1 climate: int * length: float Conditioner 1 climate: int set_status() temp: float ‣ Automatic Relationships set_climate(); set_climate() humidity: float * Proprietary Information - Distribution without Expressed Written Permission is Prohibited. get_level() setTemp() setHumidity() Management setFanLevel() start() ‣ Inheritance stop() pause(): ‣ Local Operations TConditioner cID: int TTempSernsor ‣ Local/Distributed State temp: float tID: int humidity: float temp: float fan_level: int TRoom humidity: float status: int rID: int rID: int rID: int width: float height: float length: float TFloor fID: int fID: int level: int open: int © 2009, PrismTech. All Rights Reserved

Data Centric Publish/Subscribe Publishers Subscribers TempSensor TempSensor tID temp humidity tID temp humidity 18 60 18 60 1 1 22 75 2 21 71 3 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. struct TempSensor { int tID; TempSensor TempSensor float temp; tID temp humidity float humidity; tID temp humidity 22 75 2 }; 22 75 2 #pragma keylist TempSensor tID 21 71 3 SELECT * FROM TempSensor t WHERE s.temp > 20 s.t ID TempSensor == Fully Distributed Global TempSensor 1 tID temp humidity Data Space tID temp humidity 21 71 3 18 60 1 © 2009, PrismTech. All Rights Reserved

Data Centric Publish/Subscribe Publishers Subscribers TempSensor TempSensor tID temp humidity tID temp humidity 1 21 62 1 21 62 22 75 2 21 71 3 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. struct TempSensor { int tID; TempSensor TempSensor float temp; tID temp humidity float humidity; tID temp humidity 22 75 2 }; 1 21 62 #pragma keylist TempSensor tID 22 75 2 21 71 3 SELECT * FROM TempSensor t s.t WHERE s.temp > 20 ID TempSensor == Fully Distributed Global 1 tID temp humidity TempSensor Data Space 21 71 3 tID temp humidity 1 21 62 © 2009, PrismTech. All Rights Reserved

Processing Content & Structure Complex Event Processing... ‣ Local Queries OpenSplice DDS some of the most useful features found in Complex Event Processing platforms! ‣ Continuous Queries (Content-based Subscriptions) ‣ Topics Joins & Projections ‣ Events windows (via History QoS) Proprietary Information - Distribution without Expressed Written Permission is Prohibited. S1 tID temp humidity 3 21.5 72.3 tID temp humidity {iID = 1} 2 20 78 tID temp humidity 1 21 62 S2 P {temp > 21 | humidity > 75} S3 { temp <16 } © 2009, PrismTech. All Rights Reserved

Processing Content & Structure Complex Event Processing... ‣ Local Queries OpenSplice DDS some of the most useful features found in Complex Event Processing platforms! ‣ Continuous Queries (Content-based Subscriptions) ‣ Topics Joins & Projections ‣ Events windows (via History QoS) Proprietary Information - Distribution without Expressed Written Permission is Prohibited. tID temp humidity 1 21 62 S1 {iID = 1} tID temp humidity 2 20 78 3 21.5 72.3 S2 P {temp > 21 | humidity > 75} S3 { temp <16 } © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom QoS-Enabled Pub/Sub

QoS Model ‣ QoS-Policies are used to control relevant Type Matching properties of OpenSplice DDS entities, QoS matching such as: QoS QoS QoS QoS QoS ‣ Temporal Properties Topic ‣ Priority Name Publisher Subscriber Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ... DataReader reads DataWriter writes Type ... ‣ Durability ... ‣ Availability Type reads DataReader writes DataWriter ‣ ... Name ‣ Some QoS-Policies are matched based on Topic a Request vs. Offered Model thus QoS- QoS QoS QoS enforcement ‣ Publications and Subscriptions match only if the declared vs. requested QoS are compatible ‣ e.g., it is not possible to match a publisher which delivers data unreliably with a subscriber which requires reliability © 2009, PrismTech. All Rights Reserved

Sample QoS Policies QoS Policy Applicability RxO Modiﬁable QoS matching T, DR, DW Y N DURABILITY Data Availability QoS QoS QoS T, DW N N DURABILITY SERVICE Topic QoS QoS Name DataReader reads DataWriter writes Type Subscriber T, DW - Y LIFESPAN ... Publisher Type reads DataReader T, DR, DW N N writes HISTORY DataWriter Name Topic P, S Y N PRESENTATION Data Delivery T, DR, DW Y N QoS QoS QoS RELIABILITY QoS matching P, S N Y PARTITION Proprietary Information - Distribution without Expressed Written Permission is Prohibited. T, DR, DW Y N DESTINATION ORDER ‣ Rich set of QoS allow to configure T, DR, DW Y N OWNERSHIP DW - Y OWNERSHIP several different aspects of data STRENGTH availability, delivery and timeliness T, DR, DW Y Y DEADLINE Data Timeliness T, DR, DW Y Y LATENCY BUDGET ‣ QoS can be used to control and T, DW - Y TRANSPORT PRIORITY optimize network as well as DR - Y TIME BASED FILTER Resources computing resource T, DR, DW N N RESOURCE LIMITS DP, DR, DW N Y USER_DATA Conﬁguration T N Y TOPIC_DATA P, S N Y GROUP_DATA © 2009, PrismTech. All Rights Reserved

Mapping QoS Which properties does QoS controls? TimeBasedFilter Deadline History Data Data Throughput Lifespan Durability Latency Availability Ownership LatencyBudget TransportPriority Ownership Strength Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Control over Latency/Throughput tradeoff Control over data queueing Control over data latency Control over data persistency Control over data priority Control over data sources hot-swap Reliability OpenSplice DDS provides programmatic QoS-driven support for configuring the most Destination Presentation Data Delivery important properties of data distribution! Order Control over data distribution reliability Control over data ordering Control over presentation © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Zero-Conf Pub/Sub

Zero-Conf Pub/Sub Code ‣ Dynamic Discovery. Publisher, Bloat Subscriber, and Topics are dynamically discovered ‣ Separation of Concerns. Applications Proprietary Information - Distribution without Expressed Written Permission is Prohibited. are agnostic from networking details. Thus you can re-deploy the same business logic with different network, partitioning, configurations ‣ Tool Based. All configuration is tool assisted and happens outside of your application © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Standard-Based Pub/Sub

The OMG Data Distribution Service Data Distribution Service for Real-Time Systems Application ‣ Leap forward in Topic-based Publish/Subscribe Middleware state of the art Object/Relational Mapping ‣ Carefully specified to allow very high performance, Data Local Reconstruction Layer (DLRL) DDS v1.2 scalable, predictable and high-availability Content Proprietary Information - Distribution without Expressed Written Permission is Prohibited. implementations Ownership Durability Subscription ‣ Language Independent, OS and HW architecture Minimum Proﬁle independent Data Centric Publish/Subscribe (DCPS) DDSI v2.1 Fully Standardized Solution Real-Time Publish/Subscribe Protocol ‣ Data Distribution Service (DDS) for Real-Time DDS Interoperability Wire Protocol Systems v1.2 ‣ RTPS, DDS Interoperability Wire Protocol UDP/IP The OMG Data Distribution Service satisfies the most challenging information dissemination requirements across a wide set of application domains, ranging from multi-board systems to system-of-systems © 2009, PrismTech. All Rights Reserved

DDS!Recommendations is churning… The infrastructure evolution cycle – New -> Emerging -> Standard -> Commodity Increasingly Mandated/Recommended by Administrations – Middleware is emerging as OS declines ! …DDS is maturing… ‣ US Navy: Open Architecture – OMG focus – Wire spec Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ DISR/DISA: Net-centric Systems – Tools – Enterprise integration – Multiple products fielded ‣ EuroControl:– Deployed applications! Air Traffic Control Center Operational Interoperability ! …and adoption is on the rise – Navy – DISR ‣ QinetiQ: Recommending DDS for VSI – FCS/SoSCOE © 2009, PrismTech. All Rights Reserved

Scope of Standardization AMQP OMG DDS Application Application Object Relational Mapping Object/Relational Mapping Standard API Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Data Local Reconstruction Layer (DLRL) DDS v1.2 Content Ownership Durability Subscription API (?) Pub/Sub Standard API Minimum Proﬁle Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Real-Time Publish/Subscribe Protocol Advanced Message Queuing Protocol Wire Protocol Standard DDS Interoperability Wire Protocol Interoperability Wire Protocol UDP/IP (TCP, SCTP, UDP)/IP © 2009, PrismTech. All Rights Reserved

Architectural Style AMQP OMG DDS Subscriber Subscriber Publisher Publisher Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Broker Broker Broker Subscriber Subscriber Broker Publisher Publisher Broker Broker Publisher Publisher Subscriber Subscriber © 2009, PrismTech. All Rights Reserved

Key Differences in Summary AMQP DDS ‣ Brokered Communication ‣ Peer-to-Peer Communication Architecture Architecture ‣ Wire Protocol Standard ‣ Wire Protocol + API Standard Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Interoperability across AMQP ‣ Interoperability across DDS implementations implementations ‣ Application Portability across DDS Implementation ‣ Header Based Routing ‣ Content-Based Filtering/Quering ‣ Designed bottom-up for maximum ‣ Not geared for High-Performance performance and determinism © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Best Performance

Fast and Furious The Fastest Open Source Ultra-Low Stay Ahead of Market Data Rates Latency Messaging Middleware ‣ Millions of Market Data Updates per Second ‣ < 60 micro-seconds latency on 1GBps Ethernet ‣ Negligible inter-core latency © 2009, PrismTech. All Rights Reserved

Stable and Reliable Advanced Traffic Spikes- Stability, Reliability, and Absorption Technology Predictability, Under all Conditions ‣ Stable under the most extreme traffic spikes ‣ Three configurable level of Spike Absorption ‣ Guaranteed Reliability © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Best Architecture

Networking Architecture Architecture Shared Shared Single Copy per Node ‣ Network-channels Pack Across Topics/Applications Memory Memory ‣ Priority bands Optimal Unmarshaling ‣ Network-partitions ‣ Multicast Groups ‣ Traffic-shaping Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Burst/Throughput OpenSplice DDS OpenSplice DDS Binding Binding Scalability and Efficiency Pre-emptive Network Scheduer ‣ Single shared library for applications & services Priority Scheduler Networking Networking Data Urgency Trafﬁc Pacing ‣ Ring-fenced shared memory segment ‣ Data urgency driven network-packing Network Channels Priority Bands Determinism & Safety ‣ Trafﬁc Shaping Preemptive network-scheduler ‣ Data importance based network-channel selection ‣ Partition based multicast-group selection ‣ Managed critical network-resource © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Best Technology Ecosystem

Advanced Features ‣ Relational Data Modeling ‣ Object/Relational Mapping Application MDE ‣ Event Processing Tuner Object/Relational Mapping ‣Queries PowerTools Object-Oriented Pub/Sub (DLRL) DDS v1.2 ‣Continuous Queries Content Ownership Persistence Subscription DBMS ‣Join/Projections Minimum Proﬁle Connectors ‣ High Performance Caching Real-Time Pub/Sub(DCPS) DDSI v2.1 Spike Absorber ‣ Persistency SOAP Real-Time Interoperable Wire Secure Networking Networking Protocol (DDSI) ‣ Security Connectors Networking Technology ‣ Development Tools UDP/IP © 2009, PrismTech. All Rights Reserved

High Performance Caching ‣ Fully Distributed Caching Technology OpenSplice Cache ‣ Supports SQL Queries on local Key Value OpenSplice Cache OpenSplice Cache Cache Key Value Disk Key Value Key Value ‣ Support for Continuous SQL Queries Key Value Key Value Key Value Key Value Disk Disk Key Value Key Value on the whole distributed Cache Key Value Key Value ‣ High-Performance, ultra-low-latency OpenSplice DDS communication engine based on OpenSplice Cache OpenSplice Cache OpenSplice DDS Key Value Key Value Key Value Disk ‣ Fully exploits Data Locality and Key Value Key Value Disk Key Value Key Value Key Value Prefetching © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Best of Open Source

OpenSplice DDS is Open Source OpenSplice DDS ‣ Product reorganized into Editions ‣ Community Edition Delivering Performance, Openness, and Freedom ‣ Compact Edition ‣ Professional Edition Enterprise Ed. ‣ Enterprise Edition Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Product Editions provide a a growing set of Professional Ed. functionality to address the needs of increasingly more sophisticated users Compact Ed. ‣ The Community Edition is Open Source Community Ed. ‣ Compact, Enterprise and Professional Edition are available only through Commercial Subscriptions © 2009, PrismTech. All Rights Reserved

A Feature Rich Community Edition OpenSplice DDS AMQP Delivering Performance, Openness, and Freedom Application Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Application Object/Relational Mapping Data Local Reconstruction Layer (DLRL) DDS v1.2 Content Ownership Durability Subscription API (?) Minimum Proﬁle Data Centric Publish/Subscribe (DCPS) DDSI v2.1 AMQP v0.10 Real-Time Publish/Subscribe Protocol Advanced Message Queuing Protocol DDS Interoperability Wire Protocol Interoperability Wire Protocol UDP/IP (TCP, SCTP, UDP)/IP © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Best Way of Spending Your Money

Financial Advantages OpenSplice DDS subscriptions provides key advantages: License vs. Subscription Community Edition License Subscription ‣ Free like free speech, and 100 Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Free like free beer! 75 50 Commercial Editions 25 ‣ Cash/Flow Friendly Y1 0 Y2 Y3 Y4 ‣ Lower Total Cost of Ownership (TCO) Y5 ‣ Flexible model to tune needs vs. subscription © 2009, PrismTech. All Rights Reserved

OpenSplice DDS Delivering Performance, Openness, and Freedom Hottest Technology

DDS Adoption Defense ‣ Combat Management Systems ‣ Tactical Systems ‣ Communication Systems ‣ Radar Processing Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Submarine Systems ‣ Future Combat Systems SCADA/Utilities ‣ Industrial Automation ‣ Power Grids (Smart Grids) ‣ Complex Telemetry (e.g. NASA Rocket Launch System) © 2009, PrismTech. All Rights Reserved

DDS Adoption Transportation ‣ Air Traffic Management/Control ‣ Metropolitan Traffic Management ‣ Underground Metropolitan Transportation Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Financial Services ‣ Automated Trading ‣ Market Data Platforms ‣ Compliance Systems ‣ Risk Management © 2009, PrismTech. All Rights Reserved

Concluding Remarks ‣ OpenSplice DDS is the ideal fit for addressing OpenSplice DDS several of the data distribution and management challenges faced by financial firms, such as: Delivering Performance, Openness, and Freedom ‣ Market Data Distribution ‣ High Performance Caching Enterprise Ed. ‣ Real-Time Business Intelligence Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ OpenSplice DDS has great performance on Professional Ed. standard COTS HW and its architecture is designed for exploiting at best high-end multi- Compact Ed. cores such as the Intel XEON 5500 ‣ OpenSplice DDS is Commercially Supported Open Community Ed. Source Software Embrace Performance, Openness and Freedom. Join the OpenSplice DDS Community at www.opensplice.org © 2009, PrismTech. All Rights Reserved

Online Resources http://twitter.com/acorsaro/ http://www.opensplice.com/ emailto:opensplicedds@prismtech.com Proprietary Information - Distribution without Expressed Written Permission is Prohibited. http://bit.ly/1Sreg http://opensplice.blogspot.com http://www.dds-forum.org http://portals.omg.org/dds http://www.youtube.com/OpenSpliceTube © 2009, PrismTech. All Rights Reserved

Upcoming Webcasts on April 28 An Introduction to OpenSplice DDS Open Source OpenSplice | DDS Webcast Series Proprietary Information - Distribution without Expressed Written Permission is Prohibited. April 28, 2009 Why Should I Attend? • Learn everything about the OpenSplice DDS Forge • Discover the Roadmap and Plans for OpenSplice DDS • Understand how you might contribute to incubator projects © 2009, PrismTech. All Rights Reserved

OpenSplice DDS: The Open Source Middleware Accelerating Wall Street

0 comments

Notes on slide 1

1 Favorite