OpenSplice Cache

1,616 views
1,522 views

Published on

This presentation introduces OpenSplice Cache, a thin layer over OpenSplice DDS that adds traditional <key> caching functionalities.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,616
On SlideShare
0
From Embeds
0
Number of Embeds
27
Actions
Shares
0
Downloads
82
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

OpenSplice Cache

  1. 1. OpenSplice DDS Delivering Performance, Openness, and Freedom The Design and Implementation Angelo Corsaro, Ph.D. Product Strategy Manager OMG RTESS and DDS SIG Co-Chair angelo.corsaro@prismtech.com of a Data Caching Extension for the OMG DDS
  2. 2. Why Caching? Agenda OpenSplice Cache Explained Standardization Concluding Remarks
  3. 3. Challenges Three-Tier Systems ‣ The number of clients growing exponentially due to increased adoption of e-* ‣ Amount of data that needs to Proprietary Information - Distribution without Expressed Written Permission is Prohibited. be accessed growing due increased digitalization ‣ Computing can be scaled-out adding cheap HW at the Logic- Tier, but data access does not scale-out! Relevance ‣ SaaS, ‣ Cloud Computing ‣ XTP ‣ WWW © 2009, PrismTech. All Rights Reserved
  4. 4. Challenges Sense and Respond Systems ‣ Relevant state is either stored in a DBMS, or replicated wherever needed ‣ The DBMS approach leads to inevitable bottlenecks ‣ The fully replicated data, if not dealt properly, might induce excessive resource usage, or worse waste of resources Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Relevance ‣ C2, C4I Systems ‣ ATC, ATM ‣ Vetronics ‣ DCS © 2009, PrismTech. All Rights Reserved
  5. 5. Challenges High Performance Computing ‣ Huge amount of data (exceeding the capacity of a single machine) needed for carrying on a specific computation ‣ Certain Data-points could be re- Proprietary Information - Distribution without Expressed Written Permission is Prohibited. computed to save memory, but this takes time, e.g., re-computing a financial instrument ‣ Accessing data all at once, or from a centralized data server does not scale, or is too expensive Relevance ‣ Financial Services ‣ Simulations © 2009, PrismTech. All Rights Reserved
  6. 6. The Key Idea & Why it Works Idea Main Memory ‣ Cache data close to where it is needed and keep it there for as long as it is needed, or as long as you can... Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Why it works ‣ Locality of References. Programs tends to L2 reuse data they have used recently ‣ Temporal Locality. Recently accessed data L1 items are likely to be accessed again in the near future ‣ Spatial Locality. Items whose addresses are close to each other tend to be referenced close together in time © 2009, PrismTech. All Rights Reserved
  7. 7. Cache Webster Definition (1976) ‣ a safe place for hiding or storing things. Webster Definition (2008) ‣ (a): a hiding place especially for concealing and preserving provisions or Proprietary Information - Distribution without Expressed Written Permission is Prohibited. implements (b): a secure place of storage. ‣ something hidden or stored in a cache. ‣ a computer memory with very short access time used for storage of frequently or recently used instructions or data —called also cache memory. Etymology (1797) ‣ French, from cacher to press, hide, from Vulgar Latin coacticare to press together, from Latin coactare. © 2009, PrismTech. All Rights Reserved
  8. 8. Why Caching? Agenda OpenSplice Cache Explained Standardization Concluding Remarks
  9. 9. OpenSplice Cache in Brief ‣ Fully Distributed Caching OpenSplice Cache Technology ‣ Key Value Supports SQL Queries on OpenSplice Cache Key Key Value Disk Value OpenSplice Cache local Cache Key Value Key Value Key Value ‣ Key Value Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Key Value Support for Continuous Key Key Disk Value Value Key Disk Value Key Value SQL Queries on the whole distributed Cache ‣ OpenSplice DDS High-Performance, ultra- low-latency OpenSplice Cache OpenSplice Cache communication engine Key Value based on OpenSplice DDS Key Value Key Value Disk ‣ Key Value Key Value Disk Fully exploits Data Locality Key Key Value Value Key Value © 2009, PrismTech. All Rights Reserved
  10. 10. OpenSplice Cache Architecture ‣ Fully exploits the power of OpenSplice Application MDE DDS Real-Time Pub/Sub for Tuner Caching API maintaining cache PowerTools OpenSplice Cache Proprietary Information - Distribution without Expressed Written Permission is Prohibited. coherence Content ‣ Exposes a simple DDS v1.2 Ownership Persistence Subscription DBMS API for storing and Minimum Profile Connectors retrieving items to/ Real-Time Pub/Sub(DCPS) from the cache DDSI v2.1 ‣ Can be used by SOAP Real-Time Interoperable Wire Secure Networking Networking Protocol (DDSI) applications in Connectors Networking Technology conjunction with UDP/IP OpenSplice DDS © 2009, PrismTech. All Rights Reserved
  11. 11. OpenSplice Cache Core API ‣ Simple and concise core API that keeps simple most commonly used operations ‣ Extended API providing ‣ Rendezvous ‣ Advanced Cache Configuration ‣ Cached Data QoS Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Cache Replacement Strategy Customization OpenSplice Cache Write Queries + write(key, value) Key Value Key Value + query(sql): view Read Disk Key Value + add_continuous_query(sql, Key Value listener) + read(key): value + take(key): value Notifications + add_miss_handler(handler) + add_insert_handler(handler) © 2009, PrismTech. All Rights Reserved
  12. 12. A Performance Driven Architecture © 2008, PrismTech. All Rights Reserved
  13. 13. Performances Driven Architecture ‣ Cache performances are summarized by the following equation: Average Data Access = HitTime + MissRate x MissPenalty Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Thus to optimize the Cache performances it is necessary to ‣ Reduce Hit Time ‣ Reduce Miss Rate ‣ Reduce Miss Penalty OpenSplice Cache has been designed to ensure that all these factors are optimized © 2009, PrismTech. All Rights Reserved
  14. 14. Hit Time Minimization ‣ The hit-time is the time taken to access data when it is in the cache. ‣ OpenSplice Cache has two different hit time, one local and one remote, depending on wether the data is locally available, or available somewhere on the network Local Hit-Time Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Accessing an object locally is a OpenSplice Cache matter of micro-seconds Remote Hit-Time ‣ OpenSplice Cache, relies on Read Key Value OpenSplice DDS to resolve remotely Key Value a hit. Disk + read(key): value Key Value ‣ As such, this hit time, is close to a + take(key): value Key Value message roundtrip on OpenSplice DDS, which depending on the size is measured in tens of microseconds © 2009, PrismTech. All Rights Reserved
  15. 15. Local and Remote Hits OpenSplice Cache OpenSplice Cache OpenSplice Cache Disk Local Hit read(‘a’): ‘alpha’ ‘a’ ‘alpha’ Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Disk Disk OpenSplice DDS OpenSplice Cache OpenSplice Cache ‘b’ ‘beta’ Disk Disk © 2009, PrismTech. All Rights Reserved
  16. 16. Local and Remote Hits OpenSplice Cache OpenSplice Cache OpenSplice Cache Disk Local Hit read(‘a’): ‘alpha’ ‘a’ ‘alpha’ Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Disk Remote Hit Disk read(‘b’): ‘beta’ OpenSplice DDS OpenSplice Cache OpenSplice Cache ‘b’ ‘beta’ Disk Disk © 2009, PrismTech. All Rights Reserved
  17. 17. Local and Remote Hits OpenSplice Cache OpenSplice Cache OpenSplice Cache Disk Local Hit read(‘a’): ‘alpha’ ‘a’ ‘alpha’ Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Disk Remote Hit Disk read(‘b’): ‘beta’ Miss read(‘c’): miss handler OpenSplice DDS OpenSplice Cache OpenSplice Cache ‘b’ ‘beta’ Disk Disk © 2009, PrismTech. All Rights Reserved
  18. 18. Miss Rate Minimization Cache misses can be classified in: ‣ Compulsory. The first access to a data item not in the cache. These are also called Cold Start Misses or First Reference Misses. ‣ Capacity. If the cache cannot contain all the data needed by an Proprietary Information - Distribution without Expressed Written Permission is Prohibited. application during its execution, capacity miss will occur because of blocks being discarded and later retrieved. ‣ Conflict. Depending on how the cache is organized internally, some data items might compete for resources, and lead to replacement OpenSplice Cache minimize architecturally all of these misses. © 2009, PrismTech. All Rights Reserved
  19. 19. Pre-Fetching & ‣ Compulsory Misses can be reduced OpenSplice Cache by trying to guess what an application will use next ‣ Temporal and Spatial correlation Key Value among the components of a Working Set Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Key Value Disk distributed system can help in Prefetche Key Value predicting the use d Data Key Value ‣ OpenSplice Cache, exploits unused space in a local cache instance to eagerly pre-load data items -- thus reducing compulsory misses ‣ An API is also provided to the application to influence and customize the pre-fetching © 2009, PrismTech. All Rights Reserved
  20. 20. Pre-Fetching OpenSplice Cache OpenSplice Cache OpenSplice Cache Disk write(‘s’): ‘sigma’ ‘a’ ‘alpha’ ‘s’ ‘sigma’ Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Disk Disk OpenSplice DDS OpenSplice Cache OpenSplice Cache ‘b’ ‘beta’ Disk Disk © 2009, PrismTech. All Rights Reserved
  21. 21. Pre-Fetching OpenSplice Cache OpenSplice Cache OpenSplice Cache Disk ‘s’ ‘sigma’ write(‘s’): ‘sigma’ ‘a’ ‘alpha’ ‘s’ ‘sigma’ Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Disk Disk ‘s’ ‘sigma’ OpenSplice DDS OpenSplice Cache OpenSplice Cache ‘b’ ‘beta’ Disk Disk ‘s’ ‘sigma’ ‘s’ ‘sigma’ © 2009, PrismTech. All Rights Reserved
  22. 22. Pre-Fetching OpenSplice Cache OpenSplice Cache OpenSplice Cache Disk ‘s’ ‘sigma’ write(‘s’): ‘sigma’ ‘a’ ‘alpha’ ‘s’ ‘sigma’ Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Disk Disk ‘s’ ‘sigma’ OpenSplice DDS OpenSplice Cache OpenSplice Cache ‘b’ ‘beta’ Disk Disk ‘s’ ‘sigma’ ‘s’ ‘sigma’ Local Hit read(‘s’): ‘sigma’ © 2009, PrismTech. All Rights Reserved
  23. 23. Remote Cache Hit & Capacity Miss ‣ We’ve seen that OpenSplice Cache OpenSplice Cache supports the concept of local and remote hits Read Key Value ‣ When a resource constrained Key Value Disk Proprietary Information - Distribution without Expressed Written Permission is Prohibited. cannot allocate the whole memory + read(key): value Key Value + take(key): value Key Value needed for its working set locally, it can still exploit caching done in more powerful nodes to improve its over all performance ‣ Thus, OpenSplice Cache through exploiting resources available, somewhere else, it is practically possible to limit, if not eliminate the capacity miss © 2009, PrismTech. All Rights Reserved
  24. 24. Cache Organization & Conflict Misses ‣ By default OpenSplice Cache behaves as a fully OpenSplice Cache associative Cache managed by an LRU replacement policy Key Value ‣ The default choice is motivated by the optimality of Key Value Disk LRU as a replacement strategy Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Key Value Key Value ‣ An API is provided to customize the replacement strategy so to fit a specific application need/ behavior ‣ This allows for OpenSplice Cache to always minimize the conflict misses, and ensure minimal cache miss rate -- thus best performance © 2009, PrismTech. All Rights Reserved
  25. 25. Putting it all Together OpenSplice Cache LRU Key Value OpenSplice Cache OpenSplice Cache Key Value Disk LRU Key Value Assoc. Key Value Key Value Key Value Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Working Set Key Value Key Value Disk Disk Key Value Key Value Prefetched Data Key Value Key Value OpenSplice DDS OpenSplice Cache OpenSplice Cache Custom Random Key Value Key Value Key Value Disk Key Value Key Value Disk Key Value Key Value Key Value © 2009, PrismTech. All Rights Reserved
  26. 26. Architectural Styles © 2008, PrismTech. All Rights Reserved
  27. 27. Symmetric ‣ All nodes have similar OpenSplice Cache resources ‣ Caches instances on all Key Value OpenSplice Cache nodes have similar sizes OpenSplice Cache Key Key Value Disk Value ‣ The resulting system is really Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Key Value Key Value Key Value peer-to-peer, and every node, Key Value Disk Key Key Value Disk Value Key Value can potentially serve remote Key Value Key Value hits OpenSplice DDS OpenSplice Cache OpenSplice Cache Key Value Key Value Key Value Disk Key Value Key Value Disk Key Value Key Value Key Value © 2009, PrismTech. All Rights Reserved
  28. 28. Asymmetric ‣ Some nodes have very Load-Balanced/Partitioned Super Caches limited resources ‣ As a consequence OpenSplice Cache OpenSplice Cache Cache instances have Key Value OpenSplice Cache different sized on Key Value Key Value OpenSplice Cache Proprietary Information - Distribution without Expressed Written Permission is Prohibited. Key Value Key Value various nodes Key Value Key Key Disk Value Value Key Key Value Disk Value ‣ The system, will rely on Key Value Disk Key Value Key Value Key Value Key Value Key Value Key Value a set of “super-caches” Key Value Key Key Value Value Disk to resolved remote-hits OpenSplice Cache Key Key Value Value ‣ Super-Caches can Key Value Key Value Key Value OpenSplice DDS Disk exploit partitioning as Key Value OpenSplice Cache well as load-balancing OpenSplice Cache to ensure scalability OpenSplice Cache Key Key Value Value Key Value Disk Key Value Key Value Key Value Disk Key Value Key Value Key Value Key Value Disk Key Value Key Value Key Value Key Value © 2009, PrismTech. All Rights Reserved
  29. 29. Why Caching? Agenda OpenSplice Cache Explained Standardization Concluding Remarks
  30. 30. Standardization ‣ No formal Caching Technology standard exist to date ‣ JCache is a JSR, but there not seem to be real interested toward converging over a standard Proprietary Information - Distribution without Expressed Written Permission is Prohibited. ‣ Caching functionalities extend DDS in a very useful way ‣ Possible Standardization venue: ‣ Standardize a Caching API that builds over DDS ‣ Extend the DDS QoS to be able to have “Cache Semantics” for Subscriber/Reader or Topics © 2009, PrismTech. All Rights Reserved
  31. 31. Why Caching? Agenda OpenSplice Cache Explained Standardization Concluding Remarks
  32. 32. Concluding Remarks ‣ OpenSplice Cache is an innovative caching technology whose architecture ensures maximum performance by construction ‣ It leverages OpenSplice DDS high performance messaging Proprietary Information - Distribution without Expressed Written Permission is Prohibited. middleware for implementing its distributed consistency protocol, and deliver ultra-low hit-times ‣ Architecturally supports equally well symmetric system deployments as well as highly asymmetric systems, facilitating the creation of load-balanced/partitioned super caches OpenSplice Cache provides best performance and architectural flexibility! © 2009, PrismTech. All Rights Reserved
  33. 33. Online Resources http://www.opensplice.com/ http://twitter.com/acorsaro/ emailto:openslicedds@prismtech.com Proprietary Information - Distribution without Expressed Written Permission is Prohibited. http://bit.ly/1Sreg http://dds4u.blogspot.com http://www.dds-forum.org http://www.youtube.com/OpenSpliceTube http://portals.omg.org/dds

×