How an Enterprise Data Fabric (EDF) can improve resiliency and performance

2,090 views

Published on

From the Gaming Scalability event, June 2009 in London (http://gamingscalability.org).

Mike Stolz outlines three relevant use cases for the GemFire Data Caching Technologies that clearly demonstrate a reduction in the Total Cost of Ownership, increased reliability, increased scalability, increased throughput and a reduction in overall system latency. The use cases include

* HA, DR and BCP is a pure caching play
* How EDF can improve your Affiliate Banner Advertising capability
* Advantages of global data consistency and regional edge caching

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,090
On SlideShare
0
From Embeds
0
Number of Embeds
58
Actions
Shares
0
Downloads
59
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

How an Enterprise Data Fabric (EDF) can improve resiliency and performance

  1. 1. How an Enterprise Data Fabric (EDF) can improve resiliency and performance while reducing TCO Presented by Mike Stolz, VP Architecture & Martin Hand, Business Development Director for EMEA GemStone Systems Inc. Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  2. 2. Gaming Scalability - Agenda What is an EDF 3 Relevant Use Cases  EDF Resiliency Story  Affiliate Banner Management  Global Data Distribution (Near Caching) 2 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  3. 3. What is an Enterprise Data Fabric (EDF) Subscriber Update Data Publisher Subscriber Subscriber • Data Management that pools memory, CPU, and network across many servers. • Very high performance – memory speed not disk speed • Secure, elastic, resilient data storage • Horizontal Partitioning of the Domain Data with Co-location of related data • Dynamic scalability and elasticity for both data and behavior • Automatic striping and replication • Continuous availability – “RAID for the Enterprise” • Replication to Disaster Recover Sites • Global Data Distribution – Consistent Global Views 3 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  4. 4. Enterprise Data Fabric Resiliency Story High Availability, Disaster Recovery and Business Continuity are a pure distributed caching play Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  5. 5. Use-Case: In Play Pricing and Risk Management Business requirement/drivers  Global view, speed, stability, scalability, etc. • Multiple geographical sites (i.e., Hong Kong, London, India) must see all betting positions • Sub-second position calc and global distribution • Betting volumes doubling year on year • Number of users doubling year on year • Burst rates of over 1000 bets per second • Consideration for broadcast latency • High availability/fast failover and recovery – NO TOLERANCE FOR DOWN TIME 5 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  6. 6. Use Case: In Play Pricing and Risk Management Traditional Solution Product Set:  Database Instances In Every Region  Local Area Messaging product to move data around among processes  WAN-based Messaging Product for global distribution  N+1 Clustering for High Availability  Storage Level Replication for Disaster Recovery 6 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  7. 7. Classic High Availability Node Failure: External Data Sources.  In the event of failure of a node in the N+1 cluster, the • Detect Failure +1 node is pressed into • Boot +1 Node service • Mount Shared Storage • Start up DB Typically takes about 15 minutes Applications • Start up to • recover from a single nodeFetch data from DB failure • Re-create Objects • Reconnect exchanges • Reconnect clients Clients • Recovery complete Shared Storage 7 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  8. 8. Classic Disaster Recovery Catastrophic Failure:  In the event of failure of • Detect Failure the entire cluster or Data Center, DR is • Swing Storage invoked. • Boot DR Nodes • Mount Storage • Start up DB In most installations it takes Start up Applications • between • Fetch data from DB 1Primary 4 hours to recover• Re-create objects and Replicated Storage from DR • a data center outageReconnect externals Clients External • Reconnect clients Data Sources • Recovery Complete 8 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  9. 9. Enterprise Data Fabric H/A and DR Catastrophic Node Failure:Failure:  In the event of failure of the entire cluster failure of a single Data is striped or Data Center, DR is node. invoked. across the distributed cache 1On the order Nothing Storage second plus of 1 second for• Detect Failure Shared recovery any external connectivity Transactional • Reconnect Externals Clients to recover from a full node center outage from a single data failure Hot #1 • Reconnect clients Recovery Complete Replication Hot #2 • Recovery Complete Clients External Data Sources/ Exchanges, etc. Shared Nothing Storage 9 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  10. 10. EDF Simplifies your Architecture What did we replace?  Database Instances In Every Region • Still need 1 instance for archival purposes  Separate Local Area Messaging Product  WAN Distribution Product  Traditional N+1 Clustering for H/A  Storage Level Replication for DR This architecture simplification represents a very significant savings in terms of TCO. 10 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  11. 11. Improve your Affiliate Banner Advertising How an Enterprise Data Fabric can help you scale Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  12. 12. Affiliate Banner System Bet.com 12 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  13. 13. Affiliate Banner Delivery  Banners are delivered to the affiliate’s web pages each time they are viewed by a member of the public.  The HTML code on the affiliate’s web site for the banner does not have a static image address. Instead it calls to your image server with parameters that indicate which banner should be delivered.  The parameters identify the Product (i.e. to scale this The issue here is the ability “Sport”), Classification (i.e. “Cricket”), Language (i.e. “German”) and Dimensions of the to hundreds of thousands of hits. requested Banner.  Once the banner has been displayed, you need to track every time a user actually clicks on the banner so that you know how effective your banner advertising actually is in terms of traffic generation 13 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  14. 14. Affiliate Banner Delivery Internet VB.Net VB.Net VB.Net Web Server Web Server The traditional Web Server way: A tangled web of connections from the web tier into the Geo, Banner and Click tracking databases which Geo DB become the Banner DB Clicks DB bottleneck. 14 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  15. 15. Affiliate Banner Delivery Internet With an EDF: Geo and Banner data are pre-loaded into cache, The net result is that the time spent DB’s Offloading the VB.Net VB.Net VB.Net and reducing generating the appropriate banner is Web Server Web Server Web Server latency, thereby enabling scale to minimized, thus enhancing the end-user hundreds of EDF thousands of hits. experience, and the likelihoodClicks are written to that he will the cache, which in click-through. turn does a lazy Geo DB write-behind to the Banner DB Clicks DB click tracking database. 15 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  16. 16. EDF for Global Data Distribution Advantages of global data consistency and regional edge caching Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  17. 17. Global Data Distribution – Edge Caching How do you deliver best end-user experience to gamers globally? Answer: Global Edge Caching. 17 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  18. 18. Global Data Distribution – Edge Caching Centralization of Odds Processing Engines Management:  Odds can be managed centrally or if you prefer, can be managed at multiple geographical sites (nearest point to the action). US Users US RDBMS Gateway EU Users AP Gateway Gateway EU AP Users 18 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  19. 19. Global Data Distribution – Edge Caching Centralization of Odds Processing Engines Management:  Odds data is distributed around the globe and cached locally in the regional edge caches. US Users US RDBMS Gateway EU Users AP Gateway Gateway EU AP Users 19 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  20. 20. Global Data Distribution – Edge Caching Centralization of Odds Processing Engines Management:  So that users only need to make a local hop to read the most current data. US Users US RDBMS Gateway EU Users AP Gateway Gateway EU AP Users 20 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  21. 21. Global Data Distribution – Edge Caching Centralization of Transaction Processing Engines Management:  Transaction requests are sent to the centralized resource manager. US Users US RDBMS Gateway EU Users AP Gateway Gateway EU AP Users 21 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  22. 22. Global Data Distribution – Edge Caching Centralization of Transaction Processing Engines Management:  The transaction is processed locally by the resource manager. US Users US RDBMS Gateway EU Users AP Gateway Gateway EU AP Users 22 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  23. 23. Global Data Distribution – Edge Caching Centralization of Transaction Processing Engines Management:  Response is sent to the originating user’s node. US Users US RDBMS Gateway EU Users AP Gateway Gateway EU AP Users 23 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  24. 24. Global Data Distribution – Edge Caching Centralization of Transaction Processing Engines Management:  And changes to Odds or other data are automatically propagated around the world. US Users US RDBMS Gateway EU Users AP Gateway Gateway EU AP Users 24 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  25. 25. Global Data Distribution – Edge Caching Centralization of Transaction Processing Engines Management:  And notification is published to all interested users. US Users US Global Edge Caching ensures the most efficient end-user experience without RDBMS Gateway replicating the whole application EU Users AP Gateway Gateway EU environment. AP Users 25 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  26. 26. Summary What did the EDF achieve?  Fast end-user experience  Secure, Reliable Persistence  Keep up with fast moving environment  Updates published from location of action  Rapid, Event Based Odds Calculation  Distribution of Updates Globally  Consistent Global Views  High Availability  Disaster Recovery  Reduced complexity and TCO 26 Copyright © 2009, GemStone Systems Inc. All Rights Reserved.
  27. 27. Q&A Copyright © 2009, GemStone Systems Inc. All Rights Reserved.

×