Continuent Tungsten: Proxies on Steroids for HA and Performance!


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Continuent Tungsten: Proxies on Steroids for HA and Performance!

  1. 1. Continuent Tungsten: Proxies on Steroids for HA and Performance! Technology Preview Robert Hodges, CTO of Continuent April 15, 2008 © Continuent 4/14/08
  2. 2. What We Will Cover / Defining the Scale-Out Problem / A Hard Look at MySQL Replication for Scale-Out / Can We Do Better? / Replication Reloaded: Tungsten Scale-Out Stack / How to Get Software and Get Started Today 2 © Continuent
  3. 3. A Word from Our Sponsor: Continuent / Company • Leading provider of database availability solutions for open source databases / Products • uni/cluster – multi-master database clustering that replicates data across multiple databases and load balances reads • Works with off-the-shelf database servers • Uses “database virtualization” to provide a seamless client interface / Value • Low-cost open source business critical solutions • Highly available data • Raise performance and hardware utilization through load balancing • Remove chance of data loss 3 © Continuent
  4. 4. Defining the Scale-Out Problem 4 4 © Continuent © Continuent Confidential
  5. 5. Requirement: Availability and Performance / Data Availability • Bring up alternate database after local failure • Bring up alternate database after site loss • Keep both sites on-line for immediate failover • Maintain/upgrade/migrate without disruption • Restart/restore following data loss or outage due to human error • Ensure commits to multiple databases and/or sites • Always accept transactions no matter what • Stay available in grid environments / Performance • Scale reads transparently • Scale writes transparently • Spread transaction load across multiple active sites • Rapid capacity scaling up/down in grid environments / Nirvana: increased data availability and performance scalability by adding commodity hardware 5 © Continuent
  6. 6. The Dream - Flexible Scale-Out App Scalability App Db App Db Db Availability 6 © Continuent
  7. 7. A Hard Look at MySQL Replication 7 7 © Continuent © Continuent Confidential
  8. 8. MySQL Replication for Scale-Out / MySQL database replication has set the high bar for horizontal scale-out on large websites for many years / Replication is “baked-in” to MySQL • Initial set-up takes only a few minutes • Handy SQL commands to configure masters and slaves / Flexibly copies all or some databases / Offers a choice of statement or row replication • Row replication only in MySQL 5.0 and previous releases / Works across WANs, hence can be used to link multiple sites / In short, it’s a key reason why MySQL is so popular 8 © Continuent
  9. 9. …But Is It Really That Easy? / Replication is hard -- no single approach solves all problems / Master is the weak point in the MySQL design • Replacing a crashed master when there are multiple slaves • Sync between binlog and database • Loss of committed transactions / Latency due to single threaded update on slaves / Administration is harder than it looks • Parameter settings: innodb_flush_log_at_trx_commit=2 • Provisioning, resynchronizing, monitoring • Automating failover / Read scaling requires application changes / Not a write scaling solution 9 © Continuent
  10. 10. A Hard Problem -- Master Election after a Crash Master Slave Where was I?? New Slave Slave Master Where was I?? 10 © Continuent
  11. 11. Customer Feedback from Our Clustering Work / Use cases for data base clustering driven by HA/scaling/both / Analysis of US/Europe sales shows following patterns: • Almost 100% of customers are looking for HA solution • Scalability was a requirement for ~30% / What Customers Need • Transparent, easy-to-deploy HA • Protection from data loss/corruption • Higher database performance with fewer application changes 11 © Continuent
  12. 12. MySQL vs. Customer Requirements / MySQL replication is excellent for implementing read scaling on custom applications • For the do-it-yourself set, it’s the best thing out there / BUT--It does not suit everyone: / Users who need easy-to-deploy availability / Users who cannot afford to lose data / Users who cannot afford intrusive application changes / Users who are not MySQL experts 12 © Continuent
  13. 13. Can We Do Better? New Technologies for Scale-Out 13 13 © Continuent © Continuent Confidential
  14. 14. Continuent Database Clustering History / Continuent implemented m/cluster for MySQL -- the first successful commercial cluster based on group communications / Our current uni/cluster product implements clusters using middleware and off-the-shelf databases / We have supported MySQL and PostgreSQL for years / Strong experience in commercial/open source solutions • Sequoia -- Multi-master state-machine clustering • Myosotis -- Fast native-to-JDBC proxies • Hedera -- Generic group communications adapters • Bristlecone -- Testing tools for scale-out architectures 14 © Continuent
  15. 15. What Have We Learned? / Replication works for building clusters - It’s the only approach that really scales / Database virtualization using middleware is essential for encapsulating location of copies / Group communications are hard to configure but highly effective for controlling clusters / Biggest challenges are simple management and simple application semantics / WAN support is the killer application of database clustering / There is no one-size-fits-all solution 15 © Continuent
  16. 16. What Are the Up-And-Coming Technologies? / CAP-aware databases (Consistent, Available, Partition-Tolerant) • Now appearing in non-SQL databases near you: Amazon Dynamo and SimpleDB / Pluggable replication -- Different replication mechanisms for different problems • GORDA research project has pioneered approach / Partitioning -- Dividing data and automatically routing queries • MySQL community is driving solutions / Open source databases -- This stuff is going to be big some day! 16 © Continuent
  17. 17. Replication Reloaded: The Tungsten Scale-Out Stack 17 17 © Continuent © Continuent Confidential
  18. 18. Our Goal Highly reliable, transparent, easy to deploy solutions for data availability and performance scalability that work with unaltered, stock RDBMS and economical, off-the-shelf hardware Tungsten: An architecture for scale-out that meets the goal 18 © Continuent
  19. 19. Tungsten Overview / Data Services integrate multiple copies of data within and across sites into a single highly available, performant database / Database virtualization with controlled data latency and consistency / Database-neutral replication designed for high availability / Cluster-aware management framework backed by excellent graphical tools / “Stack” organization of technology to allow substitution and extensions 19 © Continuent
  20. 20. An Open, Database-Neutral Scale-Out Stack Application Stacks PHP, Perl, Java, Ruby, etc. Perl, Tungsten Connector Native Wire Protocol Proxying; Connection semantics Proxying; Sequoia Tungsten Replicator Synchrononous Asynchronous Multi-Master Replication Master/Slave Replication Tungsten Manager Cluster-aware service management for LAN/WAN Bristlecone Hedera Scale-Out Perf Testing Group Comm Adapters DBMS Servers Group Communications MySQL, PostreSQL, Oracle MySQL, PostreSQL, JGroups, Appia, Spread Appia, 20 © Continuent
  21. 21. Why Are Stacks So Important? / No single solution can solve all data availability and scaling use cases / Stack approach allows you to reuse components on different scale-out problems / The stack is an extensible framework Caching Tungsten Connector (Myosotis) (Myosotis) Tungsten Sequoia Replicator Partioning Tungsten Manager Certification- Based Bristlecone Hedera Replication 21 © Continuent
  22. 22. Virtualized SQL Connection Semantics Application Stacks PHP, Perl, Java, Ruby, etc. Perl, Updates All Updates Own Updates Eventually Visible Visible Visible Node Node Node Node Tungsten Data Service 22 © Continuent
  23. 23. High-Availability Replication Master-master replication Master-slave replication / No master failover headaches / Wider range of supported SQL / Automatic load balancing / Faster performance / Works over WAN Node Node Node Node Node Node Sequoia Tungsten Replicator 23 © Continuent
  24. 24. Managing within and across Sites Site 2 Administrative Client Host Host Host JMX Event Bridge Group Communication Service Service Service Service Manager Manager Manager Manager Connector Connector Replicator Replicator Node Node Connector Connector Host Host Host Host Site 1 24 © Continuent
  25. 25. GUI Tools Demo 25 © Continuent
  26. 26. Now -- Multi-Master Data Services / Ensure constant availability and avoid master failure scenarios / Focus: Scalable and easily manageable multi-master replication • High quality management tools • Automatic load balancing of read requests • 3+ master nodes • Seamless connections from native clients and Java • Support for latest MySQL versions and table types • Solaris and Linux platforms 26 © Continuent
  27. 27. Now -- uni/cluster 2008 LAMP Stack Clients Clients Client Applications (PHP, Perl, C, Ruby, etc.) Java Clients uni/cluster Driver uni/cluster connector uni/cluster JDBC Driver Controller Controller Controller Controller JDBC Driver JDBC Driver JDBC Driver JDBC Driver DB DB DB DB Server Server Server Server 27 © Continuent
  28. 28. Phase 1 - Master/Slave Data Services / Constant availability with high transparency and update performance / Focus: Seamless upgrade from existing standalone database • One-step software installation • Rapid data service configuration • Lightweight, multi-platform management tools • No application code changes / High availability replication • Automatic and manual promotion of slave to master • Bounded latency between slave and master • Transparent proxying with read distribution to idle slaves 28 © Continuent
  29. 29. Phase 1 - Master/Slave Data Service Client Client Connectors Tungsten Mamagement Tools Failover, Connection Semantics Master DB Slave DB Slave DB Server Server Server Replication Replication Replication Node Mgr Node Mgr Node Mgr Service Management 29 © Continuent
  30. 30. Phase 2 - Cross-Site Data Services / Ensure availability and scaling across sites / Focus: Cross-site clustering with efficient failover • Replication of data to secondary site databases • Minimal, understandable application changes • Accept transactions across active systems on multiple sites / WAN clustering features • Tools and admin model for cross-site management • Handle WAN quality of service (latency, transient failures) • SQL partitioning / Emphasis on WAN solutions that work without corner cases or data inconsistencies 30 © Continuent
  31. 31. Phase 2 - Cross-Site Data Services DNS Load Balancing Read-Only Read-Only Read/Write Read/Write Applications, Applications, Applications Applications Reports Reports Connectors Data Service Connectors Slave DB Master DB Slave DB Server Server Server Site 1 Site 2 31 © Continuent
  32. 32. How to Get Software and Get Started Today 32 32 © Continuent © Continuent Confidential
  33. 33. Technology Roadmap / Uni/cluster 2007.1 is available today! / Uni/cluster 2008 available in late Q2 • GUI administrative tools • Tungsten management framework • Multiple controllers / Tungsten scale-out stack projects will be published soon / Tungsten master/slave services will be available early Q4 33 © Continuent
  34. 34. Getting Software / For access to commercial and open source products, visit us at the following web addresses / Visit us at the Continuent booth in the exhibit hall / Watch for more information on Tungsten very soon! 34 © Continuent
  35. 35. For More on Scale-Out Performance… / Come hear about scale-out testing using Bristlecone Portable Scale-Out Benchmarks for MySQL Wednesday 10:50-11:50am Ballroom D 35 © Continuent
  36. 36. Questions? Thank you! 36 36 © Continuent © Continuent Confidential
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.