Strategies for scaling MySQLMatthew Aslett, 451 Research                      Doron Levari & Paul Campaniello, ScaleBase  ...
Agenda  1. 451 Research – The MySQL SPRAIN  2. ScaleBase - Scale-Out your MySQL Database  3. Customer Case Studies  4. Q &...
Company Overview  One company with 3 operating              200+ staff   divisions                                 1,30...
   Matthew Aslett•   Research Manager, Data Management and Analytics   matthew.aslett@451research.com   www.twitter.com...
In a nutshell MySQL became the default database of choice for Web applications Success breeds complexity:  • Greater sca...
Previously on ScaleBase/451 webinars… The traditional relational database has been stretched beyond its    normal capacit...
Focus on MySQL MySQL filled a gap in the market:  • Low cost and freely available with source code  • Better suited to em...
Early adoption drivers: 2008 survey                             © 2013 by The 451 Group. All rights reserved
Rapid increase in adoption: 2012 database survey90.080.070.060.050.040.030.020.010.0 0.0                                  ...
Scalability, and other challenges As usage of MySQL has grown, so has the usage of applications that  depend on MySQL Th...
The road to SPRAIN: approaches to scaling MySQL                                     Example:                             ...
The road to SPRAIN: approaches to scaling MySQL                                        Example:                          ...
The road to SPRAIN: approaches to scaling MySQL                                                   Example:               ...
The road to SPRAIN: approaches to scaling MySQL                                                   Example:               ...
The road to SPRAIN: approaches to scaling MySQL                                                    Example:              ...
The road to SPRAIN: approaches to scaling MySQL                                          Example:                        ...
The road to SPRAIN: approaches to scaling MySQL                                                      Example:            ...
The road to SPRAIN: approaches to scaling MySQL                                                   Example:               ...
Alternatives NewSQL – retaining support for ACID transactions and SQL  • New databases  • Advanced storage engines, parti...
Comparing NewSQL approaches New databases  • Pros: Designed specifically to support distributed architecture  • Cons: May...
Spotlight on ScaleBase Creates a shared nothing architecture from standard databases Elastic load balancing for MySQL (o...
Conclusion Scalability isn’t simply a matter of a theoretical number of nodes It’s also about the ability to deliver pre...
Thank you.                                         matthew.aslett@451research.com                                         ...
How to Scale Out your MySQL Database
Who We Are Presenters:                                     Paul Campaniello,                                              ...
ScaleBase Mission“MySQL can cost-effectivelyscale to support an infinitenumber of application users,with NO disruption to ...
ScaleBase: Scale-Out MySQL - Fix the SPRAIN Don’t throw out the baby with the bath water! • Keep your MySQL - keep your In...
Single-Box Scalability Improvements in MySQL 5.6Partitioning Improvements                                         Replicat...
Two Approaches to Scale-Out MySQL
Scale Out                              Read                              WriteR/W Splitting                Replication    ...
Read/Write Splitting: Application ExperienceCurrent: App sees master plus multiple slaves                                 ...
Read/Write Splitting: Application ExperienceCurrent: App sees master plus multiple slaves                  Problems:      ...
Data Distribution: Application ExperienceCurrent: App is customized to support multiple shardsWith ScaleBase: App sees ONE...
Data Distribution: Application ExperienceCurrent: App is customized to support multiple shards               Problems:    ...
Typical ScaleBase Data Traffic Manager Deployment                                     Database A   Replica AApplication Se...
ScaleBase Features and Benefits     Feature                            Benefit     Automatic data distribution        Scal...
Scale…without SPRAINScale Your Database With Your Business                  160000                  140000                ...
ScaleBase Delivers Scalability                                   Real-time DB                                  Traffic Vis...
Detailed Scale-Out Case Studies     Mozilla            AppDynamics             Large Chip Co        Solar Edge     • New P...
Economic ImpactCustomer Experience                           Internal Development      – Faster, more responsive apps,    ...
ScaleBase Deployment     Environments                     Path to Scale-Out:       – Public Cloud                    1.   ...
Summary• MySQL is vast and adoption is growing• MySQL scalability is a significant problem (SPRAIN)     – App explosion (w...
Questions (please enter directly into the GTW side panel)matt.aslett@451research.com         paul.campaniello@scalebase.co...
Thank You44
Upcoming SlideShare
Loading in …5
×

ScaleBase Webinar: Strategies for scaling MySQL

2,292 views
2,073 views

Published on

Matt Aslett of 451 Research joins ScaleBase to discuss: scaling-out your MySQL DB, new high availability strategies, centrally managing a distributed MySQL environment.

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,292
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
28
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

ScaleBase Webinar: Strategies for scaling MySQL

  1. 1. Strategies for scaling MySQLMatthew Aslett, 451 Research Doron Levari & Paul Campaniello, ScaleBase © 2013 by The 451 Group. All rights reserved
  2. 2. Agenda 1. 451 Research – The MySQL SPRAIN 2. ScaleBase - Scale-Out your MySQL Database 3. Customer Case Studies 4. Q & A (please type questions directly into the GoToWebinar side panel) © 2013 by The 451 Group. All rights reserved
  3. 3. Company Overview  One company with 3 operating  200+ staff divisions  1,300+ client organizations:  Syndicated research, advisory, enterprises, vendors, service professional services, datacenter providers, and investment firms certification, and events  Organic and growth through  Global focus acquisition © 2013 by The 451 Group. All rights reserved
  4. 4.  Matthew Aslett• Research Manager, Data Management and Analytics matthew.aslett@451research.com www.twitter.com/maslett Responsible for data management and analytics research agenda Focus on operational and analytic databases, including NoSQL, NewSQL, and Hadoop With 451 Research since 2007 © 2013 by The 451 Group. All rights reserved
  5. 5. In a nutshell MySQL became the default database of choice for Web applications Success breeds complexity: • Greater scalability demands • Predictable performance problems New requirements: • Proliferation of cloud • Geo-distributed data Multiple strategies for dealing with scalability issues • Each with their plusses and minuses The emergence of new vendors aimed at masking the complexity © 2013 by The 451 Group. All rights reserved
  6. 6. Previously on ScaleBase/451 webinars… The traditional relational database has been stretched beyond its normal capacity by the needs of high-volume, highly distributed or highly complex applications. There are workarounds – such as DIY sharding – but manual, homegrown efforts can result in database administrators being stretched beyond their normal capacity in terms of managing complexity. Scalability Performance Relaxed consistency Increased willingness to look towards Agility emerging alternatives Intricacy Necessity © 2013 by The 451 Group. All rights reserved
  7. 7. Focus on MySQL MySQL filled a gap in the market: • Low cost and freely available with source code • Better suited to emerging application workloads MySQL benefits • Lightweight architecture • Fast read capabilities • Ease of use MySQL primarily adopted for: • in-house-developed applications • single-function applications • customer-facing Web applications • development-and-test environments © 2013 by The 451 Group. All rights reserved
  8. 8. Early adoption drivers: 2008 survey © 2013 by The 451 Group. All rights reserved
  9. 9. Rapid increase in adoption: 2012 database survey90.080.070.060.050.040.030.020.010.0 0.0 Survey conducted: Jan/Feb 2012 Sample: 205 © 2013 by The 451 Group. All rights reserved
  10. 10. Scalability, and other challenges As usage of MySQL has grown, so has the usage of applications that depend on MySQL This has highlighted a number of challenges • Scalability of master-slave architecture • Performance • Predictability of performance at scale • Manageability of increasing database/app sprawl External factors driving greater complexity: • Distributed computing architectures • Proliferation of cloud, and elasticity requirements • Geo-distributed application requirements © 2013 by The 451 Group. All rights reserved
  11. 11. The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users  Started with single MySQL database database © 2013 by The 451 Group. All rights reserved
  12. 12. The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption database © 2013 by The 451 Group. All rights reserved
  13. 13. The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption  Expanded its MySQL footprint every two years master slave slave © 2013 by The 451 Group. All rights reserved
  14. 14. The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption  Expanded its MySQL footprint every two years  Larger master master • Pros: Add more CPUs, disk and memory to cope with increased data • Cons: Doesn’t necessarily help slave slave with performance, temporary fix © 2013 by The 451 Group. All rights reserved
  15. 15. The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption  Expanded its MySQL footprint every two years  More slaves master • Pros: Good for scaling our read queries • Cons: Asynchronous replication can lead to write slave lag slave slave slave slave  By 2009 had 16 slaves and was beginning to hit the wall © 2013 by The 451 Group. All rights reserved
  16. 16. The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption  Expanded its MySQL footprint every two years  Considered sharding:database database database • Pros: increases scalability to serve high-traffic websites • Cons: Horizontal (across masters), required application re-write • Cons: Vertical (across the slaves), considered a temporary fix © 2013 by The 451 Group. All rights reserved
  17. 17. The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption  Expanded its MySQL footprint every two years  Read/write splitting master • Pros: Better utilization of available resources • Cons: Asynchronous replication can lead to slave lag slave slave slave slave © 2013 by The 451 Group. All rights reserved
  18. 18. The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption memcached  Expanded its MySQL footprint every two years  Memcached master • Pros: Improves read performance for frequently accessed data • Cons: Not persistent, doesn’t solve write performance issues slave slave • Looked instead for alternatives © 2013 by The 451 Group. All rights reserved
  19. 19. Alternatives NewSQL – retaining support for ACID transactions and SQL • New databases • Advanced storage engines, particularly for MySQL • Advanced clustering/shard management approaches © 2013 by The 451 Group. All rights reserved
  20. 20. Comparing NewSQL approaches New databases • Pros: Designed specifically to support distributed architecture • Cons: May lack compatibility with existing applications Advanced storage engines, particularly for MySQL • Pros: Retain familiarity with with MySQL skills, tools • Cons: Re-architecting from the inside out. Advanced clustering/shard management approaches • Pros: Retain application compatibility while adding scalability • Cons: An extra layer of complexity? Issues to consider: • Does it require a forklift move of your entire application ecosystem • Can you continue to leverage your existing MySQL skill set? • Is there a risk for your data, e.g. memory reliability? © 2013 by The 451 Group. All rights reserved
  21. 21. Spotlight on ScaleBase Creates a shared nothing architecture from standard databases Elastic load balancing for MySQL (other databases on the roadmap) Scale Out via read/write splitting or automatic data distribution Data Traffic Manager serves as a proxy between the apps and DB Provides a single point for administering the shared nothing cluster(for performance, HA, change management) And the ability to add scalability without the need to migrate to anew database architecture or make any changes to existing apps. © 2013 by The 451 Group. All rights reserved
  22. 22. Conclusion Scalability isn’t simply a matter of a theoretical number of nodes It’s also about the ability to deliver predictable performance And to do so without adding management complexity Distributed computing, proliferation of cloud, and geo-distributed applications are adding to the complexity MySQL has never been under so much strain The combination of advanced clustering/load balancing and management technologies provides a potential solution © 2013 by The 451 Group. All rights reserved
  23. 23. Thank you. matthew.aslett@451research.com @maslettPhoto credit: swisscan on Flickr http://bit.ly/HCZCRQ © 2013 by The 451 Group. All rights reserved
  24. 24. How to Scale Out your MySQL Database
  25. 25. Who We Are Presenters: Paul Campaniello, VP of Global Marketing 25 year technology veteran with marketing experience at Mendix, Lumigent, Savantis and Precise. Doron Levari, Founder & CTO A technologist and long-time veteran of the database industry. Prior to founding ScaleBase, Doron was CEO to Aluna.25
  26. 26. ScaleBase Mission“MySQL can cost-effectivelyscale to support an infinitenumber of application users,with NO disruption to the existing infrastructure”26
  27. 27. ScaleBase: Scale-Out MySQL - Fix the SPRAIN Don’t throw out the baby with the bath water! • Keep your MySQL - keep your InnoDB • Ecosystem compatibility, preserve skills • 100% application compatibility • Smoother migration, no down-time, no forklift • Your data is safe! • No “in-memory” magic • No “in-memory” size limit27
  28. 28. Single-Box Scalability Improvements in MySQL 5.6Partitioning Improvements Replication Improvements – Explicit Partition Selection: – Optimizations to Row-Based SELECT * FROM employees Replication PARTITION (p0, p2); – Multi-Threaded Slaves – Import / Export for Partitioned Tables: Bring a new data set into a partitioned – Improvements to Data Integrity table, or export a partition to manage it as a regular table ALTER TABLE e – Crash-Safe Slaves EXCHANGE PARTITION p0 WITH – Replication Checksums TABLE e2; SCALABILITY issues remain due to the limitations of a single box: To ensure ACID, you still face limitations with: - Memory management - Thread management - Semaphores - Locking - Recovery tasks No new functionality for sharing workloads across multiple boxes28 http://dev.mysql.co/tech-resources/articles/whats-new-in-mysql-5.6.html
  29. 29. Two Approaches to Scale-Out MySQL
  30. 30. Scale Out Read WriteR/W Splitting Replication DataDistribution30
  31. 31. Read/Write Splitting: Application ExperienceCurrent: App sees master plus multiple slaves Read ReplicasWith ScaleBase: App sees ONE database31
  32. 32. Read/Write Splitting: Application ExperienceCurrent: App sees master plus multiple slaves Problems: • Writing code, maintaining code • Maintaining DB ops and IPs in the app • Master DB is too occupied • Slaves are not balanced Read Replicas • Replication could break • Stale data may be readWith ScaleBase: App sees ONE database ScaleBase Benefits: • Reads/Writes throughput increases • Better resource utilization/load balancing • Improved data consistency/transaction isolation with: • Replication state lag awareness • Read stickiness after writes • Handling of edge cases32
  33. 33. Data Distribution: Application ExperienceCurrent: App is customized to support multiple shardsWith ScaleBase: App sees ONE database… …and does not require any customization33
  34. 34. Data Distribution: Application ExperienceCurrent: App is customized to support multiple shards Problems: • Major app rewrite, maintaining code • Maintaining DB ops and IPs in the app • Administration/3rd party tools are blind • Non-optimized data distribution policyWith ScaleBase: App sees ONE database… ScaleBase Benefits: • Unlimited scalability while improving performance • Real time elasticity • Verified data consistency • Real time monitoring, analysis • Carefully analyze distribution policy …and does not require any customization34
  35. 35. Typical ScaleBase Data Traffic Manager Deployment Database A Replica AApplication Servers Database B Replica B BI Unlimited Scale Database C Replica C Fault TolerantManagement Architecture Centrally Manage Complex DB Database D Replica D Environment 35
  36. 36. ScaleBase Features and Benefits Feature Benefit Automatic data distribution Scale data-, read-, write- intensive applications  Expert System Analysis Optimization of data distribution policy  Parallel query execution Best option for cross-db queries & maintenance commands Support of sophisticated cross-db queries, including ORDER  Query result aggregation BY, GROUP BY, LIMIT, Aggregate functions… Flexibility: no need to over-provision  Online data redistribution No downtime Read/Write splitting Optimal for scaling of read-intensive applications  Replication lag-based routing Improves data consistency and isolation  Read stickiness after writes Ensure consistent and isolated database operation 100% compatible MySQL proxy Applications unmodified Standard MySQL tools and interfaces MySQL databases unmodified Data is safe within MySQL InnoDB/MyISAM/etc. Data consistency verifier Data consistency validation Real-time monitoring and alerts Simplify management, reduce TCO36
  37. 37. Scale…without SPRAINScale Your Database With Your Business 160000 140000 120000 100000 Throughput 80000 84000 Throughput (TPM) Total DB Size (MB) 60000 60000 # Connections 48000 40000 36000 24000 2500 20000 2000 12000 1500 1500 6000 1000 0 500 500 1 2 4 6 8 10 14 Number of Databases37
  38. 38. ScaleBase Delivers Scalability Real-time DB Traffic Visibility Scale to Automated Unlimited Failover Users …all with NO App Rewrites38
  39. 39. Detailed Scale-Out Case Studies Mozilla AppDynamics Large Chip Co Solar Edge • New Product/ • Next gen APM • Scalability • Next Gen Next Gen App/ company • Multiple Apps Monitoring App AppStore • Scalability for the • Multiple growing • Massive Scale • Scalability Netflix users • Monitors real • Geo-clustering implementation • Availability time data from • MySQL DB thousands of distributed systems39
  40. 40. Economic ImpactCustomer Experience Internal Development – Faster, more responsive apps, – Developers focus on app, not even during peak loads writing complex sharding code – Always on: 24/7 – Eliminate development and – Improve customer acquisition ongoing maintenance costs (reduce churn) and brand Centralized ManagementScale – Troubleshoot problems and – Use commodity hardware vs. scale- investigate incidents in seconds up hardware with real-time monitoring – Cloud ready – deploys easily High AvailabilityNon-MySQL Options – Eliminate database downtime – Protect existing investment with no with automated fail-over ecosystem forklift – Improve business continuity – Leverage existing MySQL skill set 40
  41. 41. ScaleBase Deployment Environments Path to Scale-Out: – Public Cloud 1. Data Distribution Policy Analysis – AWS, Rackspace, any 2. Functional Test – Private cloud 3. Load Test – Hosted / on-premise 4. Production Migration Databases Supported (safe, online) – MySQL 5.1, 5.5, 5.6 (under certification) – AWS RDS MySQL 5.1, 5.5 – Maria DB 10.0 (under certification)41
  42. 42. Summary• MySQL is vast and adoption is growing• MySQL scalability is a significant problem (SPRAIN) – App explosion (web & mobile), Big Data transactions…• ScaleBase provides cost-effective Scale-Out solutions – Seamlessly scale to an infinite number of users – Improve DB visibility with real-time monitoring – Enhance business continuity with automated failover – Expert analysis and simple deployment – No application rewrites – Leverage existing MySQL ecosystem/skills42
  43. 43. Questions (please enter directly into the GTW side panel)matt.aslett@451research.com paul.campaniello@scalebase.com doron.levari@scalebase.com @maslett @scalebase @451research www.ScaleBase.com www.451research.com 617.630.280043
  44. 44. Thank You44

×