• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
ScaleBase Webinar: Strategies for scaling MySQL
 

ScaleBase Webinar: Strategies for scaling MySQL

on

  • 1,137 views

Matt Aslett of 451 Research joins ScaleBase to discuss: scaling-out your MySQL DB, new high availability strategies, centrally managing a distributed MySQL environment.

Matt Aslett of 451 Research joins ScaleBase to discuss: scaling-out your MySQL DB, new high availability strategies, centrally managing a distributed MySQL environment.

Statistics

Views

Total Views
1,137
Views on SlideShare
1,137
Embed Views
0

Actions

Likes
1
Downloads
17
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    ScaleBase Webinar: Strategies for scaling MySQL ScaleBase Webinar: Strategies for scaling MySQL Presentation Transcript

    • Strategies for scaling MySQLMatthew Aslett, 451 Research Doron Levari & Paul Campaniello, ScaleBase © 2013 by The 451 Group. All rights reserved
    • Agenda 1. 451 Research – The MySQL SPRAIN 2. ScaleBase - Scale-Out your MySQL Database 3. Customer Case Studies 4. Q & A (please type questions directly into the GoToWebinar side panel) © 2013 by The 451 Group. All rights reserved
    • Company Overview  One company with 3 operating  200+ staff divisions  1,300+ client organizations:  Syndicated research, advisory, enterprises, vendors, service professional services, datacenter providers, and investment firms certification, and events  Organic and growth through  Global focus acquisition © 2013 by The 451 Group. All rights reserved
    •  Matthew Aslett• Research Manager, Data Management and Analytics matthew.aslett@451research.com www.twitter.com/maslett Responsible for data management and analytics research agenda Focus on operational and analytic databases, including NoSQL, NewSQL, and Hadoop With 451 Research since 2007 © 2013 by The 451 Group. All rights reserved
    • In a nutshell MySQL became the default database of choice for Web applications Success breeds complexity: • Greater scalability demands • Predictable performance problems New requirements: • Proliferation of cloud • Geo-distributed data Multiple strategies for dealing with scalability issues • Each with their plusses and minuses The emergence of new vendors aimed at masking the complexity © 2013 by The 451 Group. All rights reserved
    • Previously on ScaleBase/451 webinars… The traditional relational database has been stretched beyond its normal capacity by the needs of high-volume, highly distributed or highly complex applications. There are workarounds – such as DIY sharding – but manual, homegrown efforts can result in database administrators being stretched beyond their normal capacity in terms of managing complexity. Scalability Performance Relaxed consistency Increased willingness to look towards Agility emerging alternatives Intricacy Necessity © 2013 by The 451 Group. All rights reserved
    • Focus on MySQL MySQL filled a gap in the market: • Low cost and freely available with source code • Better suited to emerging application workloads MySQL benefits • Lightweight architecture • Fast read capabilities • Ease of use MySQL primarily adopted for: • in-house-developed applications • single-function applications • customer-facing Web applications • development-and-test environments © 2013 by The 451 Group. All rights reserved
    • Early adoption drivers: 2008 survey © 2013 by The 451 Group. All rights reserved
    • Rapid increase in adoption: 2012 database survey90.080.070.060.050.040.030.020.010.0 0.0 Survey conducted: Jan/Feb 2012 Sample: 205 © 2013 by The 451 Group. All rights reserved
    • Scalability, and other challenges As usage of MySQL has grown, so has the usage of applications that depend on MySQL This has highlighted a number of challenges • Scalability of master-slave architecture • Performance • Predictability of performance at scale • Manageability of increasing database/app sprawl External factors driving greater complexity: • Distributed computing architectures • Proliferation of cloud, and elasticity requirements • Geo-distributed application requirements © 2013 by The 451 Group. All rights reserved
    • The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users  Started with single MySQL database database © 2013 by The 451 Group. All rights reserved
    • The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption database © 2013 by The 451 Group. All rights reserved
    • The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption  Expanded its MySQL footprint every two years master slave slave © 2013 by The 451 Group. All rights reserved
    • The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption  Expanded its MySQL footprint every two years  Larger master master • Pros: Add more CPUs, disk and memory to cope with increased data • Cons: Doesn’t necessarily help slave slave with performance, temporary fix © 2013 by The 451 Group. All rights reserved
    • The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption  Expanded its MySQL footprint every two years  More slaves master • Pros: Good for scaling our read queries • Cons: Asynchronous replication can lead to write slave lag slave slave slave slave  By 2009 had 16 slaves and was beginning to hit the wall © 2013 by The 451 Group. All rights reserved
    • The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption  Expanded its MySQL footprint every two years  Considered sharding:database database database • Pros: increases scalability to serve high-traffic websites • Cons: Horizontal (across masters), required application re-write • Cons: Vertical (across the slaves), considered a temporary fix © 2013 by The 451 Group. All rights reserved
    • The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption  Expanded its MySQL footprint every two years  Read/write splitting master • Pros: Better utilization of available resources • Cons: Asynchronous replication can lead to slave lag slave slave slave slave © 2013 by The 451 Group. All rights reserved
    • The road to SPRAIN: approaches to scaling MySQL  Example:  Photo sharing website users users users  Started with single MySQL database  Growth through acquisition and customer adoption memcached  Expanded its MySQL footprint every two years  Memcached master • Pros: Improves read performance for frequently accessed data • Cons: Not persistent, doesn’t solve write performance issues slave slave • Looked instead for alternatives © 2013 by The 451 Group. All rights reserved
    • Alternatives NewSQL – retaining support for ACID transactions and SQL • New databases • Advanced storage engines, particularly for MySQL • Advanced clustering/shard management approaches © 2013 by The 451 Group. All rights reserved
    • Comparing NewSQL approaches New databases • Pros: Designed specifically to support distributed architecture • Cons: May lack compatibility with existing applications Advanced storage engines, particularly for MySQL • Pros: Retain familiarity with with MySQL skills, tools • Cons: Re-architecting from the inside out. Advanced clustering/shard management approaches • Pros: Retain application compatibility while adding scalability • Cons: An extra layer of complexity? Issues to consider: • Does it require a forklift move of your entire application ecosystem • Can you continue to leverage your existing MySQL skill set? • Is there a risk for your data, e.g. memory reliability? © 2013 by The 451 Group. All rights reserved
    • Spotlight on ScaleBase Creates a shared nothing architecture from standard databases Elastic load balancing for MySQL (other databases on the roadmap) Scale Out via read/write splitting or automatic data distribution Data Traffic Manager serves as a proxy between the apps and DB Provides a single point for administering the shared nothing cluster(for performance, HA, change management) And the ability to add scalability without the need to migrate to anew database architecture or make any changes to existing apps. © 2013 by The 451 Group. All rights reserved
    • Conclusion Scalability isn’t simply a matter of a theoretical number of nodes It’s also about the ability to deliver predictable performance And to do so without adding management complexity Distributed computing, proliferation of cloud, and geo-distributed applications are adding to the complexity MySQL has never been under so much strain The combination of advanced clustering/load balancing and management technologies provides a potential solution © 2013 by The 451 Group. All rights reserved
    • Thank you. matthew.aslett@451research.com @maslettPhoto credit: swisscan on Flickr http://bit.ly/HCZCRQ © 2013 by The 451 Group. All rights reserved
    • How to Scale Out your MySQL Database
    • Who We Are Presenters: Paul Campaniello, VP of Global Marketing 25 year technology veteran with marketing experience at Mendix, Lumigent, Savantis and Precise. Doron Levari, Founder & CTO A technologist and long-time veteran of the database industry. Prior to founding ScaleBase, Doron was CEO to Aluna.25
    • ScaleBase Mission“MySQL can cost-effectivelyscale to support an infinitenumber of application users,with NO disruption to the existing infrastructure”26
    • ScaleBase: Scale-Out MySQL - Fix the SPRAIN Don’t throw out the baby with the bath water! • Keep your MySQL - keep your InnoDB • Ecosystem compatibility, preserve skills • 100% application compatibility • Smoother migration, no down-time, no forklift • Your data is safe! • No “in-memory” magic • No “in-memory” size limit27
    • Single-Box Scalability Improvements in MySQL 5.6Partitioning Improvements Replication Improvements – Explicit Partition Selection: – Optimizations to Row-Based SELECT * FROM employees Replication PARTITION (p0, p2); – Multi-Threaded Slaves – Import / Export for Partitioned Tables: Bring a new data set into a partitioned – Improvements to Data Integrity table, or export a partition to manage it as a regular table ALTER TABLE e – Crash-Safe Slaves EXCHANGE PARTITION p0 WITH – Replication Checksums TABLE e2; SCALABILITY issues remain due to the limitations of a single box: To ensure ACID, you still face limitations with: - Memory management - Thread management - Semaphores - Locking - Recovery tasks No new functionality for sharing workloads across multiple boxes28 http://dev.mysql.co/tech-resources/articles/whats-new-in-mysql-5.6.html
    • Two Approaches to Scale-Out MySQL
    • Scale Out Read WriteR/W Splitting Replication DataDistribution30
    • Read/Write Splitting: Application ExperienceCurrent: App sees master plus multiple slaves Read ReplicasWith ScaleBase: App sees ONE database31
    • Read/Write Splitting: Application ExperienceCurrent: App sees master plus multiple slaves Problems: • Writing code, maintaining code • Maintaining DB ops and IPs in the app • Master DB is too occupied • Slaves are not balanced Read Replicas • Replication could break • Stale data may be readWith ScaleBase: App sees ONE database ScaleBase Benefits: • Reads/Writes throughput increases • Better resource utilization/load balancing • Improved data consistency/transaction isolation with: • Replication state lag awareness • Read stickiness after writes • Handling of edge cases32
    • Data Distribution: Application ExperienceCurrent: App is customized to support multiple shardsWith ScaleBase: App sees ONE database… …and does not require any customization33
    • Data Distribution: Application ExperienceCurrent: App is customized to support multiple shards Problems: • Major app rewrite, maintaining code • Maintaining DB ops and IPs in the app • Administration/3rd party tools are blind • Non-optimized data distribution policyWith ScaleBase: App sees ONE database… ScaleBase Benefits: • Unlimited scalability while improving performance • Real time elasticity • Verified data consistency • Real time monitoring, analysis • Carefully analyze distribution policy …and does not require any customization34
    • Typical ScaleBase Data Traffic Manager Deployment Database A Replica AApplication Servers Database B Replica B BI Unlimited Scale Database C Replica C Fault TolerantManagement Architecture Centrally Manage Complex DB Database D Replica D Environment 35
    • ScaleBase Features and Benefits Feature Benefit Automatic data distribution Scale data-, read-, write- intensive applications  Expert System Analysis Optimization of data distribution policy  Parallel query execution Best option for cross-db queries & maintenance commands Support of sophisticated cross-db queries, including ORDER  Query result aggregation BY, GROUP BY, LIMIT, Aggregate functions… Flexibility: no need to over-provision  Online data redistribution No downtime Read/Write splitting Optimal for scaling of read-intensive applications  Replication lag-based routing Improves data consistency and isolation  Read stickiness after writes Ensure consistent and isolated database operation 100% compatible MySQL proxy Applications unmodified Standard MySQL tools and interfaces MySQL databases unmodified Data is safe within MySQL InnoDB/MyISAM/etc. Data consistency verifier Data consistency validation Real-time monitoring and alerts Simplify management, reduce TCO36
    • Scale…without SPRAINScale Your Database With Your Business 160000 140000 120000 100000 Throughput 80000 84000 Throughput (TPM) Total DB Size (MB) 60000 60000 # Connections 48000 40000 36000 24000 2500 20000 2000 12000 1500 1500 6000 1000 0 500 500 1 2 4 6 8 10 14 Number of Databases37
    • ScaleBase Delivers Scalability Real-time DB Traffic Visibility Scale to Automated Unlimited Failover Users …all with NO App Rewrites38
    • Detailed Scale-Out Case Studies Mozilla AppDynamics Large Chip Co Solar Edge • New Product/ • Next gen APM • Scalability • Next Gen Next Gen App/ company • Multiple Apps Monitoring App AppStore • Scalability for the • Multiple growing • Massive Scale • Scalability Netflix users • Monitors real • Geo-clustering implementation • Availability time data from • MySQL DB thousands of distributed systems39
    • Economic ImpactCustomer Experience Internal Development – Faster, more responsive apps, – Developers focus on app, not even during peak loads writing complex sharding code – Always on: 24/7 – Eliminate development and – Improve customer acquisition ongoing maintenance costs (reduce churn) and brand Centralized ManagementScale – Troubleshoot problems and – Use commodity hardware vs. scale- investigate incidents in seconds up hardware with real-time monitoring – Cloud ready – deploys easily High AvailabilityNon-MySQL Options – Eliminate database downtime – Protect existing investment with no with automated fail-over ecosystem forklift – Improve business continuity – Leverage existing MySQL skill set 40
    • ScaleBase Deployment Environments Path to Scale-Out: – Public Cloud 1. Data Distribution Policy Analysis – AWS, Rackspace, any 2. Functional Test – Private cloud 3. Load Test – Hosted / on-premise 4. Production Migration Databases Supported (safe, online) – MySQL 5.1, 5.5, 5.6 (under certification) – AWS RDS MySQL 5.1, 5.5 – Maria DB 10.0 (under certification)41
    • Summary• MySQL is vast and adoption is growing• MySQL scalability is a significant problem (SPRAIN) – App explosion (web & mobile), Big Data transactions…• ScaleBase provides cost-effective Scale-Out solutions – Seamlessly scale to an infinite number of users – Improve DB visibility with real-time monitoring – Enhance business continuity with automated failover – Expert analysis and simple deployment – No application rewrites – Leverage existing MySQL ecosystem/skills42
    • Questions (please enter directly into the GTW side panel)matt.aslett@451research.com paul.campaniello@scalebase.com doron.levari@scalebase.com @maslett @scalebase @451research www.ScaleBase.com www.451research.com 617.630.280043
    • Thank You44