ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!
Upcoming SlideShare
Loading in...5
×
 

ScaleBase Webinar: Scaling MySQL - Sharding Made Easy!

on

  • 715 views

Home-grown sharding is hard - REALLY HARD! ScaleBase scales-out MySQL, delivering all the benefits of MySQL sharding, with NONE of the sharding headaches. This webinar explains: MySQL scale-out ...

Home-grown sharding is hard - REALLY HARD! ScaleBase scales-out MySQL, delivering all the benefits of MySQL sharding, with NONE of the sharding headaches. This webinar explains: MySQL scale-out without embedding code and re-writing apps, Successful sharding on Amazon and private clouds, Single vs. multiple shards per server, Eliminating data silos, Creating a redundant, fault tolerant architecture with no single-point-of-failure, Re-balancing and splitting shards

Statistics

Views

Total Views
715
Views on SlideShare
715
Embed Views
0

Actions

Likes
1
Downloads
24
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

ScaleBase Webinar: Scaling MySQL - Sharding Made Easy! ScaleBase Webinar: Scaling MySQL - Sharding Made Easy! Presentation Transcript

  • Scaling MySQL – Sharding Made Easy
  • 2Agenda• Scalability Issues• MySQL 5.6• Why Do-It-Yourself (DIY) Sharding Sucks• ScaleBase Data Distribution:– Successful sharding on Amazon and private clouds– Single vs. multiple shards per server– Eliminating data silos– Creating a redundant, fault-tolerant architecture– Re-balancing and splitting shards• Q & A
  • 3Doron Levari, Founder & CTODoron Levari,Founder & CTOA technologist and long-time veteran of thedatabase industry. Priorto foundingScaleBase, Doron wasCEO to Aluna.
  • 4What We DoSimply and cost-effectively scaleMySQL to support an infinitenumber of users, transactions and datawith NO disruption to the existing infrastructure
  • Scalability Issues and MySQL 5.6
  • 6MySQL Scalability Challenges• Too many transactions• Too many users• Too much data• Too many writes• Capacity• Throughput• Performance inconsistencies
  • 7Improvements in MySQL 5.6 – Single BoxPartitioning Improvements– Explicit Partition Selection:SELECT * FROM employeesPARTITION (p0, p2);– Import / Export for Partitioned Tables:Bring a new data set into a partitionedtable, or export a partition to manage itas a regular table ALTER TABLE eEXCHANGE PARTITION p0 WITHTABLE e2;http://dev.mysql.co/tech-resources/articles/whats-new-in-mysql-5.6.htmlReplication Improvements– Optimizations to Row-BasedReplication– Multi-Threaded Slaves– Improvements to Data Integrity– Crash-Safe Slaves– Replication ChecksumsSCALABILITY issues remain due to the limitations of a single box:To ensure ACID, you still face limitations with:- Memory management - Thread management- Semaphores - Locking- Recovery tasksNo new functionality for sharing workloads across multiple boxes
  • 8What are my Options1. More/Bigger Hardware?– Temporary fix…you will need new hardware again– More memory…helps mostly with “reads,” but not with “writes”– Every write operation is at least 4 write operations in database, plusmultiple activities in the database engine memory2. Application re-architecture?– Steer workload away from the database– Example: introduce a caching layer– Force application re-writes; new test & QA cycles3. Do it Yourself Sharding?4. Migrate to new database architecture– Other RDBMS/NewSQL / NoSQL?– Force application re-writes; new test & QA cycles– ACID/Durability Issues
  • 9Scale Out your Existing MySQL• Keep your MySQL - keep your InnoDB• Ecosystem compatibility, preserve skills• 100% application compatibility• Smoother migration, no down-time, no forklift• Your data is safe• No “in-memory” magic• No “in-memory” size limitDon’t throw out the baby with the bath water!
  • Why Do It Yourself Sharding “Sucks”
  • 11What is Sharding?Wikipedia - Shard (database architecture) http://en.wikipedia.org/wiki/Shard_(database_architecture)A database shard is a horizontal partition in adatabase or search engine. Each individual partitionis referred to as a shard.Horizontal partitioning is a database designprinciple whereby rows of a database table are heldseparately, rather than being split into columns.Each partition forms part of a shard, which may inturn be located on a separate database server orphysical location.
  • 12DIY Sharding ChallengesApplications must be modified to support multiple shards
  • 13• Maintaining DB ops and IPs in the app• Non-optimized sharding strategies– No good way to maintain global tablesreplicated across all database• Sacrifices development agility,additional administrative complexity• Results in database silos• Database ecosystem breaks becausethe application “conceals” shardingstrategies internally• Risks for data inconsistency• Adding and removing databasesis not supported…overprovisioning…• Jeopardizes high availability, backups & disaster recovery• Demands custom application code that can fail ACID complianceDIY Sharding ChallengesChallenges exist becauseapplication code changes arerequired to support multipledatabase instances.
  • ScaleBase Data Distribution Overview
  • 15Data Distribution: Application ExperienceWithout ScaleBase: App must be customized to support shardsWith ScaleBase: App sees ONE database……and doesn’t require any customizationScaleBase acts as a proxy between the app and thedatabase, virtualizing the database environment
  • 16Manual Sharding versus ScaleBaseSharding Limitations:• Major app rewrite, maintaining code• Maintaining DB ops & IPs in the app• Administration/3rd party tools are broken• DB silos/Database ecosystem is blind– Application “hides” sharding strategies• Non-optimized data distribution policy– No good way to maintain globaltables, replicated across all database• Sacrifices development agility• Adding/removing DBs is not supported• Risks for data inconsistency• Demands custom application code thatcan fail ACID compliance• Jeopardizes highavailability, backups, and disasterrecoveryScaleBase Benefits:• No hard-coding application re-writes• Unlimited scalability• Improve performance• Real time elasticity• ACID compliance• Verified data consistency• Real time monitoring, traffic analysis• Carefully analyze distribution policy• Enable system upgrades and updates• Simplified, centralized admin– Adding users– Changing schemas– Maintenance scripts– Management queries
  • 17Typical ScaleBase Data Traffic Manager DeploymentApplicationServersBIManagementDatabase A Replica ADatabase B Replica BDatabase C Replica CDatabase D Replica DUnlimited ScaleScaleBaseArchitectureis Fault Tolerant
  • ScaleBase Data Distribution – In Detail
  • 19ScaleBase Enables MySQL Scale Out without Re-writing Apps• Data distribution and scale-out is part of the databasearchitecture, not the application• One IP to connect to, and “see a unified database”– The application– Entire ecosystem (ETL, mysqldump, PHPMyAdmin)– No special sharding wizard developer– No app re-design, re-dev, re-QA, re-test, re-deploy– No hard-coded variables lost in the code– No special documentation
  • 20ScaleBase Enable Scale Out on AWS and PrivateClouds• A virtualized DB environment makes it easy to change realinfrastructure, because it’s decoupled from the application• No cloud makes your database elastic• ScaleBase enables elasticity of MySQL in the cloud (EC2, RDS, etc.)Scale-up hitsAWS’s tieredconfigurationlimits fastScale-out isunlimited andgives cloudflexibility
  • 21ScaleBase Supports Scale Out on Single & MultipleMachinesAdvantages of severalshards on one machine:– Several smaller MySQLinstances better utilizecores, memory– When data grows, eachinstance can later onmigrate to a biggermachine of its ownAdvantages of several shardson multiple machines– Leverage commodity hardware– When reaches machine limits -ScaleBase enables online dataredistribution (resharding) andshard-split
  • 22ScaleBase Enables Splitting Shards• ScaleBase also redistributes data across the array to eliminate hotspots, splitting the hot spot into two databases
  • 23ScaleBase Re-balances Shards• Special analysis and alerts about approaching limits• ScaleBase dynamically redistributes data (resharding) - moving thedata across the array from the over-utilized to the under-utilized
  • 24ScaleBase Provides Optimal Data Distribution PoliciesA good data distribution policy ensures that a specifictransaction is directed to a specific database1,000 transactions250transactions250transactions250transactions250transactions1,000 transactions
  • 25ScaleBase Eliminates Data SilosWhen a query needs datafrom several databases,ScaleBase:– Runs the query in parallelon all databases– Aggregates results into onemeaningful result-set to be returned to the client – the sameresult-set that would have been returned from a single DB!– Including cross-db GROUP BY, ORDER BY, aggregate functions– Including cross-db JOIN operations– Enables 2-phase commit for transactions spanning multipledatabases
  • 26ScaleBase Provides a Fault Tolerant ArchitectureApplicationServersBIManagementDatabase A Replica ADatabase B Replica BDatabase C Replica CDatabase D Replica DFully RedundantResilience to failuresScheduledmaintenance withoutdowntime
  • Summary
  • 28ScaleBase Delivers ScalabilityScale toUnlimitedThroughputNo SpecializedHardwareNoRe-architectureNo ApplicationRewrites
  • 30Detailed Scale Out Case StudiesLarge Chip Co• Scalability• Multiple Apps• Multiple growingusers• Availability• MySQL DBSolar Edge• Next GenMonitoring App• Massive Scale• Monitors realtime data fromthousands ofdistributedsystemsMozilla• New Product/Next Gen App/AppStore• Scalability• Geo-clusteringAppDynamics• Next gen APMcompany• Scalability for theNetfliximplementation
  • 31ScaleBase DeploymentEnvironments– Public Cloud– AWS, Rackspace, any– Private cloud– Hosted / on-premiseDatabases Supported– MySQL 5.1, 5.5, 5.6 (undercertification)– AWS RDS MySQL 5.1, 5.5– Maria DB 10.0 (undercertification)Path to Scale-Out:1. Data DistributionPolicy Analysis2. Functional Test3. Load Test4. Production Migration(safe, online)
  • 32SummaryScaleBase provides cost-effective Scale-Out solutions• Scale to an infinite number of users, data and transactions• Improve performance• No application rewrites• Real-time elasticity• ACID Compliant• Expert analysis and simple deployment• Leverage existing MySQL ecosystem/skills• Improve database visibility with real-time monitoring• Simplified, centralized administration
  • 33Questions (please enter directly into the GTW side panel)paul.campaniello@scalebase.comdoron.levari@scalebase.comwww.ScaleBase.com617.630.2800Additional Resourceshttp://www.scalebase.com/blog/http://www.scalebase.com/resources/@scalebase
  • 34Thank You