MySQL Scaling

  • 1,032 views
Uploaded on

A look at best practices for bigger, badder MySQL -- including clustering, DRBD, read/write splitting, feeding Hadoop, NoSQL access to InnoDB/NDB, and vertical scaling.

A look at best practices for bigger, badder MySQL -- including clustering, DRBD, read/write splitting, feeding Hadoop, NoSQL access to InnoDB/NDB, and vertical scaling.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,032
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
46
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1
  • 2. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2 Insert Picture Here MySQL Scaling: Drinking from the fire hose, never being down & serving billions Dave Stokes MySQL Community Manager David.Stokes@Oracle.com @Stoker Slideshare.net/davestokes/presentations
  • 3. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3 You want to get a bigger, badder MySQL -- BUT what are the costs???  Budget  Complexity  Application  Opportunity  Sanity
  • 4. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4 Insert Picture Here Simple Stuff Insert Picture Here
  • 5. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5 Option 0 MySQL 5.5 was 20% faster than 5.1! 5.6 is another 10-15% --Make sure your config file matches your hardware --ANALYSE TABLE --Check for SLOW QUERIES, QUERIES w/o INDEXES --TUNE QUERIES This is the least complex, less upsetting, and usually lowest budget cost option Run the Latest, Greatest MySQL
  • 6. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6 Option 0.1 Cache for quick access ● Session Data ● Sports Scores ● Opens up NoSQL access to InnoDB later This is the second least complex, less upsetting, and usually lowest budget cost option but does make you change your application to become 'cache aware'. Add in a Memcached Layer
  • 7. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7 Insert Picture Here What database service Levels do you Need to Provide? Insert Picture Here
  • 8. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8 UpTime versus Response Time versus Access Up Time – How much down time can you afford? Response Time – How long is too long to wait? Access – How many do you need to serve simultaneously Things to be engineered/negotiated upfront
  • 9. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9 Insert Picture Here Simple Level 1 Read/Write Splitting Insert Picture Here
  • 10. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10 Use MySQL Replication to Split Read/Writes
  • 11. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11 Read/Write Splitting ● More complex environment ● Application Changes ● Easier with PHP & Java ● Low opportunity and sanity costs ● Less risk of one server stopping operations Most Common Scaling Technique
  • 12. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12 Read/Write Splitting—Easy Way ● mysqlserverclone ● mysqlfailover ● When master dies, most up to date slave (GTID) promoted automatically ● Mysqlrplcheck & mysqlrplshow Use MySQL Utilities to copy master, set up failover
  • 13. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13 Insert Picture Here Simple Level 2 Moderately Easy, Better Availability Insert Picture Here
  • 14. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14 Distributed Replicated Block Device (DRBD) ● More complex environment ● Proven, supported ● Please practice fail over and back ● Easy to manage but not HA panacea High Availability
  • 15. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15 CLOUD ● Slightly more complex environment ● Proven, supported ● Best for ephemeral data, may not make upper management comfortable (or you) ● Easy to manage but not HA panacea High Availability, Low Cost (maybe)
  • 16. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16 Insert Picture Here Complex Level N^N Never, Ever Down Insert Picture Here
  • 17. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17 MySQL Cluster For 99.999% uptime, upper bound on response time ● Extremely Complex ● No single point of failure = lots of $$ hardware ● Not general purpose MySQL ● Auto sharding ● Add/Subtract nodes at will ● Replicate between geographically separated locations ● Need a smart staff
  • 18. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18 MySQL Cluster Shards Data Automatically ● Single or multiple data node failures do not bring down a cluster ● Easy to add/remove data nodes
  • 19. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19 MySQL Cluster Topology ● Nodes ● MGMT – startup/add node ● Data ● Mysqld ● Can be engineered for no single point of failure ● Can replicate across data centers ● Expensive on all fronts
  • 20. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20 Insert Picture Here Big Data Insert Picture Here
  • 21. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21 Ask ten DBAs to define 'Big Data' and You Will Get At Least Eleven Answers .BIG as massive amounts ● Define MASSIVE BIG as in fast ● Define fast BIG as all of the other three!!!BIG as lots of eyes ● How many simultaneous users
  • 22. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22 The Easiest Way to Get Speed ● NoSQL/Memcached to InnoDB/NBD ● 9x faster by skipping parser, optimizer ● Key/Value pair access and SQL access of same data
  • 23. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23 Easiest Way to Gain Size ● Columnar Storage Engines for MySQL ● Not InnoDB ● Unique features, quirks ● Need > 26,000,000,000 rows to start seeing 'sweet spot' ● Use new server, don't scrimp
  • 24. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24 Calpont InfiniDB Column-oriented ● MySQL front end ● Multi threaded ● High speed data loading ● Automatic partitioning ● Enterprise Features ● Partition Drop ● UDF’s ● Physical compression ● MPP, multi-node scale-out ● Distributed, Shared Nothing data cache ● Auto Increment ● Centralized database and operating system monitoring ● Notifications and Alerts ● INSERT INTO...SELECT FROM... ● Views on InfiniDB Tables
  • 25. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.25 Infobright ● Similar Features to Calpont but different implementation
  • 26. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26 Hadoop-- getting data from MySQL to Hadoop ● 80% of Hadoop Installations feed from MySQl ● Hadoop is made up of several components ● Twisty little maze, all alike ● Use Apache Bigtop – Tested for interoperability ● Cloudera & Horton Works have great tutorials ● Kinda 'SQL-ish' ● Squoop and MySQL Hadoop Applier to move data
  • 27. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27 Hadoop Applier ● Infobright
  • 28. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28 Partitioning and Sharding / Dividing and Conquering ● Partitions built-in to MySQL ● Nice management tool – Rolling quarters/months – Fast drops ● No change to application but higher complexity, low opportunity cost ● Sharding ● Much more complex application and operations ● Not off shelf ● Hard to re-engineer ● Works
  • 29. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29 Insert Picture Here Hard Part – Thinking Insert Picture Here
  • 30. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30 Goals – What are you trying to accomplish ● Define problem completely ● Who produces, who consumes data ● Negotiate performance rates ● Plan for failure, catastrophe ● Design for long life ● What does end solution look/feel like? ● Platforms ● Speed ● Access areas ● Security ● Serenity
  • 31. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31 Where Else to Seek Info ● MySQL Connect Conference Sept 21-23 ● Planet.mysql.com and forums.mysql.com ● Hadoop ● Horton Works and Cloudera both have wonderful tutorials ● Apache Bigtop project
  • 32. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32 MySQL Training ● MySQL Marinate ● Online Virtual Class run by Boston MySQL Users Group ● 1 chapter/week ● Free ● Oracle and other vendors ● Dozens of book ● Online
  • 33. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33 Insert Picture Here David.Stokes@Oracle.com @Stoker slideshare.net/davestokes Insert Picture Here
  • 34. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34 Graphic Section Divider
  • 35. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35