CakeFest 2014 -- MySQL Scaling Without Losing Your SAnity

  • 227 views
Uploaded on

Presented at the CakePHP CakeFest 2014 Madrid …

Presented at the CakePHP CakeFest 2014 Madrid

This talk covers an overview of how to scale from 1 MySQL instance to cover needs of availability, access, and costs

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
227
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
1
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1 Insert Picture Here MySQL Scaling § Dave Stokes MySQL Community Manager David.Stokes@Oracle.com @stoker slideshare.net/davestokes 23rd August 2014
  • 2. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.2 Safe Harbor Agreement .The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decision. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  • 3. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3 Insert Picture Here MySQL Releases since 2010 § Enterprise Monitor 2.2, 2.3 § Cluster 7.1, 7.2, 7.3 § Cluster Manager 1, 1.1, 1.3 § Workbench 5.2, 6, 6.1 § Database 5.5, 5.6, DMR 5.7 § Backup 3.5, 3.7, 3.1 § Oracle VM for MySQL § Oracle Product Certifications § Windows Installer § Utilities 1.0.6 § Audit § Fabric ● Under promise/Over Deliver ● More Developers ● More QA ● More Support ● Making $$ ● Classes popular – Instructor lead – Self-paced ● We're hiring!
  • 4. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4 You want to get a bigger, badder MySQL -- BUT what are the costs??? Budget  Complexity  Application  Opportunity  Sanity.
  • 5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.5 Simple Stuff
  • 6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.6 Option 0 Run the Latest, Greatest MySQL MySQL 5.5 was 20% faster than 5.1! 5.6 is another 10-15% --Make sure your config file matches your hardware --ANALYSE TABLE --Check for SLOW QUERIES, QUERIES w/o INDEXES --TUNE QUERIES This is the least complex, less upsetting, and usually lowest budget cost option with NO changes to your application!
  • 7. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.7 Option 0.1 Add in a Memcached Layer ● Cache for quick access ● Session Data ● Sports Scores ● Opens up NoSQL access to InnoDB later This is the second least complex, less upsetting, and usually lowest budget cost option but does make you change your application to become 'cache aware'.
  • 8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.8 Up Time versus Response Time versus Access Things to be engineered/negotiated upfront Up Time – How much down time can you afford? Response Time – How long is too long to wait? Access – How many do you need to serve simultaneously.
  • 9. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9 Insert Picture Here Simple Level 1 - Splitting Reads
  • 10. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.10 Use MySQL Replication to Split Read/Writes
  • 11. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.11 READ Splitting Most Common Scaling Technique ● More complex environment ● Application Changes ● Easier with PHP & Java ● Low opportunity and sanity costs ● Less risk of one server stopping operations
  • 12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.12 Read Splitting—Easy Way Use MySQL Utilities to copy master, set up failover ● mysqlserverclone ● mysqlfailover – When master dies, most up to date slave (GTID) promoted automatically ● Mysqlrplcheck & mysqlrplshow
  • 13. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13 Insert Picture Here Simple Level 2 Moderately Easy, Better Availability
  • 14. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.14 Distributed Replicated Block Device (DRBD) ● High Availability More complex environment ● Proven, supported ● Please practice fail over and back ● Easy to manage but not HA panacea
  • 15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.15 Cloud High Availability, low cost (sometimes) Slightly more complex environment ● Proven, supported ● Best for ephemeral data, may not make upper management comfortable (or you) ● Easy to manage but not a HA panacea
  • 16. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16 Insert Picture Here Complex Level N^N Never, Ever Down
  • 17. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.17 MySQL Cluster For 99.999% up time, upper bound on response time Extremely Complex ● No single point of failure = lots of $$ hardware ● Not general purpose MySQL Auto sharding ● Add/Subtract nodes at will ● Replicate between geographically separated locations ● Need a smart staff
  • 18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.18 MySQL Cluster Shards Data Automatically ● Single or multiple data node failures do not bring down a cluster ● Easy to add/remove data nodes
  • 19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.19 MySQL Cluster Topology ● Nodes – MGMT – startup/add node – Data – Mysqld ● Can be engineered for no single point of failure ● Can replicate across data centers
  • 20. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20 Insert Picture Here Big Data
  • 21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.21 Ask ten DBAs to define 'Big Data' and You Will Get At Least Eleven Answers ● BIG as massive amounts – Define MASSIVE ● BIG as in fast ● Define fast ● BIG as lots of eyes ● BIG as all of the the above!!! ● How many simultaneous users
  • 22. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.22 The Easiest Way to Get Speed NoSQL/Memcached to InnoDB/NBD ● 9x faster by skipping parser, optimizer ● Key/Value pair access and SQL access of same data
  • 23. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.23 Easiest Way to Gain Size Columnar Storage Engines for MySQL ● Not InnoDB ● Unique features, quirks ● Need > 26,000,000,000 rows to start seeing 'sweet spot' ● Use new server, don't scrimp See Calpont and Inforbright
  • 24. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.24 Hadoop-- getting data from MySQL to Hadoop 80% of Hadoop Installations feed from MySQL ● Hadoop is made up of several components ● Twisty little maze, all alike ● Use Apache Bigtop – Tested for interoperability ● Cloudera & Horton Works have great tutorials ● Kinda 'SQL-ish' ● Squoop and MySQL Hadoop Applier to move data
  • 25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.25
  • 26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.26 Partitioning and Sharding / Dividing and Conquering Partitions built-in to MySQL ● Nice management tool – Rolling quarters/months – Fast drops ● No change to application but higher complexity, low opportunity cost Sharding ● Much more complex application and operations ● Not off shelf ● Hard to re-engineer ● Works
  • 27. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.27 MySQL Fabric ● Lets you shard and reshard ● Very new ● Looking for feedback ● Released in May
  • 28. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28 Insert Picture Here Wrap up
  • 29. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.29 Goals – What are you trying to accomplish Define problem completely ● Who produces, who consumes data ● Negotiate performance rates ● Plan for failure, catastrophe ● Design for long life What does end solution look/feel like? ● Platforms ● Speed ● Access areas ● Security ● Serenity
  • 30. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.30 MySQL Marinate ● Free virtual class to learn MySQL run by the Boston MySQL Users Group – 1 chapter a week – Homework graded helpfully
  • 31. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.31 Q&A David.Stokes@Oracle.com @stoker slideshare.net/davestokes