Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2
Insert Picture Here
MySQL Scaling:
Drinking from the...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
You want to get a bigger, badder MySQL --
BUT what a...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4
Insert Picture Here
Simple Stuff
Insert Picture Here
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5
Option 0
MySQL 5.5 was 20% faster than 5.1! 5.6 is a...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6
Option 0.1
Cache for quick access
●
Session Data
●
S...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7
Insert Picture Here
What database service
Levels do ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8
UpTime versus Response Time versus Access
Up Time
– ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9
Insert Picture Here
Simple Level 1
Read/Write Splitt...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10
Use MySQL Replication to Split Read/Writes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11
Read/Write Splitting
●
More complex environment
●
A...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12
Read/Write Splitting—Easy Way
●
mysqlserverclone
●
...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13
Insert Picture Here
Simple Level 2
Moderately Easy,...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14
Distributed Replicated
Block Device (DRBD)
●
More c...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15
CLOUD
●
Slightly more complex environment
●
Proven,...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16
Insert Picture Here
Complex Level N^N
Never, Ever D...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17
MySQL Cluster
For 99.999% uptime, upper bound on re...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18
MySQL Cluster Shards Data Automatically
● Single or...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19
MySQL Cluster Topology
● Nodes
● MGMT – startup/add...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20
Insert Picture Here
Big Data
Insert Picture Here
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21
Ask ten DBAs to define 'Big Data' and You Will
Get ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22
The Easiest Way to Get Speed
● NoSQL/Memcached to I...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23
Easiest Way to Gain Size
● Columnar Storage Engines...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24
Calpont
InfiniDB
Column-oriented
● MySQL front end
...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.25
Infobright
● Similar Features
to Calpont but
differ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26
Hadoop-- getting data from MySQL to Hadoop
● 80% of...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27
Hadoop Applier
● Infobright
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28
Partitioning and Sharding /
Dividing and Conquering...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29
Insert Picture Here
Hard Part – Thinking
Insert Pic...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30
Goals – What are you trying to
accomplish
● Define ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31
Where Else to Seek Info
● MySQL Connect Conference
...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32
MySQL Training
● MySQL Marinate
● Online Virtual Cl...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33
Insert Picture Here
David.Stokes@Oracle.com
@Stoker...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34
Graphic Section Divider
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35
Upcoming SlideShare
Loading in...5
×

MySQL Scaling

1,175

Published on

A look at best practices for bigger, badder MySQL -- including clustering, DRBD, read/write splitting, feeding Hadoop, NoSQL access to InnoDB/NDB, and vertical scaling.

Published in: Technology, News & Politics
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,175
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
51
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

MySQL Scaling

  1. 1. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1
  2. 2. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2 Insert Picture Here MySQL Scaling: Drinking from the fire hose, never being down & serving billions Dave Stokes MySQL Community Manager David.Stokes@Oracle.com @Stoker Slideshare.net/davestokes/presentations
  3. 3. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3 You want to get a bigger, badder MySQL -- BUT what are the costs???  Budget  Complexity  Application  Opportunity  Sanity
  4. 4. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4 Insert Picture Here Simple Stuff Insert Picture Here
  5. 5. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5 Option 0 MySQL 5.5 was 20% faster than 5.1! 5.6 is another 10-15% --Make sure your config file matches your hardware --ANALYSE TABLE --Check for SLOW QUERIES, QUERIES w/o INDEXES --TUNE QUERIES This is the least complex, less upsetting, and usually lowest budget cost option Run the Latest, Greatest MySQL
  6. 6. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6 Option 0.1 Cache for quick access ● Session Data ● Sports Scores ● Opens up NoSQL access to InnoDB later This is the second least complex, less upsetting, and usually lowest budget cost option but does make you change your application to become 'cache aware'. Add in a Memcached Layer
  7. 7. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7 Insert Picture Here What database service Levels do you Need to Provide? Insert Picture Here
  8. 8. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8 UpTime versus Response Time versus Access Up Time – How much down time can you afford? Response Time – How long is too long to wait? Access – How many do you need to serve simultaneously Things to be engineered/negotiated upfront
  9. 9. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9 Insert Picture Here Simple Level 1 Read/Write Splitting Insert Picture Here
  10. 10. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10 Use MySQL Replication to Split Read/Writes
  11. 11. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11 Read/Write Splitting ● More complex environment ● Application Changes ● Easier with PHP & Java ● Low opportunity and sanity costs ● Less risk of one server stopping operations Most Common Scaling Technique
  12. 12. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12 Read/Write Splitting—Easy Way ● mysqlserverclone ● mysqlfailover ● When master dies, most up to date slave (GTID) promoted automatically ● Mysqlrplcheck & mysqlrplshow Use MySQL Utilities to copy master, set up failover
  13. 13. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13 Insert Picture Here Simple Level 2 Moderately Easy, Better Availability Insert Picture Here
  14. 14. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14 Distributed Replicated Block Device (DRBD) ● More complex environment ● Proven, supported ● Please practice fail over and back ● Easy to manage but not HA panacea High Availability
  15. 15. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15 CLOUD ● Slightly more complex environment ● Proven, supported ● Best for ephemeral data, may not make upper management comfortable (or you) ● Easy to manage but not HA panacea High Availability, Low Cost (maybe)
  16. 16. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16 Insert Picture Here Complex Level N^N Never, Ever Down Insert Picture Here
  17. 17. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17 MySQL Cluster For 99.999% uptime, upper bound on response time ● Extremely Complex ● No single point of failure = lots of $$ hardware ● Not general purpose MySQL ● Auto sharding ● Add/Subtract nodes at will ● Replicate between geographically separated locations ● Need a smart staff
  18. 18. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18 MySQL Cluster Shards Data Automatically ● Single or multiple data node failures do not bring down a cluster ● Easy to add/remove data nodes
  19. 19. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19 MySQL Cluster Topology ● Nodes ● MGMT – startup/add node ● Data ● Mysqld ● Can be engineered for no single point of failure ● Can replicate across data centers ● Expensive on all fronts
  20. 20. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20 Insert Picture Here Big Data Insert Picture Here
  21. 21. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21 Ask ten DBAs to define 'Big Data' and You Will Get At Least Eleven Answers .BIG as massive amounts ● Define MASSIVE BIG as in fast ● Define fast BIG as all of the other three!!!BIG as lots of eyes ● How many simultaneous users
  22. 22. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22 The Easiest Way to Get Speed ● NoSQL/Memcached to InnoDB/NBD ● 9x faster by skipping parser, optimizer ● Key/Value pair access and SQL access of same data
  23. 23. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23 Easiest Way to Gain Size ● Columnar Storage Engines for MySQL ● Not InnoDB ● Unique features, quirks ● Need > 26,000,000,000 rows to start seeing 'sweet spot' ● Use new server, don't scrimp
  24. 24. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24 Calpont InfiniDB Column-oriented ● MySQL front end ● Multi threaded ● High speed data loading ● Automatic partitioning ● Enterprise Features ● Partition Drop ● UDF’s ● Physical compression ● MPP, multi-node scale-out ● Distributed, Shared Nothing data cache ● Auto Increment ● Centralized database and operating system monitoring ● Notifications and Alerts ● INSERT INTO...SELECT FROM... ● Views on InfiniDB Tables
  25. 25. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.25 Infobright ● Similar Features to Calpont but different implementation
  26. 26. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26 Hadoop-- getting data from MySQL to Hadoop ● 80% of Hadoop Installations feed from MySQl ● Hadoop is made up of several components ● Twisty little maze, all alike ● Use Apache Bigtop – Tested for interoperability ● Cloudera & Horton Works have great tutorials ● Kinda 'SQL-ish' ● Squoop and MySQL Hadoop Applier to move data
  27. 27. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27 Hadoop Applier ● Infobright
  28. 28. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28 Partitioning and Sharding / Dividing and Conquering ● Partitions built-in to MySQL ● Nice management tool – Rolling quarters/months – Fast drops ● No change to application but higher complexity, low opportunity cost ● Sharding ● Much more complex application and operations ● Not off shelf ● Hard to re-engineer ● Works
  29. 29. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29 Insert Picture Here Hard Part – Thinking Insert Picture Here
  30. 30. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30 Goals – What are you trying to accomplish ● Define problem completely ● Who produces, who consumes data ● Negotiate performance rates ● Plan for failure, catastrophe ● Design for long life ● What does end solution look/feel like? ● Platforms ● Speed ● Access areas ● Security ● Serenity
  31. 31. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31 Where Else to Seek Info ● MySQL Connect Conference Sept 21-23 ● Planet.mysql.com and forums.mysql.com ● Hadoop ● Horton Works and Cloudera both have wonderful tutorials ● Apache Bigtop project
  32. 32. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32 MySQL Training ● MySQL Marinate ● Online Virtual Class run by Boston MySQL Users Group ● 1 chapter/week ● Free ● Oracle and other vendors ● Dozens of book ● Online
  33. 33. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33 Insert Picture Here David.Stokes@Oracle.com @Stoker slideshare.net/davestokes Insert Picture Here
  34. 34. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34 Graphic Section Divider
  35. 35. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×