• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Scaling MySQL writes through Partitioning
 

Scaling MySQL writes through Partitioning

on

  • 18,867 views

MySQL is fairly fast for most applications, but every now and then you might come across an application that needs fast writes at a very large scale. The problem with this is that if your inserts are ...

MySQL is fairly fast for most applications, but every now and then you might come across an application that needs fast writes at a very large scale. The problem with this is that if your inserts are not ordered according to your primary key, then once you cross a limit known as the innodb_buffer_pool_size (for InnoDB tables), write performance starts to degrade because you're now hitting disk a lot. This limit is dependent on the amount of RAM you have, and that in turn is limited by how much money you can throw at the problem. At some point it makes sense to close your pocket and start thinking of a way to hack around the problem.

Faced with a similar problem, we figured out a way to hack it with MySQL 5's partitioning to scale writes to a consistently high rate.

In this talk, I'll cover all the steps we went through to get to this solution in the hopes that either the solution itself or the thought processes behind it will help others solve their own scaling problems.

Statistics

Views

Total Views
18,867
Views on SlideShare
16,779
Embed Views
2,088

Actions

Likes
45
Downloads
582
Comments
6

19 Embeds 2,088

http://blog.corujadeti.com.br 1188
http://www.slideshare.net 649
http://erickvitor.wordpress.com 85
http://www.techiegyan.com 52
http://www.corujadeti.com 37
http://phpperformance.de 24
http://talks.bluesmoon.info 15
http://www.berejeb.com 9
http://wiki.redtonic 9
http://www.linkedin.com 4
http://www.schoox.com 3
http://translate.googleusercontent.com 3
http://webcache.googleusercontent.com 3
http://www.devetdesign.com 2
https://twitter.com 1
https://si0.twimg.com 1
https://twimg0-a.akamaihd.net 1
http://www.brijj.com 1
https://www.linkedin.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

16 of 6 previous next Post a comment

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Great presentation, very helpful
    Are you sure you want to
    Your message goes here
    Processing…
  • There are two components to write rate. One is the number of inserts per second. This is simply the total number of inserts divided by total time for each run. The second is disk activity and I measure this using a Yahoo! internal tool similar to iostat
    Are you sure you want to
    Your message goes here
    Processing…
  • Hi Mathieu,

    I used YUI-Flot (my port of jQuery's Flot to YUI) to make the graphs from json data, and I used a simple awk script to generate the json from csv/tsv data generated by my perl script and from sar.
    Are you sure you want to
    Your message goes here
    Processing…
  • Nice presentation !
    How do you created these graphs ?
    Thank you
    Are you sure you want to
    Your message goes here
    Processing…
  • was having a lot of trouble with slideshare when I uploaded it. It should be downloadable now.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Scaling MySQL writes through Partitioning Scaling MySQL writes through Partitioning Presentation Transcript