Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
High Performance Rails with MySQL
Jervin Real, March 2014
I am…
• Consultant, Percona
• @dotmanila
• http://dotmanila.com/blog/
• http://www.mysqlperformanceblog.com/
Rails Fu Mastah!
http://walksalong.files.wordpress.com/2007/05/bruce_on_rails.jpg
Beginner
http://www.devonring.ca/img/ouch.png
Customer Problems
Web Apps Performance
• Powerful servers
• CPUs, higher clock speeds
• Lots of memory
• Fast storage
• Scale in the cloud
•...
Why Not?
• Premature scaling is expensive
• Cost inefficient
• Agile means less effective measurement to compensate
for fa...
Squeeze the Software
• Exhaust application optimizations first
• You cannot optimize what you can’t measure
• Cacti, NewRe...
• Dumb schemas and queries (somewhat)
Characteristics of Rails
:primary_key, :string, :text, :integer, :float, :decimal, 
...
• Dumb schemas
• Likes to use SHOW FIELDS
Characteristics of Rails
• Dumb schemas
• Likes to use SHOW FIELDS
• N+1 queries problem
• Well documented and discouraged for use
Characteristics ...
• Dumb schemas
• Likes to use SHOW FIELDS
• N+1 queries problem
• Well documented and discouraged for use
• Does SELECT FO...
• FK relationships - logical - GOOD
Rails - Good
• FK relationships - logical - GOOD
• Knows how to use PRIMARY KEYs — VERY GOOD!
Rails - Good
• FK relationships - logical - GOOD
• Knows how to use PRIMARY KEYs — VERY GOOD!
• Knows NOOP changes - GOOD
Rails - Good
• FK relationships - logical - GOOD
• Knows how to use PRIMARY KEYs — VERY GOOD!
• Knows NOOP changes - GOOD
• Database ag...
MySQL by Default
• Assumes you have less powerful hardware
• ironically on 5.5, assumes you have very powerful
CPUs - inno...
MySQL by Default
• Assumes you have less powerful hardware
• ironically on 5.5, assumes you have very powerful
CPUs - inno...
MySQL by Default
• Assumes you have less powerful hardware
• ironically on 5.5, assumes you have very powerful
CPUs - inno...
MySQL by Default
• Assumes you have less powerful hardware
• ironically on 5.5, assumes you have very powerful
CPUs - inno...
What to Optimize - MySQL
• Disable Query Cache
What to Optimize - MySQL
• Disable Query Cache
• query_cache_size = 0
What to Optimize - MySQL
• Disable Query Cache
• query_cache_size = 0
• query_cache_type = 0
What to Optimize - MySQL
• Disable Query Cache
• skip_name_resolve
What to Optimize - MySQL
• Disable Query Cache
• skip_name_resolve
• Use >= 5.5
What to Optimize - MySQL
• Disable Query Cache
• skip_name_resolve
• Use >= 5.5
• Use Indexes, EXPLAIN should be your frie...
What to Optimize - MySQL
• Disable Query Cache
• skip_name_resolve
• Use >= 5.5
• Use Indexes, EXPLAIN should be your frie...
What to Optimize - MySQL
• Slow queries - search and destroy
• long_query_time = 0
• log_slow_verbosity - Percona Server
•...
What to Optimize - MySQL
• Use InnoDB
innodb_buffer_pool_size       #  keep  hot  data  in  memory  
innodb_log_file_size ...
What to Optimize - Rails
• counter_cache
• Good for InnoDB if you often SELECT COUNT(*)
• Indexes for find_by, where and f...
What to Optimize - Rails
• dependent: :destroy
24  Query          BEGIN  
24  Query          SELECT  `comments`.*  FROM  `...
What to Optimize - Rails
• Cache SHOW FIELDS output - patches for now
• Disable innodb_stats_on_metadata
What to Optimize - Rails
• Cache SHOW FIELDS output - patches for now
• Disable innodb_stats_on_metadata
• Pull only the c...
What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affect...
What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affect...
What to Optimize - Rails
@posts  =  Post.limit(2)  
!
@posts.each  do  |post|  
   puts  post.authors.name  
end
24  SELEC...
What to Optimize - Rails
@posts  =  Post.includes(:authors).limit(2)
24  Query          SELECT    `posts`.*  FROM  `posts`...
What to Optimize - Rails
@posts  =  Post.joins(:authors).limit(2);
24  Query          SELECT    `posts`.*  FROM  `posts`  ...
What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affect...
What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affect...
What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affect...
Further Thoughts
• GDB, Strace, OProfile
• Smart aggregation, use summary tables when possible
• Rails > Caching > MySQL
•...
http://www.amyvernon.net/wp-content/uploads/2014/01/upward-graph-striving.png
Thank you!
… and
• We’re hiring!
• http://www.percona.com/about-us/careers/open-
positions
Upcoming SlideShare
Loading in …5
×

High Performance Rails with MySQL

2,583 views

Published on

RubyConfPH 2014

Published in: Software, Technology
  • Be the first to comment

High Performance Rails with MySQL

  1. 1. High Performance Rails with MySQL Jervin Real, March 2014
  2. 2. I am… • Consultant, Percona • @dotmanila • http://dotmanila.com/blog/ • http://www.mysqlperformanceblog.com/
  3. 3. Rails Fu Mastah! http://walksalong.files.wordpress.com/2007/05/bruce_on_rails.jpg
  4. 4. Beginner http://www.devonring.ca/img/ouch.png
  5. 5. Customer Problems
  6. 6. Web Apps Performance • Powerful servers • CPUs, higher clock speeds • Lots of memory • Fast storage • Scale in the cloud • Agile development techniques
  7. 7. Why Not? • Premature scaling is expensive • Cost inefficient • Agile means less effective measurement to compensate for fast deployments
  8. 8. Squeeze the Software • Exhaust application optimizations first • You cannot optimize what you can’t measure • Cacti, NewRelic, Scout • NewRelic RPM Developer Mode, Rails Footnotes (2, 3, 4!), Google PerfTools for Ruby
  9. 9. • Dumb schemas and queries (somewhat) Characteristics of Rails :primary_key, :string, :text, :integer, :float, :decimal, :datetime, :timestamp, :time, :date, :binary, :boolean
  10. 10. • Dumb schemas • Likes to use SHOW FIELDS Characteristics of Rails
  11. 11. • Dumb schemas • Likes to use SHOW FIELDS • N+1 queries problem • Well documented and discouraged for use Characteristics of Rails
  12. 12. • Dumb schemas • Likes to use SHOW FIELDS • N+1 queries problem • Well documented and discouraged for use • Does SELECT FOR UPDATE • NOOP transactions still wrapped in BEGIN/COMMIT Characteristics of Rails
  13. 13. • FK relationships - logical - GOOD Rails - Good
  14. 14. • FK relationships - logical - GOOD • Knows how to use PRIMARY KEYs — VERY GOOD! Rails - Good
  15. 15. • FK relationships - logical - GOOD • Knows how to use PRIMARY KEYs — VERY GOOD! • Knows NOOP changes - GOOD Rails - Good
  16. 16. • FK relationships - logical - GOOD • Knows how to use PRIMARY KEYs — VERY GOOD! • Knows NOOP changes - GOOD • Database agnostic - GOOD Rails - Good
  17. 17. MySQL by Default • Assumes you have less powerful hardware • ironically on 5.5, assumes you have very powerful CPUs - innodb_thread_concurrency = 0
  18. 18. MySQL by Default • Assumes you have less powerful hardware • ironically on 5.5, assumes you have very powerful CPUs - innodb_thread_concurrency = 0 • Not optimized for faster storage
  19. 19. MySQL by Default • Assumes you have less powerful hardware • ironically on 5.5, assumes you have very powerful CPUs - innodb_thread_concurrency = 0 • Not optimized for faster storage • Still have bad configuration assumptions • Query cache • MyISAM < 5.5.5
  20. 20. MySQL by Default • Assumes you have less powerful hardware • ironically on 5.5, assumes you have very powerful CPUs - innodb_thread_concurrency = 0 • Not optimized for faster storage • Still have bad configuration assumptions • Query cache • MyISAM < 5.5.5 • Still haunted by mutexes
  21. 21. What to Optimize - MySQL • Disable Query Cache
  22. 22. What to Optimize - MySQL • Disable Query Cache • query_cache_size = 0
  23. 23. What to Optimize - MySQL • Disable Query Cache • query_cache_size = 0 • query_cache_type = 0
  24. 24. What to Optimize - MySQL • Disable Query Cache • skip_name_resolve
  25. 25. What to Optimize - MySQL • Disable Query Cache • skip_name_resolve • Use >= 5.5
  26. 26. What to Optimize - MySQL • Disable Query Cache • skip_name_resolve • Use >= 5.5 • Use Indexes, EXPLAIN should be your friend!
  27. 27. What to Optimize - MySQL • Disable Query Cache • skip_name_resolve • Use >= 5.5 • Use Indexes, EXPLAIN should be your friend! • 5.6 does Subquery Optimizations
  28. 28. What to Optimize - MySQL • Slow queries - search and destroy • long_query_time = 0 • log_slow_verbosity - Percona Server • pt-query-digest/Percona Cloud Tools
  29. 29. What to Optimize - MySQL • Use InnoDB innodb_buffer_pool_size       #  keep  hot  data  in  memory   innodb_log_file_size             #  allow  more  IO  buffer   innodb_flush_method  =  O_DIRECT  #  skip  OS  cache   innodb_[read|write]_io_threads  >  4   innodb_io_capacity   innodb_adaptive_flushing_method
  30. 30. What to Optimize - Rails • counter_cache • Good for InnoDB if you often SELECT COUNT(*) • Indexes for find_by, where and family @posts  =  Post.where(title:  params[:keyword])   ! mysql>  EXPLAIN  SELECT  `posts`.*  FROM  `posts`    WHERE  `posts`.`title`  =  'LongTitles'  G   ***************************  1.  row  ***************************                        id:  1      select_type:  SIMPLE                  table:  posts                    type:  ALL   possible_keys:  NULL                      key:  NULL              key_len:  NULL                      ref:  NULL                    rows:  2                  Extra:  Using  where   1  row  in  set  (0.00  sec)
  31. 31. What to Optimize - Rails • dependent: :destroy 24  Query          BEGIN   24  Query          SELECT  `comments`.*  FROM  `comments`    WHERE  `comments`.`post_id`  =  1   24  Query          DELETE  FROM  `comments`  WHERE  `comments`.`id`  =  2   24  Query          DELETE  FROM  `posts`  WHERE  `posts`.`id`  =  1   24  Query          COMMIT 24  Query          BEGIN   24  Query          DELETE  FROM  `comments`  WHERE  `comments`.`post_id`  =  4   24  Query          DELETE  FROM  `posts`  WHERE  `posts`.`id`  =  4   24  Query          COMMIT BAD Use dependent: :delete_all
  32. 32. What to Optimize - Rails • Cache SHOW FIELDS output - patches for now • Disable innodb_stats_on_metadata
  33. 33. What to Optimize - Rails • Cache SHOW FIELDS output - patches for now • Disable innodb_stats_on_metadata • Pull only the columns you need - especially excluding BLOBS @posts  =  Post.select(“title”)
  34. 34. What to Optimize - Rails • Avoid pessimistic locks when possible - SELECT … FOR UPDATE - UPDATE directly and return affected rows
  35. 35. What to Optimize - Rails • Avoid pessimistic locks when possible - SELECT … FOR UPDATE - UPDATE directly and return affected rows • Avoid N+1 queries - use JOIN or includes
  36. 36. What to Optimize - Rails @posts  =  Post.limit(2)   ! @posts.each  do  |post|     puts  post.authors.name   end 24  SELECT    `posts`.*  FROM  `posts`    LIMIT  2   24  Query          SELECT    `authors`.*  FROM  `authors`    WHERE   `authors`.`id`  =  1    ORDER  BY  `authors`.`id`  ASC  LIMIT  1   24  Query          SELECT    `authors`.*  FROM  `authors`    WHERE   `authors`.`id`  =  2    ORDER  BY  `authors`.`id`  ASC  LIMIT  1 With this: You get this:
  37. 37. What to Optimize - Rails @posts  =  Post.includes(:authors).limit(2) 24  Query          SELECT    `posts`.*  FROM  `posts`    LIMIT  2   24  Query          SELECT  `authors`.*  FROM  `authors`     WHERE  `authors`.`id`  IN  (1,  2) With this: You get this:
  38. 38. What to Optimize - Rails @posts  =  Post.joins(:authors).limit(2); 24  Query          SELECT    `posts`.*  FROM  `posts`  INNER   JOIN  `authors`  ON  `authors`.`id`  =   `posts`.`authors_id`  LIMIT  2 With this: You get this:
  39. 39. What to Optimize - Rails • Avoid pessimistic locks when possible - SELECT … FOR UPDATE - UPDATE directly and return affected rows • Avoid N+1 queries - use JOIN or includes • Compress MySQL requests, especially transactions!
  40. 40. What to Optimize - Rails • Avoid pessimistic locks when possible - SELECT … FOR UPDATE - UPDATE directly and return affected rows • Avoid N+1 queries - use JOIN or includes • Compress MySQL requests, especially transactions! • Learn and use SQL - find_by_sql
  41. 41. What to Optimize - Rails • Avoid pessimistic locks when possible - SELECT … FOR UPDATE - UPDATE directly and return affected rows • Avoid N+1 queries - use JOIN or includes • Compress MySQL requests, especially transactions! • Learn and use SQL - find_by_sql • Don’t accept Model defaults
  42. 42. Further Thoughts • GDB, Strace, OProfile • Smart aggregation, use summary tables when possible • Rails > Caching > MySQL • Avoid: config.action_controller.session_store  =  :active_record_store
  43. 43. http://www.amyvernon.net/wp-content/uploads/2014/01/upward-graph-striving.png
  44. 44. Thank you!
  45. 45. … and • We’re hiring! • http://www.percona.com/about-us/careers/open- positions

×