Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

15 protips for mysql users

1,483 views

Published on

15 tips taken from the field on how to achieve a better performance on your MySQL server.

Published in: Technology
  • Be the first to comment

15 protips for mysql users

  1. 1. 15 Pro-tips forMySQL UsersPHPBenelux Meeting - March 2011Enrise HQ - Amersfoorthttp://joind.in/talk/view/2947 1
  2. 2. Who am I? 2
  3. 3. Who am I?Joshua Thijssen (32) 2
  4. 4. Who am I?Joshua Thijssen (32)Senior Software Engineer @ Enrise 2
  5. 5. Who am I?Joshua Thijssen (32)Senior Software Engineer @ EnriseDevelopment in PHP, Python, Perl,C, Java.... 2
  6. 6. Who am I?Joshua Thijssen (32)Senior Software Engineer @ EnriseDevelopment in PHP, Python, Perl,C, Java....Blogs: http://www.adayinthelifeof.nl http://www.enrise.com/blog 2
  7. 7. Who am I?Joshua Thijssen (32)Senior Software Engineer @ EnriseDevelopment in PHP, Python, Perl,C, Java....Blogs: http://www.adayinthelifeof.nl http://www.enrise.com/blogEmail: joshua@enrise.com 2
  8. 8. Who am I?Joshua Thijssen (32)Senior Software Engineer @ EnriseDevelopment in PHP, Python, Perl,C, Java....Blogs: http://www.adayinthelifeof.nl http://www.enrise.com/blogEmail: joshua@enrise.comTwitter: @jaytaphIdenti.ca: jaytaph 2
  9. 9. What are we going to discuss?‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD 3
  10. 10. What are we going to discuss? ‣ 15 MySQL Pro-tips‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD 3
  11. 11. What are we going to discuss? ‣ 15 MySQL Pro-tips ‣ No “theoretical tips”, all taken from the field.‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD 3
  12. 12. What are we going to discuss? ‣ 15 MySQL Pro-tips ‣ No “theoretical tips”, all taken from the field. ‣ Starting simple - ending “complex”‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD 3
  13. 13. Tip 1 1) Know how to use explain.‣ EXPLAIN IS YOUR BESTEST FRIEND 4
  14. 14. Tip 1: Know your EXPLAIN (1) 5
  15. 15. Tip 1: Know your EXPLAIN (1) ‣ I will not show you how to use EXPLAIN. 5
  16. 16. Tip 1: Know your EXPLAIN (1) ‣ I will not show you how to use EXPLAIN. ‣ Use EXPLAIN and EXPLAIN EXTENDED/ SHOW WARNINGS; 5
  17. 17. Tip 1: Know your EXPLAIN (1) 6
  18. 18. Tip 1: Know your EXPLAIN (1) 6
  19. 19. Tip 1: Know your EXPLAIN (1) 6
  20. 20. Tip 2 2) Know the most basic my.cnf settings‣ THERE ARE ONLY A FEW “BASIC” ONES. 7
  21. 21. Tip 2: My.cnf settings (1) Know the most important ones: key_buffer_size, innodb_buffer_pool_size, sort_buffer_size, max_connections 8
  22. 22. Tip 2: My.cnf settings (2) 9
  23. 23. Tip 2: My.cnf settings (2) ‣ Some settings work on global level, some per connection! 9
  24. 24. Tip 2: My.cnf settings (2) ‣ Some settings work on global level, some per connection! ‣ Know some quirks: (max_heap_table_size vs tmp_table_size, binlog- do-db, replicate-ignore-db etc) 9
  25. 25. Tip 2: My.cnf settings (3) ‣ http://www.omh.cc/mycnf/ ‣ http://rackerhacker.com/mysqltuner/ ‣ http://www.day32.com/MySQL/ ‣ phpmyadmin 10
  26. 26. Tip 3 3) Backup on table level‣ RESTORING JUST ONE TABLE CAN BE PAINFUL OTHERWISE 11
  27. 27. Tip 3: Backup on table level (1)‣ COULD YOU RESTORE TABLE x? YES! YES I CAN! 12
  28. 28. Tip 3: Backup on table level (1) ‣ mysqldump can dump per database OR by table.‣ COULD YOU RESTORE TABLE x? YES! YES I CAN! 12
  29. 29. Tip 3: Backup on table level (1) ‣ mysqldump can dump per database OR by table. ‣ Simple scripts to scan/dump tables.‣ COULD YOU RESTORE TABLE x? YES! YES I CAN! 12
  30. 30. Tip 3: Backup on table level (1) ‣ mysqldump can dump per database OR by table. ‣ Simple scripts to scan/dump tables. ‣ Easy restore for single table (or part of table)‣ COULD YOU RESTORE TABLE x? YES! YES I CAN! 12
  31. 31. Tip 4 4) Don’t use “SELECT *” when you only need 1 or 2 fields.‣ DON’T ASK WHAT YOU DON’T NEED 13
  32. 32. Tip 4: Select * (1)‣ DON’T ASK WHAT YOU DON’T NEED 14
  33. 33. Tip 4: Select * (1) ‣ Much more data to be read from disk‣ DON’T ASK WHAT YOU DON’T NEED 14
  34. 34. Tip 4: Select * (1) ‣ Much more data to be read from disk ‣ Much more data will be send over, thus slower (blobs/texts)‣ DON’T ASK WHAT YOU DON’T NEED 14
  35. 35. Tip 4: Select * (1) ‣ Much more data to be read from disk ‣ Much more data will be send over, thus slower (blobs/texts) ‣ Cannot use covering indices‣ DON’T ASK WHAT YOU DON’T NEED 14
  36. 36. Tip 4: Select * (2) 15
  37. 37. Tip 4: Select * (2) 15
  38. 38. Tip 5 5) Use triggers and stored procedures‣ ENFORCE CONSISTENCY 16
  39. 39. Tip 5: Triggers and stored procedures (1)‣ ENFORCE CONSISTENCY 17
  40. 40. Tip 5: Triggers and stored procedures (1) ‣ 6 triggers per table (insert, update, delete, before and after the mutation)‣ ENFORCE CONSISTENCY 17
  41. 41. Tip 5: Triggers and stored procedures (1) ‣ 6 triggers per table (insert, update, delete, before and after the mutation) ‣ 3rd party tools (phpmyadmin etc) can also use the database without loosing data consistency.‣ ENFORCE CONSISTENCY 17
  42. 42. Tip 5: Triggers and stored procedures (1) ‣ 6 triggers per table (insert, update, delete, before and after the mutation) ‣ 3rd party tools (phpmyadmin etc) can also use the database without loosing data consistency. ‣ Watch out with (phpmyadmin) table dumps!‣ ENFORCE CONSISTENCY 17
  43. 43. Tip 6 6) Don’t use FULLTEXT searches‣ THERE ARE MUCH BETTER SOLUTIONS 18
  44. 44. Tip 6: Don’t use FULLTEXT search (1)‣ THERE ARE MUCH BETTER SOLUTIONS 19
  45. 45. Tip 6: Don’t use FULLTEXT search (1)‣ They only work for MyISAM tables.‣ THERE ARE MUCH BETTER SOLUTIONS 19
  46. 46. Tip 6: Don’t use FULLTEXT search (1)‣ They only work for MyISAM tables.‣ Not compatible with other DB’s.‣ THERE ARE MUCH BETTER SOLUTIONS 19
  47. 47. Tip 6: Don’t use FULLTEXT search (1)‣ They only work for MyISAM tables.‣ Not compatible with other DB’s.‣ Slow (especially compared to Solr, Sphinx).‣ THERE ARE MUCH BETTER SOLUTIONS 19
  48. 48. Tip 6: Don’t use FULLTEXT search (1)‣ They only work for MyISAM tables.‣ Not compatible with other DB’s.‣ Slow (especially compared to Solr, Sphinx).‣ No extra features (faceted search, spell checking etc).‣ THERE ARE MUCH BETTER SOLUTIONS 19
  49. 49. Tip 7 7) Wildcard searches (%item%) are bad for performance‣ IT LOOKS LIKE YOU NEED MORE ADVANCED 20
  50. 50. Tip 7: Wildcard searches (1)‣ THERE ARE MUCH BETTER SOLUTIONS 21
  51. 51. Tip 7: Wildcard searches (1) ‣ MySQL cannot use indexes!‣ THERE ARE MUCH BETTER SOLUTIONS 21
  52. 52. Tip 7: Wildcard searches (1) ‣ MySQL cannot use indexes! ‣ Revert your data: search for ‘moc.esirne@%’ instead of ‘%@enrise.com’.‣ THERE ARE MUCH BETTER SOLUTIONS 21
  53. 53. Tip 7: Wildcard searches (1) ‣ MySQL cannot use indexes! ‣ Revert your data: search for ‘moc.esirne@%’ instead of ‘%@enrise.com’. ‣ Use a better solution (solr, sphinx). You probably want it.‣ THERE ARE MUCH BETTER SOLUTIONS 21
  54. 54. Tip 8 8) Shard your volatile and non- volatile data.‣ MAKE CACHING AND LOCKING HAPPY AGAIN 22
  55. 55. Tip 8: Sharding (1) 23
  56. 56. Tip 8: Sharding (2) 24
  57. 57. Tip 8: Sharding (2) ‣ Remember: an update on a table will invalidate ALL queries referring to that table. 24
  58. 58. Tip 8: Sharding (2) ‣ Remember: an update on a table will invalidate ALL queries referring to that table. ‣ UPDATE pages SET hit_count = hit_count + 1; 24
  59. 59. Tip 8: Sharding (2) ‣ Remember: an update on a table will invalidate ALL queries referring to that table. ‣ UPDATE pages SET hit_count = hit_count + 1; ‣ Thus: page table will NEVER be cached. 24
  60. 60. Tip 8: Sharding (3) 25
  61. 61. Tip 8: Sharding (3) ‣ Define hot data (volatile, changes often) and cold data (static, changes never or infrequently) 25
  62. 62. Tip 8: Sharding (3) ‣ Define hot data (volatile, changes often) and cold data (static, changes never or infrequently) ‣ move to different tables 25
  63. 63. Tip 8: Sharding (3) ‣ Define hot data (volatile, changes often) and cold data (static, changes never or infrequently) ‣ move to different tables ‣ UPDATE page_stats SET hit_count = hit_count + 1; 25
  64. 64. Tip 8: Sharding (3) ‣ Define hot data (volatile, changes often) and cold data (static, changes never or infrequently) ‣ move to different tables ‣ UPDATE page_stats SET hit_count = hit_count + 1; ‣ Query cache is happy again 25
  65. 65. Tip 9 9) Don’t use a large Primary Key for InnoDB tables.‣ PK’S ARE ON EVERY INDEX 26
  66. 66. Tip 9: Large primary keys (1) 27
  67. 67. Tip 9: Large primary keys (1) ‣ InnoDB adds the primary key to EACH index. 27
  68. 68. Tip 9: Large primary keys (1) ‣ InnoDB adds the primary key to EACH index. ‣ No primary key given? It uses an internal 6(!)-byte key. 27
  69. 69. Tip 10 10) Don’t “SELECT COUNT(*) FROM TABLE” on InnoDB.‣ COUNT(*) => MYISAM = FAST‣ COUNT(*) => INNODB = SLOW 28
  70. 70. Tip 10: SELECT COUNT(*) (1) 29
  71. 71. Tip 10: SELECT COUNT(*) (1) ‣ InnoDB implements MVCC (multi- version concurrency control). 29
  72. 72. Tip 10: SELECT COUNT(*) (1) ‣ InnoDB implements MVCC (multi- version concurrency control). ‣ COUNT(*) must be counted and is not fetched from metadata. 29
  73. 73. Tip 10: SELECT COUNT(*) (1) 30
  74. 74. Tip 10: SELECT COUNT(*) (1) ‣ What do you want to COUNT(*)? 30
  75. 75. Tip 10: SELECT COUNT(*) (1) ‣ What do you want to COUNT(*)? ‣ Just for displaying purposes (there are X amount of pages): do you need the EXACT amount? 30
  76. 76. Tip 11 11) Don’t rely on the VARCHAR()‣ IT ISN’T THAT VARIABLE AS YOU MIGHT THINK 31
  77. 77. Tip 11: VARCHAR() (1)‣ IT DOESN’T SOLVE YOUR PROBLEMS 32
  78. 78. Tip 11: VARCHAR() (3)‣ DON’T WORRY ABOUT THE UTF-8, I’LL BASH THAT LATER 33
  79. 79. Tip 11: VARCHAR() (3)‣ DON’T WORRY ABOUT THE UTF-8, I’LL BASH THAT LATER 33
  80. 80. Tip 12 12) UTF-8 is not the enemy, but it certainly isn’t your friend.‣ DON’T EXCHANGE ONE PROBLEM FOR ANOTHER 34
  81. 81. Tip 12: UTF-8 (1) 35
  82. 82. Tip 12: UTF-8 (1) ‣ Dr Jeckyl and Mr Hyde 35
  83. 83. Tip 12: UTF-8 (1) ‣ Dr Jeckyl and Mr Hyde ‣ Solves all your multi-language problems! 35
  84. 84. Tip 12: UTF-8 (1) ‣ Dr Jeckyl and Mr Hyde ‣ Solves all your multi-language problems! ‣ But gives back performance issues. 35
  85. 85. Tip 12: UTF-8 (1) 36
  86. 86. Tip 12: UTF-8 (1) 37
  87. 87. Tip 12: UTF-8 (1) 37
  88. 88. Tip 12: UTF-8 (1) ALL temporary buffers are allocated for worst-case scenario’s. This means a varchar(255) in UTF-8 uses 255*3 + 2 = 767 bytes PER row, even if you have only 1 single char inside. 38
  89. 89. Tip 13 13) Know your cardinality & selectivity‣ WHY LOOKUP DATA WHEN YOU ALREADY HAVE IT? 39
  90. 90. Tip 13: Cardinality & Selectivity (1) 40
  91. 91. Tip 13: Cardinality & Selectivity (1) ‣ Cardinality: the number of unique entries inside the index. 40
  92. 92. Tip 13: Cardinality & Selectivity (1) ‣ Cardinality: the number of unique entries inside the index. ‣ Selectivity: percentage of unique entries. 40
  93. 93. Tip 13: Cardinality & Selectivity (1) ‣ Cardinality: the number of unique entries inside the index. ‣ Selectivity: percentage of unique entries. ‣ S(I) = cardinality / count * 100% 40
  94. 94. Tip 13: Cardinality & Selectivity (2) 41
  95. 95. Tip 13: Cardinality & Selectivity (3)country_id (max +-200, but effectively +- 50, maybe less) 42
  96. 96. Tip 13: Cardinality & Selectivity (3)country_id (max +-200, but effectively +- 50, maybe less) ‣ with 10 records: 5/10 * 100% = 50% 42
  97. 97. Tip 13: Cardinality & Selectivity (3)country_id (max +-200, but effectively +- 50, maybe less) ‣ with 10 records: 5/10 * 100% = 50% ‣ with 1000 records: 75/1000 * 100 = 7.5% 42
  98. 98. Tip 13: Cardinality & Selectivity (3)country_id (max +-200, but effectively +- 50, maybe less) ‣ with 10 records: 5/10 * 100% = 50% ‣ with 1000 records: 75/1000 * 100 = 7.5% ‣ with 10.000 records: 200/10000 * 100% = 2% 42
  99. 99. Tip 13: Cardinality & Selectivity (4) ‣ A selectivity < 30% ? Full table scan! ‣ ANALYZE TABLE frequently. 43
  100. 100. Tip 13: Cardinality & Selectivity (5) 44
  101. 101. Tip 13: Cardinality & Selectivity (5) ‣ Adding records changes your cardinality and selectivity. 44
  102. 102. Tip 13: Cardinality & Selectivity (5) ‣ Adding records changes your cardinality and selectivity. ‣ Develop against a “real” dataset (10K records instead of 10 for instance). 44
  103. 103. Tip 14 14) Non-deterministic functions do not go well with query caching‣ NOW(), RAND(), UUID(), CONNECTION_ID() ETC.. 45
  104. 104. Tip 14: Query caching (1) 46
  105. 105. Tip 14: Query caching (1) 46
  106. 106. Tip 14: Query caching (2) 47
  107. 107. Tip 14: Query caching (2) 47
  108. 108. Tip 14: Query caching (3) 48
  109. 109. Tip 14: Query caching (3) 48
  110. 110. Tip 14: Query caching (4)SELECT * FROM table WHERE YEAR(created_dt) < YEAR(NOW());vsSELECT * FROM table WHERE YEAR(created_dt) < ‘2010’; 49
  111. 111. Tip 15 15) Certify yourself as a DBA and/or DBE.‣ AND GET SOME NICE TITLES WHILE YOU’RE AT IT... 50
  112. 112. Tip 15: Certify yourself (1) ‣ Oracle Certified MySQL Associate ‣ Oracle Certified Professional MySQL 5.0 Developer ‣ Oracle Certified Professional MySQL 5.0 Database Administrator ‣ Oracle Certified Expert, MySQL 5.1 Cluster Database Administrator. ‣ Get them all!‣ THEY ARE NOT EASY EXAMS, BUT WELL WORTH IT 51
  113. 113. Let’s summarize (1) Know how to use explain. (9) Don’t use a large Primary Key for InnoDB tables. (2) Know the most basic my.cnf settings. (10) Don’t “Select COUNT(*)” on InnoDB. (3) Backup on table level. (11) Don’t rely on the VARCHAR(). (4) Don’t use “SELECT *” when you only need 1 or 2 fields. (12) UTF-8 is not the enemy, but it certainly isn’t your friend. (5) Use triggers and stored procedures. (13) Know your cardinality & selectivity. (6) Don’t use FULLTEXT searches. (14) Non-deterministic functions do not (7) Wildcard searches (%item%) are bad for go well with query caching. performance. (15) Certify yourself as a DBA and/or (8) Shard your volatile and non-volatile DBE. data. 52
  114. 114. Shameless plug ‣ Enrise MySQL training/workshop ‣ Day of training into basics/DBA/DBE ‣ When, how much, what? ‣ Depends on interests.
  115. 115. Any questions? ∂ QUESTIONS?http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg
  116. 116. ‣ THANK YOU FOR YOUR ATTENTION‣ Please rate my talk: http://joind.in/talk/view/2947 55

×