15 protips for mysql users
Upcoming SlideShare
Loading in...5
×
 

15 protips for mysql users

on

  • 1,286 views

15 tips taken from the field on how to achieve a better performance on your MySQL server.

15 tips taken from the field on how to achieve a better performance on your MySQL server.

Statistics

Views

Total Views
1,286
Views on SlideShare
1,286
Embed Views
0

Actions

Likes
2
Downloads
22
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

15 protips for mysql users 15 protips for mysql users Presentation Transcript

  • 15 Pro-tips forMySQL UsersPHPBenelux Meeting - March 2011Enrise HQ - Amersfoorthttp://joind.in/talk/view/2947 1
  • Who am I? 2
  • Who am I?Joshua Thijssen (32) 2
  • Who am I?Joshua Thijssen (32)Senior Software Engineer @ Enrise 2
  • Who am I?Joshua Thijssen (32)Senior Software Engineer @ EnriseDevelopment in PHP, Python, Perl,C, Java.... 2
  • Who am I?Joshua Thijssen (32)Senior Software Engineer @ EnriseDevelopment in PHP, Python, Perl,C, Java....Blogs: http://www.adayinthelifeof.nl http://www.enrise.com/blog 2
  • Who am I?Joshua Thijssen (32)Senior Software Engineer @ EnriseDevelopment in PHP, Python, Perl,C, Java....Blogs: http://www.adayinthelifeof.nl http://www.enrise.com/blogEmail: joshua@enrise.com 2
  • Who am I?Joshua Thijssen (32)Senior Software Engineer @ EnriseDevelopment in PHP, Python, Perl,C, Java....Blogs: http://www.adayinthelifeof.nl http://www.enrise.com/blogEmail: joshua@enrise.comTwitter: @jaytaphIdenti.ca: jaytaph 2
  • What are we going to discuss?‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD 3
  • What are we going to discuss? ‣ 15 MySQL Pro-tips‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD 3
  • What are we going to discuss? ‣ 15 MySQL Pro-tips ‣ No “theoretical tips”, all taken from the field.‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD 3
  • What are we going to discuss? ‣ 15 MySQL Pro-tips ‣ No “theoretical tips”, all taken from the field. ‣ Starting simple - ending “complex”‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD 3
  • Tip 1 1) Know how to use explain.‣ EXPLAIN IS YOUR BESTEST FRIEND 4
  • Tip 1: Know your EXPLAIN (1) 5
  • Tip 1: Know your EXPLAIN (1) ‣ I will not show you how to use EXPLAIN. 5
  • Tip 1: Know your EXPLAIN (1) ‣ I will not show you how to use EXPLAIN. ‣ Use EXPLAIN and EXPLAIN EXTENDED/ SHOW WARNINGS; 5
  • Tip 1: Know your EXPLAIN (1) 6
  • Tip 1: Know your EXPLAIN (1) 6
  • Tip 1: Know your EXPLAIN (1) 6
  • Tip 2 2) Know the most basic my.cnf settings‣ THERE ARE ONLY A FEW “BASIC” ONES. 7
  • Tip 2: My.cnf settings (1) Know the most important ones: key_buffer_size, innodb_buffer_pool_size, sort_buffer_size, max_connections 8
  • Tip 2: My.cnf settings (2) 9
  • Tip 2: My.cnf settings (2) ‣ Some settings work on global level, some per connection! 9
  • Tip 2: My.cnf settings (2) ‣ Some settings work on global level, some per connection! ‣ Know some quirks: (max_heap_table_size vs tmp_table_size, binlog- do-db, replicate-ignore-db etc) 9
  • Tip 2: My.cnf settings (3) ‣ http://www.omh.cc/mycnf/ ‣ http://rackerhacker.com/mysqltuner/ ‣ http://www.day32.com/MySQL/ ‣ phpmyadmin 10
  • Tip 3 3) Backup on table level‣ RESTORING JUST ONE TABLE CAN BE PAINFUL OTHERWISE 11
  • Tip 3: Backup on table level (1)‣ COULD YOU RESTORE TABLE x? YES! YES I CAN! 12
  • Tip 3: Backup on table level (1) ‣ mysqldump can dump per database OR by table.‣ COULD YOU RESTORE TABLE x? YES! YES I CAN! 12
  • Tip 3: Backup on table level (1) ‣ mysqldump can dump per database OR by table. ‣ Simple scripts to scan/dump tables.‣ COULD YOU RESTORE TABLE x? YES! YES I CAN! 12
  • Tip 3: Backup on table level (1) ‣ mysqldump can dump per database OR by table. ‣ Simple scripts to scan/dump tables. ‣ Easy restore for single table (or part of table)‣ COULD YOU RESTORE TABLE x? YES! YES I CAN! 12
  • Tip 4 4) Don’t use “SELECT *” when you only need 1 or 2 fields.‣ DON’T ASK WHAT YOU DON’T NEED 13
  • Tip 4: Select * (1)‣ DON’T ASK WHAT YOU DON’T NEED 14
  • Tip 4: Select * (1) ‣ Much more data to be read from disk‣ DON’T ASK WHAT YOU DON’T NEED 14
  • Tip 4: Select * (1) ‣ Much more data to be read from disk ‣ Much more data will be send over, thus slower (blobs/texts)‣ DON’T ASK WHAT YOU DON’T NEED 14
  • Tip 4: Select * (1) ‣ Much more data to be read from disk ‣ Much more data will be send over, thus slower (blobs/texts) ‣ Cannot use covering indices‣ DON’T ASK WHAT YOU DON’T NEED 14
  • Tip 4: Select * (2) 15
  • Tip 4: Select * (2) 15
  • Tip 5 5) Use triggers and stored procedures‣ ENFORCE CONSISTENCY 16
  • Tip 5: Triggers and stored procedures (1)‣ ENFORCE CONSISTENCY 17
  • Tip 5: Triggers and stored procedures (1) ‣ 6 triggers per table (insert, update, delete, before and after the mutation)‣ ENFORCE CONSISTENCY 17
  • Tip 5: Triggers and stored procedures (1) ‣ 6 triggers per table (insert, update, delete, before and after the mutation) ‣ 3rd party tools (phpmyadmin etc) can also use the database without loosing data consistency.‣ ENFORCE CONSISTENCY 17
  • Tip 5: Triggers and stored procedures (1) ‣ 6 triggers per table (insert, update, delete, before and after the mutation) ‣ 3rd party tools (phpmyadmin etc) can also use the database without loosing data consistency. ‣ Watch out with (phpmyadmin) table dumps!‣ ENFORCE CONSISTENCY 17
  • Tip 6 6) Don’t use FULLTEXT searches‣ THERE ARE MUCH BETTER SOLUTIONS 18
  • Tip 6: Don’t use FULLTEXT search (1)‣ THERE ARE MUCH BETTER SOLUTIONS 19
  • Tip 6: Don’t use FULLTEXT search (1)‣ They only work for MyISAM tables.‣ THERE ARE MUCH BETTER SOLUTIONS 19
  • Tip 6: Don’t use FULLTEXT search (1)‣ They only work for MyISAM tables.‣ Not compatible with other DB’s.‣ THERE ARE MUCH BETTER SOLUTIONS 19
  • Tip 6: Don’t use FULLTEXT search (1)‣ They only work for MyISAM tables.‣ Not compatible with other DB’s.‣ Slow (especially compared to Solr, Sphinx).‣ THERE ARE MUCH BETTER SOLUTIONS 19
  • Tip 6: Don’t use FULLTEXT search (1)‣ They only work for MyISAM tables.‣ Not compatible with other DB’s.‣ Slow (especially compared to Solr, Sphinx).‣ No extra features (faceted search, spell checking etc).‣ THERE ARE MUCH BETTER SOLUTIONS 19
  • Tip 7 7) Wildcard searches (%item%) are bad for performance‣ IT LOOKS LIKE YOU NEED MORE ADVANCED 20
  • Tip 7: Wildcard searches (1)‣ THERE ARE MUCH BETTER SOLUTIONS 21
  • Tip 7: Wildcard searches (1) ‣ MySQL cannot use indexes!‣ THERE ARE MUCH BETTER SOLUTIONS 21
  • Tip 7: Wildcard searches (1) ‣ MySQL cannot use indexes! ‣ Revert your data: search for ‘moc.esirne@%’ instead of ‘%@enrise.com’.‣ THERE ARE MUCH BETTER SOLUTIONS 21
  • Tip 7: Wildcard searches (1) ‣ MySQL cannot use indexes! ‣ Revert your data: search for ‘moc.esirne@%’ instead of ‘%@enrise.com’. ‣ Use a better solution (solr, sphinx). You probably want it.‣ THERE ARE MUCH BETTER SOLUTIONS 21
  • Tip 8 8) Shard your volatile and non- volatile data.‣ MAKE CACHING AND LOCKING HAPPY AGAIN 22
  • Tip 8: Sharding (1) 23
  • Tip 8: Sharding (2) 24
  • Tip 8: Sharding (2) ‣ Remember: an update on a table will invalidate ALL queries referring to that table. 24
  • Tip 8: Sharding (2) ‣ Remember: an update on a table will invalidate ALL queries referring to that table. ‣ UPDATE pages SET hit_count = hit_count + 1; 24
  • Tip 8: Sharding (2) ‣ Remember: an update on a table will invalidate ALL queries referring to that table. ‣ UPDATE pages SET hit_count = hit_count + 1; ‣ Thus: page table will NEVER be cached. 24
  • Tip 8: Sharding (3) 25
  • Tip 8: Sharding (3) ‣ Define hot data (volatile, changes often) and cold data (static, changes never or infrequently) 25
  • Tip 8: Sharding (3) ‣ Define hot data (volatile, changes often) and cold data (static, changes never or infrequently) ‣ move to different tables 25
  • Tip 8: Sharding (3) ‣ Define hot data (volatile, changes often) and cold data (static, changes never or infrequently) ‣ move to different tables ‣ UPDATE page_stats SET hit_count = hit_count + 1; 25
  • Tip 8: Sharding (3) ‣ Define hot data (volatile, changes often) and cold data (static, changes never or infrequently) ‣ move to different tables ‣ UPDATE page_stats SET hit_count = hit_count + 1; ‣ Query cache is happy again 25
  • Tip 9 9) Don’t use a large Primary Key for InnoDB tables.‣ PK’S ARE ON EVERY INDEX 26
  • Tip 9: Large primary keys (1) 27
  • Tip 9: Large primary keys (1) ‣ InnoDB adds the primary key to EACH index. 27
  • Tip 9: Large primary keys (1) ‣ InnoDB adds the primary key to EACH index. ‣ No primary key given? It uses an internal 6(!)-byte key. 27
  • Tip 10 10) Don’t “SELECT COUNT(*) FROM TABLE” on InnoDB.‣ COUNT(*) => MYISAM = FAST‣ COUNT(*) => INNODB = SLOW 28
  • Tip 10: SELECT COUNT(*) (1) 29
  • Tip 10: SELECT COUNT(*) (1) ‣ InnoDB implements MVCC (multi- version concurrency control). 29
  • Tip 10: SELECT COUNT(*) (1) ‣ InnoDB implements MVCC (multi- version concurrency control). ‣ COUNT(*) must be counted and is not fetched from metadata. 29
  • Tip 10: SELECT COUNT(*) (1) 30
  • Tip 10: SELECT COUNT(*) (1) ‣ What do you want to COUNT(*)? 30
  • Tip 10: SELECT COUNT(*) (1) ‣ What do you want to COUNT(*)? ‣ Just for displaying purposes (there are X amount of pages): do you need the EXACT amount? 30
  • Tip 11 11) Don’t rely on the VARCHAR()‣ IT ISN’T THAT VARIABLE AS YOU MIGHT THINK 31
  • Tip 11: VARCHAR() (1)‣ IT DOESN’T SOLVE YOUR PROBLEMS 32
  • Tip 11: VARCHAR() (3)‣ DON’T WORRY ABOUT THE UTF-8, I’LL BASH THAT LATER 33
  • Tip 11: VARCHAR() (3)‣ DON’T WORRY ABOUT THE UTF-8, I’LL BASH THAT LATER 33
  • Tip 12 12) UTF-8 is not the enemy, but it certainly isn’t your friend.‣ DON’T EXCHANGE ONE PROBLEM FOR ANOTHER 34
  • Tip 12: UTF-8 (1) 35
  • Tip 12: UTF-8 (1) ‣ Dr Jeckyl and Mr Hyde 35
  • Tip 12: UTF-8 (1) ‣ Dr Jeckyl and Mr Hyde ‣ Solves all your multi-language problems! 35
  • Tip 12: UTF-8 (1) ‣ Dr Jeckyl and Mr Hyde ‣ Solves all your multi-language problems! ‣ But gives back performance issues. 35
  • Tip 12: UTF-8 (1) 36
  • Tip 12: UTF-8 (1) 37
  • Tip 12: UTF-8 (1) 37
  • Tip 12: UTF-8 (1) ALL temporary buffers are allocated for worst-case scenario’s. This means a varchar(255) in UTF-8 uses 255*3 + 2 = 767 bytes PER row, even if you have only 1 single char inside. 38
  • Tip 13 13) Know your cardinality & selectivity‣ WHY LOOKUP DATA WHEN YOU ALREADY HAVE IT? 39
  • Tip 13: Cardinality & Selectivity (1) 40
  • Tip 13: Cardinality & Selectivity (1) ‣ Cardinality: the number of unique entries inside the index. 40
  • Tip 13: Cardinality & Selectivity (1) ‣ Cardinality: the number of unique entries inside the index. ‣ Selectivity: percentage of unique entries. 40
  • Tip 13: Cardinality & Selectivity (1) ‣ Cardinality: the number of unique entries inside the index. ‣ Selectivity: percentage of unique entries. ‣ S(I) = cardinality / count * 100% 40
  • Tip 13: Cardinality & Selectivity (2) 41
  • Tip 13: Cardinality & Selectivity (3)country_id (max +-200, but effectively +- 50, maybe less) 42
  • Tip 13: Cardinality & Selectivity (3)country_id (max +-200, but effectively +- 50, maybe less) ‣ with 10 records: 5/10 * 100% = 50% 42
  • Tip 13: Cardinality & Selectivity (3)country_id (max +-200, but effectively +- 50, maybe less) ‣ with 10 records: 5/10 * 100% = 50% ‣ with 1000 records: 75/1000 * 100 = 7.5% 42
  • Tip 13: Cardinality & Selectivity (3)country_id (max +-200, but effectively +- 50, maybe less) ‣ with 10 records: 5/10 * 100% = 50% ‣ with 1000 records: 75/1000 * 100 = 7.5% ‣ with 10.000 records: 200/10000 * 100% = 2% 42
  • Tip 13: Cardinality & Selectivity (4) ‣ A selectivity < 30% ? Full table scan! ‣ ANALYZE TABLE frequently. 43
  • Tip 13: Cardinality & Selectivity (5) 44
  • Tip 13: Cardinality & Selectivity (5) ‣ Adding records changes your cardinality and selectivity. 44
  • Tip 13: Cardinality & Selectivity (5) ‣ Adding records changes your cardinality and selectivity. ‣ Develop against a “real” dataset (10K records instead of 10 for instance). 44
  • Tip 14 14) Non-deterministic functions do not go well with query caching‣ NOW(), RAND(), UUID(), CONNECTION_ID() ETC.. 45
  • Tip 14: Query caching (1) 46
  • Tip 14: Query caching (1) 46
  • Tip 14: Query caching (2) 47
  • Tip 14: Query caching (2) 47
  • Tip 14: Query caching (3) 48
  • Tip 14: Query caching (3) 48
  • Tip 14: Query caching (4)SELECT * FROM table WHERE YEAR(created_dt) < YEAR(NOW());vsSELECT * FROM table WHERE YEAR(created_dt) < ‘2010’; 49
  • Tip 15 15) Certify yourself as a DBA and/or DBE.‣ AND GET SOME NICE TITLES WHILE YOU’RE AT IT... 50
  • Tip 15: Certify yourself (1) ‣ Oracle Certified MySQL Associate ‣ Oracle Certified Professional MySQL 5.0 Developer ‣ Oracle Certified Professional MySQL 5.0 Database Administrator ‣ Oracle Certified Expert, MySQL 5.1 Cluster Database Administrator. ‣ Get them all!‣ THEY ARE NOT EASY EXAMS, BUT WELL WORTH IT 51
  • Let’s summarize (1) Know how to use explain. (9) Don’t use a large Primary Key for InnoDB tables. (2) Know the most basic my.cnf settings. (10) Don’t “Select COUNT(*)” on InnoDB. (3) Backup on table level. (11) Don’t rely on the VARCHAR(). (4) Don’t use “SELECT *” when you only need 1 or 2 fields. (12) UTF-8 is not the enemy, but it certainly isn’t your friend. (5) Use triggers and stored procedures. (13) Know your cardinality & selectivity. (6) Don’t use FULLTEXT searches. (14) Non-deterministic functions do not (7) Wildcard searches (%item%) are bad for go well with query caching. performance. (15) Certify yourself as a DBA and/or (8) Shard your volatile and non-volatile DBE. data. 52
  • Shameless plug ‣ Enrise MySQL training/workshop ‣ Day of training into basics/DBA/DBE ‣ When, how much, what? ‣ Depends on interests.
  • Any questions? ∂ QUESTIONS?http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg
  • ‣ THANK YOU FOR YOUR ATTENTION‣ Please rate my talk: http://joind.in/talk/view/2947 55