Your SlideShare is downloading. ×
0
15 Pro-tips for MySQL Users PFCongress - 17 september 2011 Utrecht - Netherlands
Joshua Thijssen Senior Software Engineer @ Enrise Development in PHP, Python, Perl, C, Java.... Email:  [email_address] Tw...
<ul><li>15 MySQL Pro-tips </li></ul><ul><li>No “theoretical tips”, all taken from the field. </li></ul><ul><li>Starting si...
<ul><li>0) Use the correct MySQL version </li></ul>Tip  0 <ul><li>MySQL 5.0 != 5.5 </li></ul>
<ul><li>RHEL 5-7: 5.0.77 </li></ul><ul><li>RHEL 6-1: 5.1.52  </li></ul><ul><li>Debian (lenny) 5.0.51a </li></ul><ul><li>De...
<ul><li>1) Know how to use explain (and profiler) </li></ul>Tip  1 <ul><li>EXPLAIN IS YOUR BESTEST FRIEND </li></ul>
<ul><li>I will not show you how to use EXPLAIN. </li></ul><ul><li>Use EXPLAIN and EXPLAIN EXTENDED/  SHOW WARNINGS; </li><...
Tip 1:   Know your EXPLAIN  (2) mysql> desc varchartest;+----------+--------------+------+-----+---------+-------+| Field ...
Tip 1:   Know your EXPLAIN  (3) mysql> EXPLAIN EXTENDED  ->  SELECT * FROM varchartest WHERE name LIKE 'joshua';+----+----...
Tip 1:   Know your EXPLAIN  (4) mysql> SHOW WARNINGSg+-------+------+-----------------------------------------------------...
Tip 1:   Know your PROFILER  (1) mysql> SET profiling=1;mysql> SELECT * FROM table; mysql> SHOW PROFILE CPU, BLOCK IO; +--...
<ul><li>2) Know the most basic my.cnf settings </li></ul>Tip  2 <ul><li>THERE ARE ONLY A FEW “BASIC” ONES. </li></ul>
Tip 2: My.cnf settings  (1) Know the most important ones: key_buffer_size, innodb_buffer_pool_size,  sort_buffer_size, max...
<ul><li>Some settings work on global level, some per connection! </li></ul><ul><li>Know some quirks:  (max_heap_table_size...
<ul><li>http://www.omh.cc/mycnf/ </li></ul><ul><li>http://rackerhacker.com/mysqltuner/ </li></ul><ul><li>http://www.day32....
<ul><li>3) Backup on table level </li></ul>Tip  3 <ul><li>RESTORING JUST ONE TABLE CAN BE PAINFUL OTHERWISE </li></ul>
<ul><li>mysqldump can dump per database OR by table. </li></ul><ul><li>Simple scripts to scan/dump tables. </li></ul><ul><...
<ul><li>4) Don’t use “SELECT *” when you only need one or two fields. </li></ul>Tip  4 <ul><li>DON’T ASK WHAT YOU DON’T NE...
<ul><li>Much more data to be read from disk </li></ul><ul><li>Much more data will be send over, thus slower (blobs/texts) ...
Tip 4:   Select *  (2) mysql> SHOW FULL COLUMNS FROM `covering`;+--------------+------------------+------------------+----...
Tip 4:   Select *  (2) mysql> EXPLAIN SELECT want_mailing FROM `covering` WHERE email LIKE ' [email_address] ';+----+-----...
<ul><li>5) Use triggers and stored procedures </li></ul>Tip  5 <ul><li>ENFORCE CONSISTENCY </li></ul>
<ul><li>6 triggers per table (insert, update, delete, before and after the mutation) </li></ul><ul><li>3rd party tools (ph...
<ul><li>6) Don’t use FULLTEXT searches </li></ul>Tip  6 <ul><li>THERE ARE MUCH BETTER SOLUTIONS </li></ul>
<ul><li>They only work for MyISAM tables. </li></ul><ul><li>Not compatible with other DB’s. </li></ul><ul><li>Slow (especi...
<ul><li>7) Some wildcard searches (%item%) are bad for performance </li></ul>Tip  7 <ul><li>IT LOOKS LIKE YOU NEED MORE AD...
<ul><li>MySQL cannot use indexing! </li></ul><ul><li>Revert your data:  search for ‘moc.esirne@%’ instead of ‘%@enrise.com...
<ul><li>8) Shard your volatile and non-volatile data. </li></ul>Tip  8 <ul><li>MAKE CACHING AND LOCKING HAPPY AGAIN </li><...
Tip 8:   Sharding  (1) mysql> SHOW FULL COLUMNS FROM `pages`;+------------+------------------+-------------------+------+-...
<ul><li>Remember: an update on a table will invalidate ALL caches referring to that table. </li></ul><ul><li>UPDATE pages ...
<ul><li>Define hot data (volatile, changes often) and cold data (static, changes never or infrequently) </li></ul><ul><li>...
<ul><li>MySQL 5.1 and higher has got partitioning (but you should shard anyway). </li></ul>Tip 8:   Sharding  (4)
<ul><li>9) Don’t use a large primary key for InnoDB tables. </li></ul>Tip  9 <ul><li>PK’S ARE ON EVERY INDEX </li></ul>
<ul><li>InnoDB adds the primary key to EACH index. </li></ul><ul><li>No primary key given? It uses an internal 6(!)-byte k...
<ul><li>10) Don’t “SELECT COUNT(*) FROM TABLE” on InnoDB. </li></ul>Tip  10 <ul><li>COUNT(*) => MYISAM = FAST </li></ul><u...
<ul><li>InnoDB implements MVCC (multi-version concurrency control). </li></ul><ul><li>COUNT(*) must be counted and is not ...
<ul><li>What do you want to COUNT(*)? </li></ul><ul><li>Just for displaying purposes (there are X amount of pages): do you...
<ul><li>11) Don’t rely on the VARCHAR() </li></ul>Tip  11 <ul><li>IT ISN’T THAT VARIABLE AS YOU MIGHT THINK </li></ul>
Tip 11:   VARCHAR()  (1) <ul><li>IT DOESN’T SOLVE YOUR PROBLEMS </li></ul>
<ul><li>DON’T WORRY ABOUT THE UTF-8, I’LL BASH THAT LATER </li></ul>Tip 11:   VARCHAR()  (3) mysql> SHOW FULL COLUMNS FROM...
<ul><li>DON’T WORRY ABOUT THE UTF-8, I’LL BASH THAT LATER </li></ul>Tip 11:   VARCHAR()  (3) mysql> EXPLAIN SELECT * FROM ...
<ul><li>12) UTF-8 is not the enemy, but it certainly isn’t your friend. </li></ul>Tip  12 <ul><li>DON’T EXCHANGE ONE PROBL...
<ul><li>Dr Jeckyl and Mr Hyde </li></ul><ul><li>Solves all your multi-language problems! </li></ul><ul><li>But gives back ...
Tip 12:   UTF-8  (1) mysql> SHOW FULL COLUMNS FROM `varchartest`;+----------+--------------+-------------------+------+---...
Tip 12:   UTF-8  (1) mysql> EXPLAIN SELECT * FROM `varchartest` WHERE name LIKE 'jthijssen';+----+-------------+----------...
<ul><li>ALL temporary buffers are allocated for worst-case scenario’s.  </li></ul><ul><li>This means a varchar(255) in UTF...
<ul><li>13) Know your cardinality & selectivity </li></ul>Tip  13
<ul><li>Cardinality: the number of unique entries inside the index. </li></ul><ul><li>Selectivity: percentage of unique en...
Tip 13:   Cardinality & Selectivity  (2)
<ul><li>with 10 records: 5/10 * 100% = 50% </li></ul><ul><li>with 1000 records: 75/1000 * 100 = 7.5% </li></ul><ul><li>wit...
<ul><li>A selectivity < 30% ? Full table scan! </li></ul><ul><li>ANALYZE TABLE frequently. </li></ul>Tip 13:   Cardinality...
<ul><li>Adding records changes your cardinality and thus selectivity.  </li></ul><ul><li>Develop against a “real” dataset....
<ul><li>14) Non-deterministic functions do not go well with query caching  </li></ul>Tip  14 <ul><li>NOW(), RAND(), UUID()...
Tip 14:   Query caching  (1) mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name  | V...
Tip 14:   Query caching  (2) mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------...
Tip 14:   Query caching  (3) mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------...
SELECT * FROM table WHERE YEAR(created_dt) < YEAR(NOW()); vs SELECT * FROM table WHERE YEAR(created_dt) < ‘2010’; Tip 14: ...
<ul><li>15) Certify yourself as a DBA and/or DBE. </li></ul>Tip  15 <ul><li>AND GET SOME NICE TITLES WHILE YOU’RE AT IT......
<ul><li>Oracle Certified MySQL Associate  </li></ul><ul><li>Oracle Certified Professional MySQL 5.0 Developer </li></ul><u...
Let’s  summarize <ul><li>Know how to use explain. </li></ul><ul><li>Know the most basic my.cnf settings. </li></ul><ul><li...
Any  questions? <ul><li>QUESTIONS? </li></ul>http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg
Daycamp for developers http://www.enrise.com http://www.daycamp4developers.com /
<ul><li>THANK YOU FOR YOUR ATTENTION </li></ul><ul><li>Please rate my talk:  http://joind.in/3662 </li></ul>
Upcoming SlideShare
Loading in...5
×

15 protips for mysql users pfz

2,879

Published on

Published in: Technology

Transcript of "15 protips for mysql users pfz"

  1. 1. 15 Pro-tips for MySQL Users PFCongress - 17 september 2011 Utrecht - Netherlands
  2. 2. Joshua Thijssen Senior Software Engineer @ Enrise Development in PHP, Python, Perl, C, Java.... Email: [email_address] Twitter: @jaytaph Blogs: http://www.adayinthelifeof.nl http://www.enrise.com/blog
  3. 3. <ul><li>15 MySQL Pro-tips </li></ul><ul><li>No “theoretical tips”, all taken from the field. </li></ul><ul><li>Starting simple - ending “complex” </li></ul>What are we going to discuss? <ul><li>QUESTIONS? RAISE YOUR HAND OR YELL LOUD </li></ul>
  4. 4. <ul><li>0) Use the correct MySQL version </li></ul>Tip 0 <ul><li>MySQL 5.0 != 5.5 </li></ul>
  5. 5. <ul><li>RHEL 5-7: 5.0.77 </li></ul><ul><li>RHEL 6-1: 5.1.52 </li></ul><ul><li>Debian (lenny) 5.0.51a </li></ul><ul><li>Debian (squeeze): 5.1.49 </li></ul><ul><li>Debian (sid): 5.1.58 </li></ul>Tip 0: Use the correct MySQL version http://distrowatch.com/table.php?distribution=redhat http://distrowatch.com/table.php?distribution=debian
  6. 6. <ul><li>1) Know how to use explain (and profiler) </li></ul>Tip 1 <ul><li>EXPLAIN IS YOUR BESTEST FRIEND </li></ul>
  7. 7. <ul><li>I will not show you how to use EXPLAIN. </li></ul><ul><li>Use EXPLAIN and EXPLAIN EXTENDED/ SHOW WARNINGS; </li></ul>Tip 1: Know your EXPLAIN (1)
  8. 8. Tip 1: Know your EXPLAIN (2) mysql> desc varchartest;+----------+--------------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+----------+--------------+------+-----+---------+-------+| id | int(11) | NO | PRI | 0 | || name | varchar(255) | NO | MUL | NULL | || utf8name | varchar(255) | NO | MUL | NULL | |+----------+--------------+------+-----+---------+-------+3 rows in set (0.01 sec)
  9. 9. Tip 1: Know your EXPLAIN (3) mysql> EXPLAIN EXTENDED -> SELECT * FROM varchartest WHERE name LIKE 'joshua';+----+-------------+-------------+-------+---------------+----------+---------+------+------+----------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |+----+-------------+-------------+-------+---------------+----------+---------+------+------+----------+-------------+| 1 | SIMPLE | varchartest | range | idx_name | idx_name | 257 | NULL | 1 | 100.00 | Using where |+----+-------------+-------------+-------+---------------+----------+---------+------+------+----------+-------------+1 row in set, 1 warning (0.01 sec)
  10. 10. Tip 1: Know your EXPLAIN (4) mysql> SHOW WARNINGSg+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Level | Code | Message |+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Note | 1003 | select `pfz`.`varchartest`.`id` AS `id`,`pfz`.`varchartest`.`name` AS `name`,`pfz`.`varchartest`.`utf8name` AS `utf8name` from `pfz`.`varchartest` where (`pfz`.`varchartest`.`name` like 'joshua') |+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  11. 11. Tip 1: Know your PROFILER (1) mysql> SET profiling=1;mysql> SELECT * FROM table; mysql> SHOW PROFILE CPU, BLOCK IO; +--------------------------------+----------+----------+------------+--------------+---------------+| Status | Duration | CPU_user | CPU_system | Block_ops_in | Block_ops_out |+--------------------------------+----------+----------+------------+--------------+---------------+| starting | 0.000930 | 0.000000 | 0.000000 | 0 | 8 || checking query cache for query | 0.000547 | 0.000000 | 0.000000 | 0 | 0 || checking permissions | 0.000045 | 0.000000 | 0.000000 | 0 | 0 || Opening tables | 0.000142 | 0.000000 | 0.000000 | 0 | 0 || System lock | 0.000030 | 0.000000 | 0.000000 | 0 | 0 || Table lock | 0.000045 | 0.000000 | 0.000000 | 0 | 0 || init | 0.000207 | 0.000000 | 0.000000 | 0 | 0 || optimizing | 0.000007 | 0.000000 | 0.000000 | 0 | 0 || statistics | 0.000068 | 0.000000 | 0.000000 | 0 | 0 || preparing | 0.001393 | 0.004000 | 0.000000 | 0 | 0 || Creating tmp table | 0.001658 | 0.000000 | 0.000000 | 0 | 8 || executing | 0.000005 | 0.000000 | 0.000000 | 0 | 0 || Copying to tmp table | 0.000834 | 0.000000 | 0.004001 | 0 | 8 || Sorting result | 0.000179 | 0.000000 | 0.000000 | 0 | 0 || Sending data | 0.000089 | 0.000000 | 0.000000 | 0 | 0 || end | 0.000022 | 0.000000 | 0.000000 | 0 | 0 || removing tmp table | 0.000089 | 0.000000 | 0.000000 | 0 | 0 || end | 0.000005 | 0.000000 | 0.000000 | 0 | 0 || query end | 0.000004 | 0.000000 | 0.000000 | 0 | 0 || freeing items | 0.000844 | 0.000000 | 0.000000 | 0 | 0 || logging slow query | 0.000023 | 0.000000 | 0.000000 | 0 | 0 || logging slow query | 0.000060 | 0.000000 | 0.000000 | 0 | 8 || cleaning up | 0.000007 | 0.000000 | 0.000000 | 0 | 0 |+--------------------------------+----------+----------+------------+--------------+---------------+
  12. 12. <ul><li>2) Know the most basic my.cnf settings </li></ul>Tip 2 <ul><li>THERE ARE ONLY A FEW “BASIC” ONES. </li></ul>
  13. 13. Tip 2: My.cnf settings (1) Know the most important ones: key_buffer_size, innodb_buffer_pool_size, sort_buffer_size, max_connections
  14. 14. <ul><li>Some settings work on global level, some per connection! </li></ul><ul><li>Know some quirks: (max_heap_table_size vs tmp_table_size, binlog-do-db, replicate-ignore-db etc) </li></ul>Tip 2: My.cnf settings (2)
  15. 15. <ul><li>http://www.omh.cc/mycnf/ </li></ul><ul><li>http://rackerhacker.com/mysqltuner/ </li></ul><ul><li>http://www.day32.com/MySQL/ </li></ul><ul><li>phpmyadmin </li></ul>Tip 2: My.cnf settings (3)
  16. 16. <ul><li>3) Backup on table level </li></ul>Tip 3 <ul><li>RESTORING JUST ONE TABLE CAN BE PAINFUL OTHERWISE </li></ul>
  17. 17. <ul><li>mysqldump can dump per database OR by table. </li></ul><ul><li>Simple scripts to scan/dump tables. </li></ul><ul><li>Easy restore for single table (or part of table) </li></ul>Tip 3: Backup on table level (1) <ul><li>COULD YOU RESTORE TABLE x? YES! YES I CAN! </li></ul>
  18. 18. <ul><li>4) Don’t use “SELECT *” when you only need one or two fields. </li></ul>Tip 4 <ul><li>DON’T ASK WHAT YOU DON’T NEED </li></ul>
  19. 19. <ul><li>Much more data to be read from disk </li></ul><ul><li>Much more data will be send over, thus slower (blobs/texts) </li></ul><ul><li>Cannot use covering indices </li></ul>Tip 4: Select * (1) <ul><li>DON’T ASK WHAT YOU DON’T NEED </li></ul>
  20. 20. Tip 4: Select * (2) mysql> SHOW FULL COLUMNS FROM `covering`;+--------------+------------------+------------------+------+-----+---------+----------------+ | Field | Type | Collation | Null | Key | Default | Extra | +--------------+------------------+------------------+------+-----+---------+----------------+| id | int(10) unsigned | NULL | NO | PRI | NULL | auto_increment || email | varchar(255) | ascii_general_ci | NO | MUL | NULL | | | want_mailing | tinyint(1) | NULL | NO | MUL | NULL | | | extra_info | varchar(255) | ascii_general_ci | NO | MUL | NULL | |+--------------+------------------+------------------+------+-----+---------+----------------+3 rows in set (0.00 sec)mysql> SHOW INDEXES FROM `covering`;+----------+------------+------------+--------------+--------------+-----------+-------------+| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | +----------+------------+------------+--------------+--------------+-----------+-------------+| covering | 0 | PRIMARY | 1 | id | A | 3 || covering | 1 | idx_email | 1 | email | A | 3 || covering | 1 | idx_email | 2 | want_mailing | A | 3 || covering | 1 | idx_email2 | 1 | want_mailing | A | 1 || covering | 1 | idx_email2 | 2 | email | A | 3 |+----------+------------+------------+--------------+--------------+-----------+-------------+5 rows in set (0.01 sec)
  21. 21. Tip 4: Select * (2) mysql> EXPLAIN SELECT want_mailing FROM `covering` WHERE email LIKE ' [email_address] ';+----+-------------+----------+-------+---------------+-----------+---------+------+------+--------------------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+----------+-------+---------------+-----------+---------+------+------+--------------------------+| 1 | SIMPLE | covering | range | idx_email | idx_email | 257 | NULL | 1 | Using where; Using index |+----+-------------+----------+-------+---------------+-----------+---------+------+------+--------------------------+1 row in set (0.00 sec)
  22. 22. <ul><li>5) Use triggers and stored procedures </li></ul>Tip 5 <ul><li>ENFORCE CONSISTENCY </li></ul>
  23. 23. <ul><li>6 triggers per table (insert, update, delete, before and after the mutation) </li></ul><ul><li>3rd party tools (phpmyadmin etc) can also use the database without loosing data consistency. </li></ul><ul><li>Watch out with (phpmyadmin) table dumps! </li></ul>Tip 5: Triggers and stored procedures (1) <ul><li>ENFORCE CONSISTENCY </li></ul>
  24. 24. <ul><li>6) Don’t use FULLTEXT searches </li></ul>Tip 6 <ul><li>THERE ARE MUCH BETTER SOLUTIONS </li></ul>
  25. 25. <ul><li>They only work for MyISAM tables. </li></ul><ul><li>Not compatible with other DB’s. </li></ul><ul><li>Slow (especially compared to Solr, Sphinx). </li></ul><ul><li>No extra features (faceted search, spell checking etc). </li></ul>Tip 6: Don’t use FULLTEXT search (1) <ul><li>THERE ARE MUCH BETTER SOLUTIONS </li></ul>
  26. 26. <ul><li>7) Some wildcard searches (%item%) are bad for performance </li></ul>Tip 7 <ul><li>IT LOOKS LIKE YOU NEED MORE ADVANCED SEARCH? </li></ul>
  27. 27. <ul><li>MySQL cannot use indexing! </li></ul><ul><li>Revert your data: search for ‘moc.esirne@%’ instead of ‘%@enrise.com’. </li></ul><ul><li>Use a better solution (solr, sphinx). You probably want it (no, really) </li></ul>Tip 7: Wildcard searches (1) <ul><li>THERE ARE MUCH BETTER SOLUTIONS </li></ul>
  28. 28. <ul><li>8) Shard your volatile and non-volatile data. </li></ul>Tip 8 <ul><li>MAKE CACHING AND LOCKING HAPPY AGAIN </li></ul>
  29. 29. Tip 8: Sharding (1) mysql> SHOW FULL COLUMNS FROM `pages`;+------------+------------------+-------------------+------+-----+-------------------+-------+| Field | Type | Collation | Null | Key | Default | Extra |+------------+------------------+-------------------+------+-----+-------------------+-------+| page_id | int(10) unsigned | NULL | NO | PRI | NULL | || created_dt | timestamp | NULL | NO | | CURRENT_TIMESTAMP | || creator_id | int(11) | NULL | NO | | NULL | || title | varchar(100) | latin1_swedish_ci | NO | | NULL | || contents | text | latin1_swedish_ci | NO | | NULL | || hit_count | int(11) | NULL | NO | | 0 | |+------------+------------------+-------------------+------+-----+-------------------+-------+6 rows in set (0.01 sec)
  30. 30. <ul><li>Remember: an update on a table will invalidate ALL caches referring to that table. </li></ul><ul><li>UPDATE pages SET hit_count = hit_count + 1; </li></ul><ul><li>Thus: page table will NEVER be cached. </li></ul>Tip 8: Sharding (2)
  31. 31. <ul><li>Define hot data (volatile, changes often) and cold data (static, changes never or infrequently) </li></ul><ul><li>move to different tables </li></ul><ul><li>UPDATE page_stats SET hit_count = hit_count + 1; </li></ul><ul><li>Query cache is happy again </li></ul>Tip 8: Sharding (3)
  32. 32. <ul><li>MySQL 5.1 and higher has got partitioning (but you should shard anyway). </li></ul>Tip 8: Sharding (4)
  33. 33. <ul><li>9) Don’t use a large primary key for InnoDB tables. </li></ul>Tip 9 <ul><li>PK’S ARE ON EVERY INDEX </li></ul>
  34. 34. <ul><li>InnoDB adds the primary key to EACH index. </li></ul><ul><li>No primary key given? It uses an internal 6(!)-byte key. </li></ul>Tip 9: Large primary keys (1)
  35. 35. <ul><li>10) Don’t “SELECT COUNT(*) FROM TABLE” on InnoDB. </li></ul>Tip 10 <ul><li>COUNT(*) => MYISAM = FAST </li></ul><ul><li>COUNT(*) => INNODB = SLOW </li></ul>
  36. 36. <ul><li>InnoDB implements MVCC (multi-version concurrency control). </li></ul><ul><li>COUNT(*) must be counted and is not fetched from metadata. </li></ul>Tip 10: SELECT COUNT(*) (1)
  37. 37. <ul><li>What do you want to COUNT(*)? </li></ul><ul><li>Just for displaying purposes (there are X amount of pages): do you need the EXACT amount? (guess &| cache) </li></ul>Tip 10: SELECT COUNT(*) (2)
  38. 38. <ul><li>11) Don’t rely on the VARCHAR() </li></ul>Tip 11 <ul><li>IT ISN’T THAT VARIABLE AS YOU MIGHT THINK </li></ul>
  39. 39. Tip 11: VARCHAR() (1) <ul><li>IT DOESN’T SOLVE YOUR PROBLEMS </li></ul>
  40. 40. <ul><li>DON’T WORRY ABOUT THE UTF-8, I’LL BASH THAT LATER </li></ul>Tip 11: VARCHAR() (3) mysql> SHOW FULL COLUMNS FROM `varchartest`;+----------+--------------+-------------------+------+-----+---------+-------+| Field | Type | Collation | Null | Key | Default | Extra |+----------+--------------+-------------------+------+-----+---------+-------+| id | int(11) | NULL | NO | PRI | 0 | || name | varchar(255) | latin1_swedish_ci | NO | MUL | NULL | || utf8name | varchar(255) | utf8_general_ci | NO | MUL | NULL | |+----------+--------------+-------------------+------+-----+---------+-------+3 rows in set (0.01 sec)mysql> SHOW INDEXES FROM `varchartest`;+-------------+------------+--------------+--------------+-------------+-----------+-------------+| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality |+-------------+------------+--------------+--------------+-------------+-----------+-------------+| varchartest | 0 | PRIMARY | 1 | id | A | 3 || varchartest | 1 | idx_name | 1 | name | A | 3 || varchartest | 1 | idx_utf8name | 1 | utf8name | A | 3 |+-------------+------------+--------------+--------------+-------------+-----------+-------------+3 rows in set (0.00 sec)
  41. 41. <ul><li>DON’T WORRY ABOUT THE UTF-8, I’LL BASH THAT LATER </li></ul>Tip 11: VARCHAR() (3) mysql> EXPLAIN SELECT * FROM `varchartest` WHERE name LIKE 'jthijssen';+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+| 1 | SIMPLE | varchartest | range | idx_name | idx_name | 257 | NULL | 1 | Using where |+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+1 row in set (0.00 sec)
  42. 42. <ul><li>12) UTF-8 is not the enemy, but it certainly isn’t your friend. </li></ul>Tip 12 <ul><li>DON’T EXCHANGE ONE PROBLEM FOR ANOTHER </li></ul>
  43. 43. <ul><li>Dr Jeckyl and Mr Hyde </li></ul><ul><li>Solves all your multi-language problems! </li></ul><ul><li>But gives back performance issues. </li></ul>Tip 12: UTF-8 (1)
  44. 44. Tip 12: UTF-8 (1) mysql> SHOW FULL COLUMNS FROM `varchartest`;+----------+--------------+-------------------+------+-----+---------+-------+| Field | Type | Collation | Null | Key | Default | Extra |+----------+--------------+-------------------+------+-----+---------+-------+| id | int(11) | NULL | NO | PRI | 0 | || name | varchar(255) | latin1_swedish_ci | NO | MUL | NULL | || utf8name | varchar(255) | utf8_general_ci | NO | MUL | NULL | |+----------+--------------+-------------------+------+-----+---------+-------+3 rows in set (0.01 sec)mysql> SHOW INDEXES FROM `varchartest`;+-------------+------------+--------------+--------------+-------------+-----------+-------------+| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality |+-------------+------------+--------------+--------------+-------------+-----------+-------------+| varchartest | 0 | PRIMARY | 1 | id | A | 3 || varchartest | 1 | idx_name | 1 | name | A | 3 || varchartest | 1 | idx_utf8name | 1 | utf8name | A | 3 |+-------------+------------+--------------+--------------+-------------+-----------+-------------+3 rows in set (0.00 sec)
  45. 45. Tip 12: UTF-8 (1) mysql> EXPLAIN SELECT * FROM `varchartest` WHERE name LIKE 'jthijssen';+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+| 1 | SIMPLE | varchartest | range | idx_name | idx_name | 257 | NULL | 1 | Using where |+----+-------------+-------------+-------+---------------+----------+---------+------+------+-------------+1 row in set (0.00 sec) mysql> EXPLAIN SELECT * FROM `varchartest` WHERE utf8name LIKE 'jthijssen';+----+-------------+-------------+-------+---------------+--------------+---------+------+------+-------------+| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |+----+-------------+-------------+-------+---------------+--------------+---------+------+------+-------------+| 1 | SIMPLE | varchartest | range | idx_utf8name | idx_utf8name | 767 | NULL | 1 | Using where |+----+-------------+-------------+-------+---------------+--------------+---------+------+------+-------------+1 row in set (0.01 sec)
  46. 46. <ul><li>ALL temporary buffers are allocated for worst-case scenario’s. </li></ul><ul><li>This means a varchar(255) in UTF-8 uses 255*3 + 2 = 767 bytes PER row, even if you have only 1 single char inside. </li></ul>Tip 12: UTF-8 (1)
  47. 47. <ul><li>13) Know your cardinality & selectivity </li></ul>Tip 13
  48. 48. <ul><li>Cardinality: the number of unique entries inside the index. </li></ul><ul><li>Selectivity: percentage of unique entries. </li></ul><ul><li>S(I) = cardinality / count * 100% </li></ul>Tip 13: Cardinality & Selectivity (1)
  49. 49. Tip 13: Cardinality & Selectivity (2)
  50. 50. <ul><li>with 10 records: 5/10 * 100% = 50% </li></ul><ul><li>with 1000 records: 75/1000 * 100 = 7.5% </li></ul><ul><li>with 10.000 records: 200/10000 * 100% = 2% </li></ul>Tip 13: Cardinality & Selectivity (3) country_id (max +-200, but effectively +- 50, maybe less)
  51. 51. <ul><li>A selectivity < 30% ? Full table scan! </li></ul><ul><li>ANALYZE TABLE frequently. </li></ul>Tip 13: Cardinality & Selectivity (4)
  52. 52. <ul><li>Adding records changes your cardinality and thus selectivity. </li></ul><ul><li>Develop against a “real” dataset. </li></ul>Tip 13: Cardinality & Selectivity (5)
  53. 53. <ul><li>14) Non-deterministic functions do not go well with query caching </li></ul>Tip 14 <ul><li>NOW(), RAND(), UUID(), CONNECTION_ID() ETC.. </li></ul>
  54. 54. Tip 14: Query caching (1) mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16768400 || Qcache_hits | 3860 || Qcache_inserts | 975 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 0 || Qcache_total_blocks | 1 |+-------------------------+----------+8 rows in set (0.00 sec) mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------+----------+| 1 | j | joshua || 2 | j | jeroen || 3 | d | david |+----+------+----------+3 rows in set (0.00 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3860 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
  55. 55. Tip 14: Query caching (2) mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------+----------+| 1 | j | joshua || 2 | j | jeroen || 3 | d | david |+----+------+----------+3 rows in set (0.00 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3861 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec) mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------+----------+| 1 | j | joshua || 2 | j | jeroen || 3 | d | david |+----+------+----------+3 rows in set (0.00 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3860 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
  56. 56. Tip 14: Query caching (3) mysql> select * from varchartest;+----+------+----------+| id | name | utf8name |+----+------+----------+| 1 | j | joshua || 2 | j | jeroen || 3 | d | david |+----+------+----------+3 rows in set (0.00 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3861 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 486 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec) mysql> select * from varchartest ORDER BY RAND();+----+------+----------+| id | name | utf8name |+----+------+----------+| 2 | j | jeroen || 1 | j | joshua || 3 | d | david |+----+------+----------+3 rows in set (0.05 sec)mysql> show status like '%qcache%';+-------------------------+----------+| Variable_name | Value |+-------------------------+----------+| Qcache_free_blocks | 1 || Qcache_free_memory | 16766864 || Qcache_hits | 3861 || Qcache_inserts | 976 || Qcache_lowmem_prunes | 0 || Qcache_not_cached | 487 || Qcache_queries_in_cache | 1 || Qcache_total_blocks | 4 |+-------------------------+----------+8 rows in set (0.00 sec)
  57. 57. SELECT * FROM table WHERE YEAR(created_dt) < YEAR(NOW()); vs SELECT * FROM table WHERE YEAR(created_dt) < ‘2010’; Tip 14: Query caching (4)
  58. 58. <ul><li>15) Certify yourself as a DBA and/or DBE. </li></ul>Tip 15 <ul><li>AND GET SOME NICE TITLES WHILE YOU’RE AT IT... </li></ul>
  59. 59. <ul><li>Oracle Certified MySQL Associate </li></ul><ul><li>Oracle Certified Professional MySQL 5.0 Developer </li></ul><ul><li>Oracle Certified Professional MySQL 5.0 Database Administrator </li></ul><ul><li>Oracle Certified Expert, MySQL 5.1 Cluster Database Administrator </li></ul><ul><li>“ Old”, but still, get them all! </li></ul>Tip 15: Certify yourself (1) <ul><li>THEY ARE NOT VERY EASY EXAMS, BUT WELL WORTH IT </li></ul>
  60. 60. Let’s summarize <ul><li>Know how to use explain. </li></ul><ul><li>Know the most basic my.cnf settings. </li></ul><ul><li>Backup on table level. </li></ul><ul><li>Don’t use “SELECT *” when you only need 1 or 2 fields. </li></ul><ul><li>Use triggers and stored procedures. </li></ul><ul><li>Don’t use FULLTEXT searches. </li></ul><ul><li>Wildcard searches (%item%) are bad for performance. </li></ul><ul><li>Shard your volatile and non-volatile data. </li></ul><ul><li>Don’t use a large Primary Key for InnoDB tables.Don’t “Select COUNT(*)” on InnoDB. </li></ul><ul><li>Don’t rely on the VARCHAR(). </li></ul><ul><li>UTF-8 is not the enemy, but it certainly isn’t your friend.Know your cardinality & selectivity.Non-deterministic functions do not go well with query caching. Certify yourself as a DBA and/or DBE. </li></ul>
  61. 61. Any questions? <ul><li>QUESTIONS? </li></ul>http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg
  62. 62. Daycamp for developers http://www.enrise.com http://www.daycamp4developers.com /
  63. 63. <ul><li>THANK YOU FOR YOUR ATTENTION </li></ul><ul><li>Please rate my talk: http://joind.in/3662 </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×