10 things you might not know about MySQL

11,646 views
11,532 views

Published on

Published in: Technology
0 Comments
14 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
11,646
On SlideShare
0
From Embeds
0
Number of Embeds
6,743
Actions
Shares
0
Downloads
193
Comments
0
Likes
14
Embeds 0
No embeds

No notes for slide

10 things you might not know about MySQL

  1. 1. 10 things you might not know about MySQL Jorge Bernal <jbernal@warp.es> Version 0.1
  2. 2. 1 Query cache
  3. 3. Before mysql> SELECT AVG(Population) FROM City_huge; +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ | AVG(Population) | +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ |     354359.9948 |  +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ 1 row in set (0.58 sec) mysql> SELECT AVG(Population) FROM City_huge; +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ | AVG(Population) | +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ |     354359.9948 |  +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ 1 row in set (0.56 sec)
  4. 4. After mysql> SELECT AVG(Population) FROM City_huge; +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ | AVG(Population) | +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ |     354359.9948 |  +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ 1 row in set (0.56 sec) mysql> SELECT AVG(Population) FROM City_huge; +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ | AVG(Population) | +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ |     354359.9948 |  +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ 1 row in set (0.00 sec)
  5. 5. The magic SET global query_cache_size=8 * 1024 * 1024; mysql> SHOW GLOBAL VARIABLES LIKE 'query_cache%'; +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+ | Variable_name                | Value   | +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+ | query_cache_limit            | 1048576 |  | query_cache_min_res_unit     | 4096    |  | query_cache_size             | 8388608 |  | query_cache_type             | ON      |  | query_cache_wlock_invalidate | OFF     |  +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+ 5 rows in set (0.00 sec) mysql> SHOW GLOBAL STATUS LIKE 'Qc%'; +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+ | Variable_name           | Value   | +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+ | Qcache_free_blocks      | 1       |  | Qcache_free_memory      | 8378312 |  | Qcache_hits             | 1       |  | Qcache_inserts          | 1       |  | Qcache_lowmem_prunes    | 0       |  | Qcache_not_cached       | 0       |  | Qcache_queries_in_cache | 1       |  | Qcache_total_blocks     | 4       |  +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+ 8 rows in set (0.00 sec)
  6. 6. 2 Life beyond MyISAM/ InnoDB
  7. 7. Archive mysql> SELECT ENGINE,  (INDEX_LENGTH+DATA_LENGTH) AS Size  FROM INFORMATION_SCHEMA.TABLES  WHERE TABLE_NAME = 'City_huge'; +‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+ | ENGINE | Size     | +‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+ | MyISAM | 63203585 |  +‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+ mysql> SELECT ENGINE,  (INDEX_LENGTH+DATA_LENGTH) AS Size  FROM INFORMATION_SCHEMA.TABLES  WHERE TABLE_NAME = 'City_huge'; +‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+ | ENGINE  | Size     | +‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+ | ARCHIVE | 13520399 |  +‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+
  8. 8. ARCHIVE MyISAM 80% compression
  9. 9. But... • Only INSERT and SELECT • No indexing
  10. 10. CSV mysql> ALTER TABLE City ENGINE=CSV; Query OK, 0 rows affected (0.05 sec) Records: 0  Duplicates: 0  Warnings: 0 root@warhol:/usr/local/mysql/data/world$ head City.CSV  1,quot;Kabulquot;,quot;AFGquot;,quot;Kabolquot;,1780000 2,quot;Qandaharquot;,quot;AFGquot;,quot;Qandaharquot;,237500 3,quot;Heratquot;,quot;AFGquot;,quot;Heratquot;,186800 4,quot;Mazar‐e‐Sharifquot;,quot;AFGquot;,quot;Balkhquot;,127800 5,quot;Amsterdamquot;,quot;NLDquot;,quot;Noord‐Hollandquot;,731200 6,quot;Rotterdamquot;,quot;NLDquot;,quot;Zuid‐Hollandquot;,593321 7,quot;Haagquot;,quot;NLDquot;,quot;Zuid‐Hollandquot;,440900 8,quot;Utrechtquot;,quot;NLDquot;,quot;Utrechtquot;,234323 9,quot;Eindhovenquot;,quot;NLDquot;,quot;Noord‐Brabantquot;,201843 10,quot;Tilburgquot;,quot;NLDquot;,quot;Noord‐Brabantquot;,193238
  11. 11. Merge mysql> SHOW CREATE TABLE allweekG [...] Create Table: CREATE TABLE `allweek` (   `ID` int(11) NOT NULL DEFAULT '0',   `Name` char(35) NOT NULL DEFAULT '',   `CountryCode` char(3) NOT NULL DEFAULT '',   `District` char(20) NOT NULL DEFAULT '',   `Population` int(11) NOT NULL DEFAULT '0',   `modtime` datetime DEFAULT NULL,   KEY `modtime` (`modtime`) ) ENGINE=MRG_MyISAM DEFAULT CHARSET=latin1  UNION=(`monday`,`tuesday`,`wednesday`,`thursday`,`city date`) 1 row in set (0.00 sec)
  12. 12. 3 AUTO_INCREMENT woes
  13. 13. mysql>   CREATE TABLE t (     ‐>    id INT NOT NULL PRIMARY KEY AUTO_INCREMENT     ‐>   ); Query OK, 0 rows affected (0.00 sec) mysql>   INSERT INTO t (id)     ‐>   VALUES (NULL); Query OK, 1 row affected (0.04 sec) mysql>   SELECT *     ‐>   FROM t     ‐>   WHERE id IS NULL; +‐‐‐‐+ | id | +‐‐‐‐+ |  1 |  +‐‐‐‐+
  14. 14. But... • id is AUTO_INCREMENT, so it’s 1 • let’s run that again mysql>   CREATE TABLE t (     ‐>    id INT NOT NULL PRIMARY KEY AUTO_INCREMENT     ‐>   ); Query OK, 0 rows affected (0.00 sec) mysql>   INSERT INTO t (id)     ‐>   VALUES (NULL); Query OK, 1 row affected (0.04 sec) mysql>   SELECT *     ‐>   FROM t     ‐>   WHERE id IS NULL; +‐‐‐‐+ | id | +‐‐‐‐+ |  1 |  +‐‐‐‐+
  15. 15. mysql>   CREATE TABLE t (     ‐>    id INT NOT NULL PRIMARY KEY AUTO_INCREMENT     ‐>   ); Query OK, 0 rows affected (0.00 sec) mysql>   INSERT INTO t (id)     ‐>   VALUES (NULL); Query OK, 1 row affected (0.04 sec) mysql>   SELECT *     ‐>   FROM t     ‐>   WHERE id IS NULL; +‐‐‐‐+ | id | +‐‐‐‐+ |  1 |  +‐‐‐‐+ mysql>   SELECT *     ‐>   FROM t     ‐>   WHERE id IS NULL; Empty set (0.00 sec) mysql> SELECT *   FROM t; +‐‐‐‐+ | id | +‐‐‐‐+ |  1 |  +‐‐‐‐+ 1 row in set (0.00 sec)
  16. 16. WTF??? • WHERE ID IS NULL acts as WHERE ID = LAST_ISERT_ID() • But only the first time • Brought to you by some weird ODBC compatibility decision
  17. 17. 4 How to get random data
  18. 18. SELECT name FROM random ORDER BY RAND() LIMIT 1
  19. 19. X SELECT name FROM random ORDER BY RAND() LIMIT 1 Wrong!
  20. 20. There is a better way SELECT name FROM random JOIN (  SELECT CEIL(   RAND() *   (SELECT MAX(id) FROM random))    AS id ) AS r2 USING (id);
  21. 21. Really?
  22. 22. Really? ORDER BY RAND() Subquery 10.000,00 1.000,00 100,00 Seconds 10,00 1,00 0,10 100 1000 10000 100000 Table Size
  23. 23. So... Every time you use ORDER BY RAND()... God kills a kitten Please, think of the kittens
  24. 24. 5 Prefix indexes
  25. 25. There’s no need to index the whole column
  26. 26. Name is CHAR(52) mysql> SHOW CREATE TABLE CountryG *************************** 1. row ***************************        Table: Country Create Table: CREATE TABLE `Country` (   `Code` char(3) NOT NULL DEFAULT '',   `Name` char(52) NOT NULL DEFAULT '',   `Continent` enum('Asia','Europe','North  America','Africa','Oceania','Antarctica','South America') NOT NULL  DEFAULT 'Asia',   `Region` char(26) NOT NULL DEFAULT '',   `SurfaceArea` float(10,2) NOT NULL DEFAULT '0.00',   `IndepYear` smallint(6) DEFAULT NULL,   `Population` int(11) NOT NULL DEFAULT '0',   `LifeExpectancy` float(3,1) DEFAULT NULL,   `GNP` float(10,2) DEFAULT NULL,   `GNPOld` float(10,2) DEFAULT NULL,   `LocalName` char(45) NOT NULL DEFAULT '',   `GovernmentForm` char(45) NOT NULL DEFAULT '',   `HeadOfState` char(60) DEFAULT NULL,   `Capital` int(11) DEFAULT NULL,   `Code2` char(2) NOT NULL DEFAULT '',   PRIMARY KEY (`Code`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 1 row in set (0.02 sec)
  27. 27. mysql> SELECT COUNT(*) AS Total, COUNT(DISTINCT Name) AS Diff, COUNT(*)  ‐ COUNT(DISTINCT Name) AS Dupes FROM Country; +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Total | Diff | Dupes | +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ |   239 |  239 |     0 |  +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (1.04 sec) mysql> SELECT COUNT(*) AS Total, COUNT(DISTINCT LEFT(Name,50)) AS Diff,  COUNT(*) ‐ COUNT(DISTINCT LEFT(Name,50)) AS Dupes FROM Country; +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Total | Diff | Dupes | +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ |   239 |  239 |     0 |  +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (2.02 sec) mysql> SELECT COUNT(*) AS Total, COUNT(DISTINCT LEFT(Name,20)) AS Diff,  COUNT(*) ‐ COUNT(DISTINCT LEFT(Name,20)) AS Dupes FROM Country; +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Total | Diff | Dupes | +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ |   239 |  239 |     0 |  +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (0.93 sec)
  28. 28. mysql> SELECT COUNT(*) AS Total, COUNT(DISTINCT Name) AS Diff, COUNT(*)  ‐ COUNT(DISTINCT Name) AS Dupes FROM Country; +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Total | Diff | Dupes | +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ |   239 |  239 |     0 |  +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (1.04 sec) mysql> SELECT COUNT(*) AS Total, COUNT(DISTINCT LEFT(Name,50)) AS Diff,  COUNT(*) ‐ COUNT(DISTINCT LEFT(Name,50)) AS Dupes FROM Country; +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Total | Diff | Dupes | +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ |   239 |  239 |     0 |  +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (2.02 sec) mysql> SELECT COUNT(*) AS Total, COUNT(DISTINCT LEFT(Name,20)) AS Diff,  COUNT(*) ‐ COUNT(DISTINCT LEFT(Name,20)) AS Dupes FROM Country; +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Total | Diff | Dupes | +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ Got it? Cool! Now... |   239 |  239 |     0 |  +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (0.93 sec)
  29. 29. There is a better way CREATE DEFINER=`root`@`localhost` PROCEDURE `pref_index`(t_name CHAR(255), c_name CHAR(255)) BEGIN DECLARE plength INT DEFAULT 1; SET @q = CONCAT('SELECT CHARACTER_MAXIMUM_LENGTH INTO @maxlen                                                          FROM INFORMATION_SCHEMA.COLUMNS                                                         WHERE TABLE_SCHEMA = quot;world2quot; AND                                                                         TABLE_NAME = quot;',  t_name,'quot; AND                                                                         COLUMN_NAME = quot;',  c_name, 'quot;'); PREPARE q FROM @q; EXECUTE q; DEALLOCATE PREPARE q; REPEAT         SET @qq = CONCAT('SELECT COUNT(*) ‐ COUNT(DISTINCT ',c_name,')                                                                  into @dupe FROM ',t_name);         SET @pq = CONCAT('SELECT COUNT(*) ‐ COUNT(DISTINCT LEFT(',c_name,',',plength,'))                                                                  into @pdupe FROM ',t_name);         PREPARE qs FROM @qq;         EXECUTE qs;         DEALLOCATE PREPARE qs;         PREPARE ps FROM @pq;         EXECUTE ps;         DEALLOCATE PREPARE ps;         SET plength = plength + 1; UNTIL plength >= @maxlen OR @pdupe = @dupe END REPEAT; SELECT plength, @pdupe, @dupe; END
  30. 30. mysql> CALL pref_index('Country', 'Name'); +‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | plength | @pdupe | @dupe | +‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ |      18 |      0 |     0 |  +‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (1.18 sec) Query OK, 0 rows affected (1.18 sec) mysql> SELECT COUNT(*) AS Total, COUNT(DISTINCT LEFT(Name,18)) AS  Diff, COUNT(*) ‐ COUNT(DISTINCT LEFT(Name,18)) AS Dupes FROM Country; +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Total | Diff | Dupes | +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ |   239 |  239 |     0 |  +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (0.00 sec) mysql> ALTER TABLE Country ADD KEY (Name(18)); Query OK, 239 rows affected (1.48 sec) Records: 239  Duplicates: 0  Warnings: 0
  31. 31. mysql> CALL pref_index('Country', 'Name'); +‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | plength | @pdupe | @dupe | +‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ |      18 |      0 |     0 |  +‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (1.18 sec) Query OK, 0 rows affected (1.18 sec) mysql> SELECT COUNT(*) AS Total, COUNT(DISTINCT LEFT(Name,18)) AS  Diff, COUNT(*) ‐ COUNT(DISTINCT LEFT(Name,18)) AS Dupes FROM Country; +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Total | Diff | Dupes | +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ |   239 |  239 |     0 |  We just saved 34 bytes per row! +‐‐‐‐‐‐‐+‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (0.00 sec) mysql> ALTER TABLE Country ADD KEY (Name(18)); Query OK, 239 rows affected (1.48 sec) Records: 239  Duplicates: 0  Warnings: 0
  32. 32. 6 InnoDB clustered index
  33. 33. Data stored in PK order Primary key Index Leaf nodes Indexes point to PK instead of actual data
  34. 34. So what?
  35. 35. Choose your PK wisely
  36. 36. Load data in order mysql> load data infile '/tmp/city_order.txt' into  table City_huge; Query OK, 818027 rows affected (15.86 sec) Records: 818027  Deleted: 0  Skipped: 0  Warnings: 0 mysql> load data infile '/tmp/city_rand.txt' into  table City_huge; Query OK, 818027 rows affected (33 min 23.57 sec) Records: 818027  Deleted: 0  Skipped: 0  Warnings: 0
  37. 37. 7 Profiling
  38. 38. mysql> select * from v_client_portfolio_high; +‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ | client_id | client_first_name | client_last_name | portfolio_value | +‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ |         5 | ABNER             | ROSSELLETT       |      1252115.50 | |       500 | CANDICE           | BARTLETT         |      1384877.50 | +‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ 2 rows in set (4.01 sec)
  39. 39. mysql> select * from v_client_portfolio_high; +‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ | client_id | client_first_name | client_last_name | portfolio_value | +‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ |         5 | ABNER             | ROSSELLETT       |      1252115.50 | |       500 | CANDICE           | BARTLETT         |      1384877.50 | +‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ W hy? 2 rows in set (4.01 sec)
  40. 40. mysql> set profiling=1; Query OK, 0 rows affected (0.00 sec) mysql> select * from v_client_portfolio_high; +‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ | client_id | client_first_name | client_last_name | portfolio_value | +‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ |         5 | ABNER             | ROSSELLETT       |      1252115.50 | |       500 | CANDICE           | BARTLETT         |      1384877.50 | +‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ 2 rows in set (4.01 sec) mysql> show profiles; +‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ | Query_ID | Duration   | Query                                         | +‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ |        1 | 0.00007600 | set profiling=1                               | |        2 | 4.01965600 | select * from v_client_portfolio_high         | +‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+
  41. 41. mysql> select min(seq) seq,state,count(*) numb_ops,     ‐> round(sum(duration),5) sum_dur, round(avg(duration),5) avg_dur,     ‐> round(sum(cpu_user),5) sum_cpu, round(avg(cpu_user),5) avg_cpu     ‐> from information_schema.profiling     ‐> where query_id = 2     ‐> group by state     ‐> order by seq; +‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+ | seq   | state                | numb_ops | sum_dur | avg_dur | sum_cpu | avg_cpu | +‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+ |     0 | (initialization)     |        1 | 0.00004 | 0.00004 | 0.00000 | 0.00000 | |     1 | Opening tables       |        1 | 0.00023 | 0.00023 | 0.00000 | 0.00000 | |     2 | System lock          |        1 | 0.00001 | 0.00001 | 0.00000 | 0.00000 | |     3 | Table lock           |        1 | 0.00001 | 0.00001 | 0.00000 | 0.00000 | |     4 | checking permissions |        1 | 0.00010 | 0.00010 | 0.00000 | 0.00000 | |     5 | optimizing           |        4 | 0.00004 | 0.00001 | 0.00000 | 0.00000 | |     6 | statistics           |        4 | 0.00007 | 0.00002 | 0.00100 | 0.00025 | |     7 | preparing            |        4 | 0.00005 | 0.00001 | 0.00000 | 0.00000 | |     8 | Creating tmp table   |        1 | 0.00003 | 0.00003 | 0.00000 | 0.00000 | |     9 | executing            |    37352 | 0.16631 | 0.00000 | 0.05899 | 0.00000 | |    10 | Copying to tmp table |        1 | 0.00006 | 0.00006 | 0.00000 | 0.00000 | |    15 | Sending data         |    37353 | 3.85151 | 0.00010 | 3.72943 | 0.00010 | | 74717 | Sorting result       |        1 | 0.00112 | 0.00112 | 0.00100 | 0.00100 | | 74719 | removing tmp table   |        2 | 0.00003 | 0.00001 | 0.00000 | 0.00000 | | 74721 | init                 |        1 | 0.00002 | 0.00002 | 0.00000 | 0.00000 | | 74727 | end                  |        1 | 0.00001 | 0.00001 | 0.00000 | 0.00000 | | 74728 | query end            |        1 | 0.00000 | 0.00000 | 0.00000 | 0.00000 | | 74729 | freeing items        |        1 | 0.00002 | 0.00002 | 0.00000 | 0.00000 | | 74730 | closing tables       |        2 | 0.00001 | 0.00001 | 0.00000 | 0.00000 | | 74733 | logging slow query   |        1 | 0.00000 | 0.00000 | 0.00000 | 0.00000 | +‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐‐‐+
  42. 42. But... • Only in 5.0 • 5.1 implementation under review • http://forge.mysql.com/wiki/ Testing_Show_Profiles_5_1
  43. 43. 8 Usage statistics Know what’s going on
  44. 44. SHOW STATUS mysql> SHOW STATUS LIKE 'Com_select'; +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Variable_name | Value | +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Com_select    | 13    |  +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (1.65 sec) mysql> SELECT 1; +‐‐‐+ | 1 | +‐‐‐+ | 1 |  +‐‐‐+ 1 row in set (0.00 sec) mysql> SHOW STATUS LIKE 'Com_select'; +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Variable_name | Value | +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Com_select    | 14    |  +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (0.00 sec)
  45. 45. SHOW STATUS mysql> SHOW STATUS LIKE 'Com_select'; +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Variable_name | Value | +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Com_select    | 14    |  +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (0.00 sec) mysql> FLUSH STATUS; Query OK, 0 rows affected (0.00 sec) mysql> SHOW STATUS LIKE 'Com_select'; +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Variable_name | Value | +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ | Com_select    | 0     |  +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+‐‐‐‐‐‐‐+ 1 row in set (0.00 sec)
  46. 46. External tools • mysqlsla http://hackmysql.com/mysqlsla mysqlsla parses, filters, analyzes and sorts MySQL slow, general, binary and microslow patched logs in order to create a customizable report of the queries and their meta-property values.
  47. 47. External tools • mysqlidxchk http://hackmysql.com/mysqlidxchk mysqlidxchk (MySQL Index Checker) checks MySQL databases/tables for unused indexes. Given one or more slow, general, or quot;rawquot; log files, mysqlidxchk reports which indexes in the database schema are not used by the queries in the log files.
  48. 48. 9 Understanding REPLACE
  49. 49. mysql> CREATE TABLE fk_relations (     ‐>   key1 INT NOT NULL PRIMARY KEY,     ‐>   key2 INT NOT NULL UNIQUE     ‐> ); Query OK, 0 rows affected (0.02 sec) mysql> INSERT INTO fk_relations VALUES (1,1), (2,2); Query OK, 2 rows affected (0.00 sec) Records: 2  Duplicates: 0  Warnings: 0 mysql> SELECT * FROM fk_relations; +‐‐‐‐‐‐+‐‐‐‐‐‐+ | key1 | key2 | +‐‐‐‐‐‐+‐‐‐‐‐‐+ |    1 |    1 |  |    2 |    2 |  +‐‐‐‐‐‐+‐‐‐‐‐‐+ 2 rows in set (0.00 sec)
  50. 50. mysql> REPLACE INTO fk_relations VALUES (1,3); Query OK, 2 rows affected (0.03 sec)
  51. 51. mysql> REPLACE INTO fk_relations VALUES (1,3); Query OK, 2 rows affected (0.03 sec) mysql> SELECT * FROM fk_relations; +‐‐‐‐‐‐+‐‐‐‐‐‐+ | key1 | key2 | +‐‐‐‐‐‐+‐‐‐‐‐‐+ |    1 |    3 |  |    2 |    2 |  +‐‐‐‐‐‐+‐‐‐‐‐‐+ 2 rows in set (0.00 sec) mysql> REPLACE INTO fk_relations VALUES (1,2); Query OK, 3 rows affected (0.06 sec) ???
  52. 52. mysql> REPLACE INTO fk_relations VALUES (1,2); Query OK, 3 rows affected (0.06 sec) mysql> SELECT * FROM fk_relations; +‐‐‐‐‐‐+‐‐‐‐‐‐+ | key1 | key2 | +‐‐‐‐‐‐+‐‐‐‐‐‐+ |    1 |    2 |  +‐‐‐‐‐‐+‐‐‐‐‐‐+ 1 row in set (0.00 sec)
  53. 53. +‐‐‐‐‐‐+‐‐‐‐‐‐+ | key1 | key2 | +‐‐‐‐‐‐+‐‐‐‐‐‐+ |    1 |    3 |  |    2 |    2 |  +‐‐‐‐‐‐+‐‐‐‐‐‐+ mysql> REPLACE INTO fk_relations VALUES (1,2); ‐‐ Equals...
  54. 54. +‐‐‐‐‐‐+‐‐‐‐‐‐+ | key1 | key2 | +‐‐‐‐‐‐+‐‐‐‐‐‐+ |    1 |    3 |  |    2 |    2 |  +‐‐‐‐‐‐+‐‐‐‐‐‐+ mysql> REPLACE INTO fk_relations VALUES (1,2); ‐‐ Equals... ‐‐ key1 is PK mysql> DELETE FROM fk_relations WHERE key1 = 1;
  55. 55. +‐‐‐‐‐‐+‐‐‐‐‐‐+ | key1 | key2 | +‐‐‐‐‐‐+‐‐‐‐‐‐+ |    2 |    2 |  +‐‐‐‐‐‐+‐‐‐‐‐‐+ mysql> REPLACE INTO fk_relations VALUES (1,2); ‐‐ Equals... ‐‐ key1 is PK mysql> DELETE FROM fk_relations WHERE key1 = 1; ‐‐ key2 is Unique mysql> DELETE FROM fk_relations WHERE key2 = 2;
  56. 56. +‐‐‐‐‐‐+‐‐‐‐‐‐+ | key1 | key2 | +‐‐‐‐‐‐+‐‐‐‐‐‐+ |    1 |    2 |  +‐‐‐‐‐‐+‐‐‐‐‐‐+ mysql> REPLACE INTO fk_relations VALUES (1,2); ‐‐ Equals... ‐‐ key1 is PK mysql> DELETE FROM fk_relations WHERE key1 = 1; ‐‐ key2 is Unique mysql> DELETE FROM fk_relations WHERE key2 = 2; mysql> INSERT INTO fk_relations VALUES (1,2);
  57. 57. That’s why mysql> REPLACE INTO fk_relations VALUES (1,2); Query OK, 3 rows affected (0.06 sec) mysql> SELECT * FROM fk_relations; +‐‐‐‐‐‐+‐‐‐‐‐‐+ | key1 | key2 | +‐‐‐‐‐‐+‐‐‐‐‐‐+ |    1 |    2 |  +‐‐‐‐‐‐+‐‐‐‐‐‐+ 1 row in set (0.00 sec)
  58. 58. That’s why mysql> REPLACE INTO fk_relations VALUES (1,2); Query OK, 3 rows affected (0.06 sec) mysql> SELECT * FROM fk_relations; +‐‐‐‐‐‐+‐‐‐‐‐‐+ + 1 insert 2 deletes | key1 | key2 | +‐‐‐‐‐‐+‐‐‐‐‐‐+ |    1 |    2 |  +‐‐‐‐‐‐+‐‐‐‐‐‐+ 1 row in set (0.00 sec)
  59. 59. 10 The good and bad about temporary tables
  60. 60. Performance • Be careful with complex queries • Extremely slow on large datasets • Use summary tables instead
  61. 61. Too complex select d.dept_name, SUM(salary) from departments d  LEFT JOIN dept_emp de USING (dept_no) LEFT JOIN  salaries s USING (emp_no) where s.from_date >=  '2000‐01‐01' and s.to_date < '2001‐01‐01' group by  dept_no;
  62. 62. Instead, split in two select d.dept_name, SUM(salary) from departments d  LEFT JOIN dept_emp de USING (dept_no) LEFT JOIN  salaries s USING (emp_no) where s.from_date >=  '2000‐01‐01' and s.to_date < '2001‐01‐01' group by  dept_no; CREATE TEMPORARY TABLE salaries2000  SELECT * FROM salaries s  WHERE s.from_date >= '2000‐01‐01'  AND s.to_date < '2001‐01‐01'; SELECT d.dept_name, SUM(salary)  FROM departments d  LEFT JOIN dept_emp de USING (dept_no)  LEFT JOIN salaries2000 s USING (emp_no)  GROUP BY dept_no;
  63. 63. Big performance gain (If indexes are in their place)
  64. 64. Big performance gain (If indexes are in their place) I’ve seen 10X!
  65. 65. one more thing...
  66. 66. one more thing... I had to do it ;-)
  67. 67. Shit happens mysql> select * from t; +‐‐‐‐‐‐+ | id   | +‐‐‐‐‐‐+ |    2 |  +‐‐‐‐‐‐+ 1 row in set (0.00 sec) mysql> drop table t; Query OK, 0 rows affected (0.04 sec) WTF?!? mysql> drop table t; Query OK, 0 rows affected (0.00 sec) mysql> drop table t; ERROR 1051 (42S02): Unknown table 't'
  68. 68. Some seconds ago...
  69. 69. mysql> select * from t; +‐‐‐‐+ | id | +‐‐‐‐+ |  1 |  +‐‐‐‐+ 1 row in set (0.00 sec) WTF?!? mysql> create temporary table t(id int); ...again Query OK, 0 rows affected (0.08 sec) mysql> select * from t; Empty set (0.00 sec) mysql> insert into t values (2); Query OK, 1 row affected (0.05 sec) mysql> select * from t; +‐‐‐‐‐‐+ | id   | +‐‐‐‐‐‐+ |    2 |  +‐‐‐‐‐‐+ 1 row in set (0.00 sec)
  70. 70. • Temporary tables exist per session • Be careful using connection pools • They overlap current tables
  71. 71. There’s a reason for everything mysql> select * from t; +‐‐‐‐‐‐+ | id   | +‐‐‐‐‐‐+ |    2 |  +‐‐‐‐‐‐+ 1 row in set (0.00 sec) temporary mysql> drop table t; Query OK, 0 rows affected (0.04 sec) mysql> select * from t; +‐‐‐‐+ | id | +‐‐‐‐+ |  1 |  +‐‐‐‐+ regular 1 row in set (0.00 sec) mysql> drop table t; Query OK, 0 rows affected (0.00 sec) mysql> drop table t; ERROR 1051 (42S02): Unknown table 't'
  72. 72. What else? • MySQL Proxy https://launchpad.net/mysql-proxy • MySQL Sandbox https://launchpad.net/mysql-sandbox • MySQL Random Query Generator https://launchpad.net/randgen
  73. 73. Questions? Probably out of time at this point, but it’s the standard
  74. 74. Thanks!

×