How to Design                             Indexes, Really                            Bill Karwin, Percona Inc.Tuesday, Dec...
It’s About Performance       • What’s the most frequent recommendation in                database performance audits?     ...
Indexing Mistakes                                           www.percona.comTuesday, December 4, 2012
Index Shotgun                “Indexes improve performance,                    so I index every column.”                   ...
Index Shotgun       • Many such indexes are never used.       • Indexes occupy space on disk in memory buffers.       • Ea...
Index Aversion                       “Indexes have overhead,                     so I never create any indexes.”          ...
Index Aversion       • The right indexes are crucial to speed up queries.       • Most workloads are read-heavy, so the ne...
SQL Aversion                            “I use a NoSQL database                              because it just works.” *    ...
SQL Aversion       • Many non-relational databases require you to                define indexes explicitly too.       • Th...
Naive Questions                             “Here’s my schema...                            what indexes do I need?”      ...
Naive Questions       • Index choice depends on...              • What tables and columns you need to query              •...
Lesson #1              relational                                         index               schema              design i...
Where Are the Queries?                              Slow query log                             General query log          ...
Collecting Log of Queries       • Enable slow-query log:              mysql> SET GLOBAL slow_query_log = ON;       • Inclu...
Where to Start?                                pt-query-digest                    http://www.percona.com/doc/percona-toolk...
pt-query-digest       • Analyzes your query log.       • Reports the queries accounting for greatest load.              • ...
Ranked Queries   # Profile   # Rank Query ID             Response time     Calls   R/Call    Apdx V/M     Item   # ==== ==...
Slow Queries   # Profile   # Rank Query ID              Response time     Calls   R/Call    Apdx V/M     Item   # ==== ===...
Quick Queries   # Profile   # Rank Query ID             Response time     Calls   R/Call    Apdx V/M     Item   # ==== ===...
Understanding                            Indexes by Analogy                                            www.percona.comTues...
This is a Telephone Book       • A telephone book is like an index sorted          by last name first, first name last.   ...
Implementation       • In MySQL syntax, this would look like the                following:              CREATE INDEX phone...
Simple Searches       • Since the index is pre-sorted, it benefits queries                for last name:              ! SE...
Compound Searches       • The sort order benefits us further if we search on                last name and first name toget...
When the Index Fails       • But if we search only on first_name, the sort order                doesn’t help:             ...
Order Matters       • From this, we conclude that the order we define                the columns in an index matters.     ...
Both Indexes Can Be Helpful       • If both criteria use equality comparisons, then the                order of columns sh...
Range Comparisons       • Searching for multiple names is another case.       • An index still helps us to narrow the sear...
Compound Range Comparisons       • If we search by a range of last names, but a                specific first name, then t...
Order Matters       • From this we see that once we do a range                comparison on a column in an index, any crit...
Sorting by Index       • Since the telephone book is pre-sorted, we can                rely on this if we need to read thr...
When the Index Can’t Help Sorting       • Sorting by any other column is another story.              ! SELECT * FROM Telep...
Order Matters       • From this, we conclude that an index helps if we                sort by the column immediately follo...
Index-Only Searches       • Once we find the matching entries, we may need                other fields of information:    ...
When the Index Can’t Help       • If we need other information that isn’t included in                the index, we may nee...
Columns Matter       • From this, we conclude that putting columns in an                index can be very important for ce...
Lesson #2                             rate your                               indexes                              using t...
The Star System       • Rate an index with respect to a                given query by three “stars”                which d...
The Star System       ★ First star:                    Rows referenced by your query are grouped                    togeth...
Look at this Terrible Query           mysql> EXPLAIN SELECT person_id,             person_role_id FROM cast_info          ...
★ First Star       ★ Pick all columns from equality predicates.                    Define the index with these columns fir...
★ Second Star       ★ Add the column(s) in the GROUP BY or ORDER                    BY clause, if the query has one.      ...
★ Third Star       ★ Add any remaining columns referenced in the                    SELECT list.              ! SELECT per...
Explain That              mysql> EXPLAIN SELECT person_id,               person_role_id FROM cast_info               WHERE...
Complications       • Can’t achieve First-Star indexes when:              • WHERE clause has more complex expressions     ...
Complications       • Can’t achieve Second-Star indexes when:              • WHERE clause includes range predicates       ...
Complications       • Can’t achieve Third-Star indexes when:              • Query requires more columns than the maximum o...
Example Query                                            www.percona.comTuesday, December 4, 2012
Example Query       mysql> EXPLAIN SELECT STRAIGHT_JOIN n.*         FROM char_name AS c         INNER JOIN cast_info AS i ...
Not Optimized           ***************************   1. row ***************************                      id: 1       ...
Add Indexes       ★ First-star index:              mysql> ALTER TABLE char_name               ADD INDEX n (name(20));     ...
Optimized           ***************************   1. row ***************************                      id: 1           ...
Why Not Second Star?       • Not possible to create a second-star index here.       • The order of the join column isn’t n...
Lesson #3                              tidy up                             indexes                            you don’t   ...
Indexes You Don’t Need?                create                         drop               indexes                       ind...
Redundant Indexes                       pt-duplicate-key-checker                      http://www.percona.com/doc/percona-t...
Redundant Indexes       mysql> create table foo (a int, b int,         key (a), key (b), key (a,b));       $ pt-duplicate-...
Unused Indexes                                     pt-index-usage                            http://www.percona.com/doc/pe...
Unused Indexes       $ sudo pt-index-usage /var/lib/mysql/mysql-slow.log       /var/lib/mysql/mysql-slow.log: 12% 01:07 re...
Index Usage Statistics                                Index Statistics                              in Percona Server     ...
Index Usage Statistics       • INFORMATION_SCHEMA.INDEX_STATISTICS       • Enable with:              mysql> SET GLOBAL use...
Index Usage Statistics       • Find indexes that were never used:       mysql> SELECT CONCAT(         !    ALTER TABLE `, ...
Lesson #4                            make it a                             habit.                                        w...
Review Queries Regularly       • Repeat the query analysis periodically:              • As new application code introduces...
Review Queries Regularly       • Track query types you have reviewed before:              • pt-query-digest --review      ...
Review Indexes Regularly       • Run pt-duplicate-key-checker periodically:              • After schema changes.          ...
Review Indexes Regularly       • Run pt-index-usage or review                INDEX_STATISTICS periodically:              •...
Lessons Redux                            identify   rate your                            queries.    indexes.             ...
Percona Training for MySQL                               Senior Industry Experts                            In-Person and ...
Visit Our Sponsors       diamond       platinum       exhibitors       and the rest!                                      ...
SQL Antipatterns                            http://www.pragprog.com/titles/bksqla/                                        ...
License and Copyright                                Copyright 2012 Bill Karwin                               www.slidesha...
Upcoming SlideShare
Loading in...5
×

How to Design Indexes, Really

41,599

Published on

MySQL users commonly ask: Here's my table, what indexes do I need? Why aren't my indexes helping me? Don't indexes cause overhead? This talk gives you some practical answers, with a step by step method for finding the queries you need to optimize, and choosing the best indexes for them.

Published in: Technology
2 Comments
104 Likes
Statistics
Notes
No Downloads
Views
Total Views
41,599
On Slideshare
0
From Embeds
0
Number of Embeds
25
Actions
Shares
0
Downloads
717
Comments
2
Likes
104
Embeds 0
No embeds

No notes for slide

How to Design Indexes, Really

  1. 1. How to Design Indexes, Really Bill Karwin, Percona Inc.Tuesday, December 4, 2012
  2. 2. It’s About Performance • What’s the most frequent recommendation in database performance audits? • What’s the easiest way to speed up SQL queries, without schema changes or code changes? • What’s a commonly neglected part of database development and maintenance? indexes. www.percona.comTuesday, December 4, 2012
  3. 3. Indexing Mistakes www.percona.comTuesday, December 4, 2012
  4. 4. Index Shotgun “Indexes improve performance, so I index every column.” www.percona.comTuesday, December 4, 2012
  5. 5. Index Shotgun • Many such indexes are never used. • Indexes occupy space on disk in memory buffers. • Each index needs to be modified during INSERT, UPDATE, DELETE. • Query optimizer considers indexes during every query, more indexes makes this work harder. www.percona.comTuesday, December 4, 2012
  6. 6. Index Aversion “Indexes have overhead, so I never create any indexes.” www.percona.comTuesday, December 4, 2012
  7. 7. Index Aversion • The right indexes are crucial to speed up queries. • Most workloads are read-heavy, so the net benefit of indexes outweighs the overhead of updating. • Indexes are compact, so they are a more efficient use of buffer memory. www.percona.comTuesday, December 4, 2012
  8. 8. SQL Aversion “I use a NoSQL database because it just works.” * (* after I create the right indexes.) www.percona.comTuesday, December 4, 2012
  9. 9. SQL Aversion • Many non-relational databases require you to define indexes explicitly too. • The ANSI/ISO SQL standard doesn’t specify anything about indexes. • So if you want to use NoSQL, just use indexes! :-) www.percona.comTuesday, December 4, 2012
  10. 10. Naive Questions “Here’s my schema... what indexes do I need?” Copyright 2012 Gene Ontology Consortium http://www.geneontology.org/ www.percona.comTuesday, December 4, 2012
  11. 11. Naive Questions • Index choice depends on... • What tables and columns you need to query • What JOINs you need to perform • What GROUP BY’s and ORDER BY’s you need • Index design is not implicit from table design. www.percona.comTuesday, December 4, 2012
  12. 12. Lesson #1 relational index schema design is based on but design is based on queries. data. www.percona.comTuesday, December 4, 2012
  13. 13. Where Are the Queries? Slow query log General query log Application logging Tcpdump Binary log Process list www.percona.comTuesday, December 4, 2012
  14. 14. Collecting Log of Queries • Enable slow-query log: mysql> SET GLOBAL slow_query_log = ON; • Include all queries, not just those that are slow: mysql> SET GLOBAL long_query_time = 0; • Percona Server can log more information: mysql> SET GLOBAL log_slow_verbosity = full; www.percona.comTuesday, December 4, 2012
  15. 15. Where to Start? pt-query-digest http://www.percona.com/doc/percona-toolkit/pt-query-digest.html www.percona.comTuesday, December 4, 2012
  16. 16. pt-query-digest • Analyzes your query log. • Reports the queries accounting for greatest load. • Slow queries. • Quick queries that run frequently. • Outputs a ranked list of top query patterns. $ pt-query-digest /var/lib/mysql/mysql-slow.log > ~/pqd.txt www.percona.comTuesday, December 4, 2012
  17. 17. Ranked Queries # Profile # Rank Query ID Response time Calls R/Call Apdx V/M Item # ==== ================== =============== ===== ======= ==== ===== ======= # 1 0xA8D2BBDE7EBE7822 4932.2992 28.8% 78 63.2346 0.00 5.22 SELECT person_info # 2 0xFE25DAF5DBB71F49 4205.2160 24.6% 130 32.3478 0.00 3.47 SELECT title # 3 0x70DAC639802CA233 1299.6269 7.6% 14 92.8305 0.00 0.17 SELECT cast_info # 4 0xE336B880F4FEC4B8 1184.5101 6.9% 294 4.0289 0.36 2.29 SELECT cast_info # 5 0x60550B93960F1837 905.1648 5.3% 60 15.0861 0.05 1.33 SELECT name # 6 0xF46D5C09B4E0CA2F 777.2446 4.5% 16340 0.0476 1.00 0.17 SELECT char_name # 7 0x09FCFFF0E5BC929F 747.4346 4.4% 130 5.7495 0.53 7.69 SELECT name # 8 0x9433950BE12B9470 744.1755 4.4% 14368 0.0518 1.00 0.18 SELECT name # 9 0x4DC0E044996DA715 448.5637 2.6% 130 3.4505 0.65 8.31 SELECT title # 10 0x09FB72D72ED18E93 361.1904 2.1% 78 4.6306 0.28 1.89 SELECT cast_info title www.percona.comTuesday, December 4, 2012
  18. 18. Slow Queries # Profile # Rank Query ID Response time Calls R/Call Apdx V/M Item # ==== ================== =============== ===== ======= ==== ===== ======= # 1 0xA8D2BBDE7EBE7822 4932.2992 28.8% 78 63.2346 0.00 5.22 SELECT person_info # 2 0xFE25DAF5DBB71F49 4205.2160 24.6% 130 32.3478 0.00 3.47 SELECT title # 3 0x70DAC639802CA233 1299.6269 7.6% 14 92.8305 0.00 0.17 SELECT cast_info # 4 0xE336B880F4FEC4B8 1184.5101 6.9% 294 4.0289 0.36 2.29 SELECT cast_info # 5 0x60550B93960F1837 905.1648 5.3% 60 15.0861 0.05 1.33 SELECT name # 6 0xF46D5C09B4E0CA2F 777.2446 4.5% 16340 0.0476 1.00 0.17 SELECT char_name # 7 0x09FCFFF0E5BC929F 747.4346 4.4% 130 5.7495 0.53 7.69 SELECT name # 8 0x9433950BE12B9470 744.1755 4.4% 14368 0.0518 1.00 0.18 SELECT name # 9 0x4DC0E044996DA715 448.5637 2.6% 130 3.4505 0.65 8.31 SELECT title # 10 0x09FB72D72ED18E93 361.1904 2.1% 78 4.6306 0.28 1.89 SELECT cast_info title this query executed a few times, but each takes over 1 minute www.percona.comTuesday, December 4, 2012
  19. 19. Quick Queries # Profile # Rank Query ID Response time Calls R/Call Apdx V/M Item # ==== ================== =============== ===== ======= ==== ===== ======= # 1 0xA8D2BBDE7EBE7822 4932.2992 28.8% 78 63.2346 0.00 5.22 SELECT person_info # 2 0xFE25DAF5DBB71F49 4205.2160 24.6% 130 32.3478 0.00 3.47 SELECT title # 3 0x70DAC639802CA233 1299.6269 7.6% 14 92.8305 0.00 0.17 SELECT cast_info # 4 0xE336B880F4FEC4B8 1184.5101 6.9% 294 4.0289 0.36 2.29 SELECT cast_info # 5 0x60550B93960F1837 905.1648 5.3% 60 15.0861 0.05 1.33 SELECT name # 6 0xF46D5C09B4E0CA2F 777.2446 4.5% 16340 0.0476 1.00 0.17 SELECT char_name # 7 0x09FCFFF0E5BC929F 747.4346 4.4% 130 5.7495 0.53 7.69 SELECT name # 8 0x9433950BE12B9470 744.1755 4.4% 14368 0.0518 1.00 0.18 SELECT name # 9 0x4DC0E044996DA715 448.5637 2.6% 130 3.4505 0.65 8.31 SELECT title # 10 0x09FB72D72ED18E93 361.1904 2.1% 78 4.6306 0.28 1.89 SELECT cast_info title another query executed frequently, each call is quick www.percona.comTuesday, December 4, 2012
  20. 20. Understanding Indexes by Analogy www.percona.comTuesday, December 4, 2012
  21. 21. This is a Telephone Book • A telephone book is like an index sorted by last name first, first name last. • Also includes the person’s phone number. Thomas Siencki http://en.wikipedia.org/wiki/File:Telefonbog_ubt-1.JPG www.percona.comTuesday, December 4, 2012
  22. 22. Implementation • In MySQL syntax, this would look like the following: CREATE INDEX phone_idx ON TelephoneBook (last_name, first_name, phone_number); www.percona.comTuesday, December 4, 2012
  23. 23. Simple Searches • Since the index is pre-sorted, it benefits queries for last name: ! SELECT * FROM TelephoneBook WHERE last_name = Smith; • By benefit, we mean that it helps to narrow down the search quickly, because the entries are already in sorted order. www.percona.comTuesday, December 4, 2012
  24. 24. Compound Searches • The sort order benefits us further if we search on last name and first name together: ! SELECT * FROM TelephoneBook WHERE last_name = Smith AND first_name = John; www.percona.comTuesday, December 4, 2012
  25. 25. When the Index Fails • But if we search only on first_name, the sort order doesn’t help: ! SELECT * FROM TelephoneBook WHERE first_name = John; • The ‘John’ entries occur throughout the book, in an unpredictable distribution. Anyone’s first name can be ‘John’. We have to search every page of the book. www.percona.comTuesday, December 4, 2012
  26. 26. Order Matters • From this, we conclude that the order we define the columns in an index matters. • An index with the columns declared in the opposite order would benefit a search for first name alone, but then it wouldn’t help a search for last name alone. CREATE INDEX phone_idx2 ON TelephoneBook (first_name, last_name, phone_number); www.percona.comTuesday, December 4, 2012
  27. 27. Both Indexes Can Be Helpful • If both criteria use equality comparisons, then the order of columns shouldn’t matter. • Either index would benefit a query that searches for a specific value in both columns. ! SELECT * FROM TelephoneBook WHERE last_name = Smith AND first_name = John; www.percona.comTuesday, December 4, 2012
  28. 28. Range Comparisons • Searching for multiple names is another case. • An index still helps us to narrow the search even if we search for all names that match a pattern: ! SELECT * FROM TelephoneBook WHERE last_name LIKE S%; • The book groups all the names starting with ‘S’ together, so the index can help us to avoid reading the whole book. www.percona.comTuesday, December 4, 2012
  29. 29. Compound Range Comparisons • If we search by a range of last names, but a specific first name, then the first names are again distributed in an unpredictable way among the matching last names: ! SELECT * FROM TelephoneBook WHERE last_name LIKE S% AND first_name = John; • The ‘John’ entries are not grouped together among all ‘S’ names, so we have to search manually every ‘S’ name to find the ones we want. www.percona.comTuesday, December 4, 2012
  30. 30. Order Matters • From this we see that once we do a range comparison on a column in an index, any criteria we have for subsequent columns cannot benefit from the index. CREATE INDEX phone_idx ON TelephoneBook (last_name, first_name, phone_number); this column in the index but then subsequent columns helps the range search do not help searching www.percona.comTuesday, December 4, 2012
  31. 31. Sorting by Index • Since the telephone book is pre-sorted, we can rely on this if we need to read through the entries in that order. ! SELECT * FROM TelephoneBook WHERE last_name = Smith ORDER BY first_name; • There’s no extra work to do here; we can simply read the entries in the order they are naturally written in the book. • So the ORDER BY is a no-op. www.percona.comTuesday, December 4, 2012
  32. 32. When the Index Can’t Help Sorting • Sorting by any other column is another story. ! SELECT * FROM TelephoneBook WHERE last_name = Smith ORDER BY phone_number; • The book is already ordered by last name and then by first name, but not numerically by phone number. • To get the result sorted in the manner above, we could copy the ‘Smith’ entries onto sticky notes, then re-order the sticky notes by phone number manually. www.percona.comTuesday, December 4, 2012
  33. 33. Order Matters • From this, we conclude that an index helps if we sort by the column immediately following the columns in the search criteria. • But if we need to sort by another column later in the index, or by a column that isn’t in the index at all, we have to do it the hard way. www.percona.comTuesday, December 4, 2012
  34. 34. Index-Only Searches • Once we find the matching entries, we may need other fields of information: ! SELECT phone_number FROM TelephoneBook WHERE last_name = Smith AND first_name = John; • Because the phone number is included in the index entry, we get it for free. The entry includes all three fields, even if our query didn’t search for the phone number explicitly. www.percona.comTuesday, December 4, 2012
  35. 35. When the Index Can’t Help • If we need other information that isn’t included in the index, we may need to do extra work. ! SELECT business_hours FROM TelephoneBook WHERE last_name = Smith Plumbing; • In this case, we’d have to use the phone number of the business, and call them to ask their hours. • That’s the extra work. www.percona.comTuesday, December 4, 2012
  36. 36. Columns Matter • From this, we conclude that putting columns in an index can be very important for certain queries, even if that query doesn’t use the columns for searching or sorting. • This is called a covering index. www.percona.comTuesday, December 4, 2012
  37. 37. Lesson #2 rate your indexes using the star system. www.percona.comTuesday, December 4, 2012
  38. 38. The Star System • Rate an index with respect to a given query by three “stars” which describe effectiveness. • This technique is covered in Relational Database Index Design and the Optimizers by Tapio Lahdenmäki and Michael Leach. • http://www.wiley.com/WileyCDA/WileyTitle/ productCd-0471719994.html www.percona.comTuesday, December 4, 2012
  39. 39. The Star System ★ First star: Rows referenced by your query are grouped together in the index. ★ Second star: Rows referenced by your query are ordered in the index the way you want them. ★ Third star: The index contains all columns referenced by your query (covering index). www.percona.comTuesday, December 4, 2012
  40. 40. Look at this Terrible Query mysql> EXPLAIN SELECT person_id, person_role_id FROM cast_info WHERE movie_id = 91280 AND role_id = 1 ORDER BY nr_order ASCG id: 1 scans whole table select_type: SIMPLE table: cast_info uses no index type: ALL possible_keys: NULL scans a lot of rows key: NULL key_len: NULL sorts the hard way ref: NULL rows: 24008907 Extra: Using where; Using filesort www.percona.comTuesday, December 4, 2012
  41. 41. ★ First Star ★ Pick all columns from equality predicates. Define the index with these columns first. ! SELECT person_id, person_role_id FROM cast_info WHERE movie_id = 91280 AND role_id = 1 ORDER BY nr_order ASC; equality predicates ALTER TABLE cast_info ADD INDEX (movie_id, role_id); www.percona.comTuesday, December 4, 2012
  42. 42. ★ Second Star ★ Add the column(s) in the GROUP BY or ORDER BY clause, if the query has one. ! SELECT person_id, person_role_id FROM cast_info WHERE movie_id = 91280 AND role_id = 1 ORDER BY nr_order ASC; sorting column ALTER TABLE cast_info ADD INDEX (movie_id, role_id, nr_order); www.percona.comTuesday, December 4, 2012
  43. 43. ★ Third Star ★ Add any remaining columns referenced in the SELECT list. ! SELECT person_id, person_role_id FROM cast_info WHERE movie_id = 91280 AND role_id = 1 ORDER BY nr_order ASC; select-list columns ALTER TABLE cast_info ADD INDEX (movie_id, role_id, nr_order, person_id, person_role_id); www.percona.comTuesday, December 4, 2012
  44. 44. Explain That mysql> EXPLAIN SELECT person_id, person_role_id FROM cast_info WHERE movie_id = 91280 AND role_id = 1 ORDER BY nr_order ASCG ★1: effective index id: 1 select_type: SIMPLE table: cast_info ★2: no filesort type: ref possible_keys: movie_id key: movie_id ★3: covering index key_len: 8 ref: const,const rows: 57 Extra: Using where; Using index www.percona.comTuesday, December 4, 2012
  45. 45. Complications • Can’t achieve First-Star indexes when: • WHERE clause has more complex expressions including OR predicates (disjunction). ! SELECT * FROM TelephoneBook WHERE last_name = Smith OR first_name = John disjunction www.percona.comTuesday, December 4, 2012
  46. 46. Complications • Can’t achieve Second-Star indexes when: • WHERE clause includes range predicates (>, <, !=, NOT IN, BETWEEN, LIKE, etc.). • Query includes both GROUP BY and ORDER BY referencing different columns. • Query must sort by multiple columns over different tables (can’t make a single index span tables). • Query includes ORDER BY in a different order than the order in which rows are accessed. www.percona.comTuesday, December 4, 2012
  47. 47. Complications • Can’t achieve Third-Star indexes when: • Query requires more columns than the maximum of 16 columns per index. • Query requires BLOB, TEXT, or VARCHAR columns longer than the maximum of 1000 bytes per index (utf8 counts three bytes per character). www.percona.comTuesday, December 4, 2012
  48. 48. Example Query www.percona.comTuesday, December 4, 2012
  49. 49. Example Query mysql> EXPLAIN SELECT STRAIGHT_JOIN n.* FROM char_name AS c INNER JOIN cast_info AS i ! ON c.id=i.person_role_id INNER JOIN name AS n ! ON i.person_id=n.id WHERE c.name = James Bond ORDER BY n.nameG char_name cast_info name www.percona.comTuesday, December 4, 2012
  50. 50. Not Optimized *************************** 1. row *************************** id: 1 select_type: SIMPLE table: c type: ALL possible_keys: PRIMARY 2.4 million × 21.7 million key: NULL key_len: NULL = 52 trillion comparisons ref: NULL rows: 2402968 Extra: Using where; Using temporary; Using filesort *************************** 2. row *************************** id: 1 select_type: SIMPLE table: i type: index possible_keys: person_id key: person_id key_len: 9 ref: NULL rows: 21676661 Extra: Using where; Using index; Using join buffer *************************** 3. row *************************** id: 1 select_type: SIMPLE table: n type: eq_ref www.percona.comTuesday, December 4, 2012
  51. 51. Add Indexes ★ First-star index: mysql> ALTER TABLE char_name ADD INDEX n (name(20)); ★ Third-star index: mysql> ALTER TABLE cast_info ADD INDEX pr_p (person_role_id, person_id); char_name cast_info name www.percona.comTuesday, December 4, 2012
  52. 52. Optimized *************************** 1. row *************************** id: 1 select_type: SIMPLE table: c type: ref possible_keys: PRIMARY,n improved by a factor key: n key_len: 62 of 480 million! ref: const rows: 1 Extra: Using where; Using temporary; Using filesort *************************** 2. row *************************** id: 1 select_type: SIMPLE table: i type: ref possible_keys: pr_p key: pr_p key_len: 5 ref: imdb.c.id rows: 108383 Extra: Using where; Using index *************************** 3. row *************************** id: 1 select_type: SIMPLE table: n type: eq_ref www.percona.comTuesday, December 4, 2012
  53. 53. Why Not Second Star? • Not possible to create a second-star index here. • The order of the join column isn’t necessarily the same as the order of the sort column: +---------+--------------------------+ | id | name | +---------+--------------------------+ | 127817 | Connery, Sean | | 201693 | Lazenby, George | | 213877 | Moore, Roger | | 228614 | Dalton, Timothy | | 581060 | Brosnan, Pierce | | 707448 | Craig, Daniel | +---------+--------------------------+ www.percona.comTuesday, December 4, 2012
  54. 54. Lesson #3 tidy up indexes you don’t need. www.percona.comTuesday, December 4, 2012
  55. 55. Indexes You Don’t Need? create drop indexes indexes that help your and that don’t help your queries queries www.percona.comTuesday, December 4, 2012
  56. 56. Redundant Indexes pt-duplicate-key-checker http://www.percona.com/doc/percona-toolkit/pt-duplicate-key-checker.html www.percona.comTuesday, December 4, 2012
  57. 57. Redundant Indexes mysql> create table foo (a int, b int, key (a), key (b), key (a,b)); $ pt-duplicate-key-checker h=localhost # ############################################################# # test.foo # ############################################################# # a is a left-prefix of a_2 # Key definitions: # KEY `a` (`a`), # KEY `a_2` (`a`,`b`) # Column types: #! `a` int(11) default null #! `b` int(11) default null # To remove this duplicate index, execute: ALTER TABLE `test`.`foo` DROP INDEX `a`; www.percona.comTuesday, December 4, 2012
  58. 58. Unused Indexes pt-index-usage http://www.percona.com/doc/percona-toolkit/pt-index-usage.html www.percona.comTuesday, December 4, 2012
  59. 59. Unused Indexes $ sudo pt-index-usage /var/lib/mysql/mysql-slow.log /var/lib/mysql/mysql-slow.log: 12% 01:07 remain /var/lib/mysql/mysql-slow.log: 25% 00:57 remain /var/lib/mysql/mysql-slow.log: 37% 00:49 remain /var/lib/mysql/mysql-slow.log: 49% 00:41 remain /var/lib/mysql/mysql-slow.log: 61% 00:31 remain /var/lib/mysql/mysql-slow.log: 73% 00:21 remain /var/lib/mysql/mysql-slow.log: 84% 00:12 remain /var/lib/mysql/mysql-slow.log: 97% 00:02 remain . . . ALTER TABLE `imdb`.`cast_info` DROP KEY `movie_id`; -- type:non-unique . . . www.percona.comTuesday, December 4, 2012
  60. 60. Index Usage Statistics Index Statistics in Percona Server http://www.percona.com/doc/percona-server/5.5/diagnostics/user_stats.html www.percona.comTuesday, December 4, 2012
  61. 61. Index Usage Statistics • INFORMATION_SCHEMA.INDEX_STATISTICS • Enable with: mysql> SET GLOBAL userstat = ON; • Statistics are cleared when you restart mysqld. • Negligible performance impact: • http://www.mysqlperformanceblog.com/2012/06/02/how-expensive-is- user_statistics/ www.percona.comTuesday, December 4, 2012
  62. 62. Index Usage Statistics • Find indexes that were never used: mysql> SELECT CONCAT( ! ALTER TABLE `, s.table_schema, `.`, s.table_name, ! DROP KEY `, s.index_name, `;) AS ddl FROM INFORMATION_SCHEMA.STATISTICS AS s LEFT OUTER JOIN INFORMATION_SCHEMA.INDEX_STATISTICS AS i ! USING (table_schema, table_name, index_name) WHERE s.table_schema = imdb AND i.rows_read IS NULL; +---------------------------------------------------------+ | ddl | +---------------------------------------------------------+ | ALTER TABLE `imdb`.`cast_info DROP KEY `movie_id`; | | ALTER TABLE `imdb`.`cast_info DROP KEY `person_id`; | | ALTER TABLE `imdb`.`comp_cast_type DROP KEY `kind`; | | ALTER TABLE `imdb`.`company_type DROP KEY `kind`; | . . . www.percona.comTuesday, December 4, 2012
  63. 63. Lesson #4 make it a habit. www.percona.comTuesday, December 4, 2012
  64. 64. Review Queries Regularly • Repeat the query analysis periodically: • As new application code introduces new queries. • As the volume of data grows, making trivial queries more expensive. • As site traffic changes, making some queries more frequent. www.percona.comTuesday, December 4, 2012
  65. 65. Review Queries Regularly • Track query types you have reviewed before: • pt-query-digest --review Saves query types to a table, so you can add notes to them, mark them as reviewed, omit them from future reports. • pt-query-digest --review-history Saves query performance statistics to a table, so you can analyze trends in the data over time. www.percona.comTuesday, December 4, 2012
  66. 66. Review Indexes Regularly • Run pt-duplicate-key-checker periodically: • After schema changes. • You can run this on a development or test instance. www.percona.comTuesday, December 4, 2012
  67. 67. Review Indexes Regularly • Run pt-index-usage or review INDEX_STATISTICS periodically: • On a regular schedule (e.g. weekly). • Especially after application code changes. www.percona.comTuesday, December 4, 2012
  68. 68. Lessons Redux identify rate your queries. indexes. make it a tidy up. habit. www.percona.comTuesday, December 4, 2012
  69. 69. Percona Training for MySQL Senior Industry Experts In-Person and Online Classes Custom Onsite Training http://percona.com/training www.percona.comTuesday, December 4, 2012
  70. 70. Visit Our Sponsors diamond platinum exhibitors and the rest! www.percona.comTuesday, December 4, 2012
  71. 71. SQL Antipatterns http://www.pragprog.com/titles/bksqla/ www.percona.comTuesday, December 4, 2012
  72. 72. License and Copyright Copyright 2012 Bill Karwin www.slideshare.net/billkarwin Released under a Creative Commons 3.0 License: http://creativecommons.org/licenses/by-nc-nd/3.0/ You are free to share - to copy, distribute and transmit this work, under the following conditions: Attribution. Noncommercial. No Derivative Works. You must attribute this work You may not use this work You may not alter, transform, to Bill Karwin. for commercial purposes. or build upon this work. www.percona.comTuesday, December 4, 2012
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×