Advanced MySQL Query Tuning
Alexander Rubin
July 21, 2013
www.percona.com
About Me
•  Working with MySQL for over 10 years
–  Started at MySQL AB, then Sun Microsystems,
–  then Or...
www.percona.com
§  Indexes
– How B-tree works
– Range scans
§  Queries
– Temporary Tables and Filesort in MySQL
– GROUP ...
www.percona.com
How to Deal with Slow Performance
Indexes
www.percona.com
MySQL Index Types: B-Tree
•  When you add index (except for MEMORY)
MySQL will use B-Tree
•  Support equal...
www.percona.com
MySQL Index Types: B-Tree
•  Scan thru the tree and go directly to 1 leaf
•  Stop
Equality search: select ...
www.percona.com
MySQL Index Types: B-Tree
•  Scan thru the tree and visit many leafs/nodes
Range: select * from table wher...
www.percona.com
CREATE TABLE City (!
ID int(11) NOT NULL AUTO_INCREMENT,!
Name char(35) NOT NULL DEFAULT '',!
CountryCode ...
www.percona.com
• MySQL will use 1 (best) index
mysql> explain select * from City where ID = 1;!
+-------+-------+--------...
www.percona.com
•  Leftmost part of combined index
mysql> alter table City add key !
comb(CountryCode, District, Populatio...
www.percona.com
• Leftmost part of combined index
mysql> explain select * from City !
where CountryCode = 'USA'G!
********...
www.percona.com
•  Key_len = total size (in bytes) of index parts used
Index: comb(CountryCode, District, Population)!
!
!...
www.percona.com
• 2 Leftmost Fields
mysql> explain select * from City !
where CountryCode = 'USA' and District = 'Californ...
www.percona.com
• 3 Leftmost Fields
mysql> explain select * from City !
where CountryCode = 'USA' and District = 'Californ...
www.percona.com
• Can’t use combined index – not a leftmost part
mysql> explain select * from City where !
District = 'Cal...
www.percona.com
• Covered index = cover all fields in query
select name from City where CountryCode = 'USA'
and District =...
www.percona.com
• Explain
mysql> explain select name from City where CountryCode =
'USA' and District = 'Alaska' and popul...
www.percona.com
Order of Fields in Index
•  select * from City where district = 'California' and
population > 30000
•  Ind...
www.percona.com
Order of Fields in Index: Example
mysql> alter table City add key comb1(district, population);!
!
mysql> e...
www.percona.com
Order of Fields in Index: Example
mysql> alter table City add key comb2(population, district);!
!
mysql> e...
www.percona.com
Simplified BTree Scan Example I
Root
CA
10K –
15K
… …
100K-105K
… …
… … NC
Comb1(district,population)
1.  ...
www.percona.com
Simplified BTree Scan Example II
Root
10K –
15K
CA
…
NC
… …
100K
-105K
CA
…
NC
Comb1(population,district)
...
www.percona.com
Index Cardinality: Example
mysql> alter table City !
add key comb2(population, District);!
!
explain selec...
www.percona.com
How to Deal with Slow Performance
Queries
www.percona.com
Complex Slow Queries
The World’s Most Popular Open Source Database
… Group By …
… Order By …
Select distin...
www.percona.com
GROUP BY Queries
www.percona.com
mysql> explain select CountryCode, count(*) from City
group by CountryCodeG
id: 1
select_type: SIMPLE
tabl...
www.percona.com
Temporary Tables: Theory
www.percona.com
•  MySQL can create temporary tables when query uses:
•  GROUP BY
•  Range + ORDER BY
•  Some other expres...
www.percona.com
•  First, MySQL tries to create temporary table in memory
•  MySQL configuration variables:
•  tmp_table_s...
www.percona.com
MySQL
temp table
>
tmp_table_
size
OR
MySQL
temp table
>
max_heap_
table_size
convert to
MyISAM
temporary
...
www.percona.com
•  MEMORY engine does not support BLOB/TEXT
•  select blob_field from table group by field1
•  select conc...
www.percona.com
Temporary Tables: Practice
www.percona.com
6M rows, ~2G in size
CREATE TABLE ontime_2012 (
YearD int(11) DEFAULT NULL,
MonthD tinyint(4) DEFAULT NULL...
www.percona.com
SELECT max(DepDelayMinutes),
carrier, dayofweek
FROM ontime_2012
WHERE dayofweek = 7
GROUP BY Carrier
•  F...
www.percona.com
select max(DepDelayMinutes), carrier, dayofweek
from ontime_2012
where dayofweek = 7
group by Carrier
type...
www.percona.com
mysql> alter table ontime_2012 add key (dayofweek);
explain select max(DepDelayMinutes), Carrier,
dayofwee...
www.percona.com
mysql> alter table ontime_2012
add key covered(dayofweek, Carrier, DepDelayMinutes);
explain select max(De...
www.percona.com
mysql> explain select max(DepDelayMinutes), Carrier,
dayofweek from ontime_2012
where dayofweek > 3 group ...
www.percona.com
(select max(DepDelayMinutes), Carrier, dayofweek
from ontime_2012
where dayofweek = 3
group by Carrier, da...
www.percona.com
*************************** 1. row ***************************
table: ontime_2012
key: covered
...
Extra: ...
www.percona.com
GROUP BY: Loose index scan
•  Loose index scan:
•  considers only a fraction of the keys in an
index
•  Fo...
www.percona.com
Loose index scan example
mysql> alter table ontime_2012 add key loose_index_scan
(Carrier, dayofweek, DepD...
www.percona.com
GROUP BY: Tight index scan
•  Full index scan or a range index scan
•  Can work if loose index scan can t ...
www.percona.com
Loose index scan vs. tight index scan
Table: ontime_2012, 6M rows, data: 2G, index: 210M
CREATE TABLE onti...
www.percona.com
Loose index scan vs. tight index scan
Loose index scan
select max(DepDelayMinutes) as ddm, Carrier, dayofw...
www.percona.com
Loose index scan vs. tight index scan
Loose index scan
select max(DepDelayMinutes) as ddm, Carrier, dayofw...
www.percona.com
Loose index scan vs. tight index scan
Loose index scan
select max(DepDelayMinutes) as ddm, Carrier, dayofw...
www.percona.com
Loose index scan vs. tight index scan
Temporary table
select max(DepDelayMinutes) as ddm, Carrier, dayofwe...
www.percona.com
Loose index scan vs. tight index scan
Tight Index Scan
select max(DepDelayMinutes) as ddm, Carrier, dayofw...
www.percona.com
Loose index scan vs. tight index scan
Results
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Loose index scan Tight Index...
www.percona.com
Where loose index scan is not supported
•  AVG() + Group By – loose index scan is not supported
mysql> exp...
www.percona.com
ORDER BY and filesort
www.percona.com
mysql> explain select district, name, population from City
where CountryCode = 'USA' order by population d...
www.percona.com
mysql> alter table City
add key my_sort2 (CountryCode, population);
mysql> explain select district, name, ...
www.percona.com
mysql> alter table ontime_2012 add key (DepDelayMinutes);
Query OK, 0 rows affected (38.68 sec)
mysql> exp...
www.percona.com
If Index points to the beginning of the table (physically) = fast
As it stops after 10 rows (LIMIT 10)
www.percona.com
If Index points to the end of table (physically) or random = slower
Much more rows to scan (and skip)
www.percona.com
Example: Social Graph
www.percona.com
https://blog.twitter.com/2010/introducing-flockdb
•  Source_id = <YOU>
•  Destination_id = IDs of people f...
www.percona.com
CREATE TABLE social_graph (
source_id bigint(20) unsigned NOT NULL,
destination_id bigint(20) unsigned NOT...
www.percona.com
mysql> explain select * from social_graph where source_id = 927
and state in (0) order by position desc li...
www.percona.com
mysql> alter table social_graph add key
src_state_position(source_id, state, position);
mysql> select * fr...
www.percona.com
mysql> explain select * from social_graph where
source_id = 927 and state in (0) order by position
desc li...
www.percona.com
mysql> explain select * from where source_id = 927
and state in (0,2) order by position desc limit 100G
se...
www.percona.com
mysql> alter table social_graph
add key src_position(source_id, position);
mysql> explain select * from so...
www.percona.com
mysql> explain select * from social_graph where
source_id = 927 and state in (1,2) order by position
desc ...
www.percona.com
mysql> explain select * from social_graph
ignore index (src_state_position)
where source_id = 927 and stat...
www.percona.com
Questions?
Thank you!
Upcoming SlideShare
Loading in …5
×

Webinar 2013 advanced_query_tuning

6,080 views

Published on

Advanced MySQL Query Tuning from
Percona Database Performance for share.
https://www.youtube.com/watch?v=TPFibi2G_oo

Published in: Software
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,080
On SlideShare
0
From Embeds
0
Number of Embeds
5,602
Actions
Shares
0
Downloads
33
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Webinar 2013 advanced_query_tuning

  1. 1. Advanced MySQL Query Tuning Alexander Rubin July 21, 2013
  2. 2. www.percona.com About Me •  Working with MySQL for over 10 years –  Started at MySQL AB, then Sun Microsystems, –  then Oracle (MySQL Consulting) –  Joined Percona recently •  Helping customers improve MySQL performance –  performance tuning –  full text search –  high availability –  Reporting, database infrastructure scale-outs –  Big data My name is Alexander Rubin
  3. 3. www.percona.com §  Indexes – How B-tree works – Range scans §  Queries – Temporary Tables and Filesort in MySQL – GROUP BY Optimizations – ORDER BY Optimizations §  Social Graph Example Agenda
  4. 4. www.percona.com How to Deal with Slow Performance Indexes
  5. 5. www.percona.com MySQL Index Types: B-Tree •  When you add index (except for MEMORY) MySQL will use B-Tree •  Support equality and “range” operations Default index type (except for MEMORY tables) http://en.wikipedia.org/wiki/B-tree
  6. 6. www.percona.com MySQL Index Types: B-Tree •  Scan thru the tree and go directly to 1 leaf •  Stop Equality search: select * from table where id = 12 http://en.wikipedia.org/wiki/B-tree
  7. 7. www.percona.com MySQL Index Types: B-Tree •  Scan thru the tree and visit many leafs/nodes Range: select * from table where id in (6, 12, 18) http://en.wikipedia.org/wiki/B-tree
  8. 8. www.percona.com CREATE TABLE City (! ID int(11) NOT NULL AUTO_INCREMENT,! Name char(35) NOT NULL DEFAULT '',! CountryCode char(3) NOT NULL DEFAULT '',! District char(20) NOT NULL DEFAULT '',! Population int(11) NOT NULL DEFAULT '0',! PRIMARY KEY (ID),! KEY CountryCode (CountryCode)! ) Engine=InnoDB;!
  9. 9. www.percona.com • MySQL will use 1 (best) index mysql> explain select * from City where ID = 1;! +-------+-------+---------------+---------+---------+-------+------+-------+! | table | type | possible_keys | key | key_len | ref | rows | Extra |! +-------+-------+---------------+---------+---------+-------+------+-------+! | City | const | PRIMARY | PRIMARY | 4 | const | 1 | |! +-------+-------+---------------+---------+---------+-------+------+-------+! ! mysql> explain select * from City where CountryCode = 'USA';! +-------+------+---------------+-------------+---------+-------+------+------------+! | table | type | possible_keys | key | key_len | ref | rows | Extra |! +-------+------+---------------+-------------+---------+-------+------+------------+! | City | ref | CountryCode | CountryCode | 3 | const | 274 | Using where|! +-------+------+---------------+-------------+---------+-------+------+------------+! !
  10. 10. www.percona.com •  Leftmost part of combined index mysql> alter table City add key ! comb(CountryCode, District, Population), ! drop key CountryCode;! ! ! ! !
  11. 11. www.percona.com • Leftmost part of combined index mysql> explain select * from City ! where CountryCode = 'USA'G! ********************** 1. row ******************! table: City! type: ref! possible_keys: comb! key: comb! key_len: 3! ref: const! rows: 273! ! Uses first field from the comb key
  12. 12. www.percona.com •  Key_len = total size (in bytes) of index parts used Index: comb(CountryCode, District, Population)! ! ! ! Explain: key: comb! key_len: 3! ! ! Fields: CountryCode char(3)! District char(20) ! Population int(11)! ! ! ! 3 -> Char(3) -> First field is used
  13. 13. www.percona.com • 2 Leftmost Fields mysql> explain select * from City ! where CountryCode = 'USA' and District = 'California'G! ********************** 1. row ******************! table: City! type: ref! possible_keys: comb! key: comb! key_len: 23! ref: const,const! rows: 68! ! Uses 2 first fields from the comb key CountryCode = 3 chars District = 20 chars Total = 23
  14. 14. www.percona.com • 3 Leftmost Fields mysql> explain select * from City ! where CountryCode = 'USA' and District = 'California’! and population > 10000G! ********************** 1. row ******************! table: City! type: range! possible_keys: comb! key: comb! key_len: 27! ref: NULL! rows: 68! Uses all fields from the comb key CountryCode = 3 chars/bytes District = 20 chars/bytes Population = 4 bytes (INT) Total = 27
  15. 15. www.percona.com • Can’t use combined index – not a leftmost part mysql> explain select * from City where ! District = 'California' and population > 10000G! ********************** 1. row ******************! table: City! type: ALL! possible_keys: NULL! key: NULL! key_len: NULL! ref: NULL! rows: 3868 ! Does not have the CountryCode in the where clause = can’t use comb index
  16. 16. www.percona.com • Covered index = cover all fields in query select name from City where CountryCode = 'USA' and District = 'Alaska' and population > 10000! ! mysql> alter table City add key ! cov1(CountryCode, District, population, name);! ! ! Uses all fields in the query in particular order: 1.  Where part 2.  Group By/Order (not used now) 3.  Select part (here: name)
  17. 17. www.percona.com • Explain mysql> explain select name from City where CountryCode = 'USA' and District = 'Alaska' and population > 10000G! *************************** 1. row ***********! table: City! type: range! possible_keys: cov1! key: cov1! key_len: 27! ref: NULL! rows: 1! Extra: Using where; Using index! ! Using index = covered index is used MySQL will only use index Will not go to the data file
  18. 18. www.percona.com Order of Fields in Index •  select * from City where district = 'California' and population > 30000 •  Index (district, population) in this order •  Rule of thumb: “Const” first, “Range” second –  Depends on query Range and “const” scans: use “effective” cardinality
  19. 19. www.percona.com Order of Fields in Index: Example mysql> alter table City add key comb1(district, population);! ! mysql> explain select name from City where ! district = 'California' and population > 10000G! ********************** 1. row ***************************! table: City! type: range! possible_keys: comb1! key: comb1! key_len: 24! ref: NULL! rows: 68! ! Good: Index is used to restrict rows Key_len = 24 – both fields used
  20. 20. www.percona.com Order of Fields in Index: Example mysql> alter table City add key comb2(population, district);! ! mysql> explain select name from City where ! district = 'California' and population > 3000G! ! ! ! ! ! table: City! type: ALL! possible_keys: comb2! key: NULL! key_len: NULL! ref: NULL! rows: 4162! Extra: Using where! BAD! MySQL decided not to use index at all Why? MySQL can only use “population” part Too many cities with population > 3000
  21. 21. www.percona.com Simplified BTree Scan Example I Root CA 10K – 15K … … 100K-105K … … … … NC Comb1(district,population) 1.  Go “directly”* to the district (CA) 2.  Do range scan by population starting with “CA” *via index scan
  22. 22. www.percona.com Simplified BTree Scan Example II Root 10K – 15K CA … NC … … 100K -105K CA … NC Comb1(population,district) 1.  Do range scan by population 2.  For each scanned index record Check for correct district (CA) 3.  = Only use “population” part of the index
  23. 23. www.percona.com Index Cardinality: Example mysql> alter table City ! add key comb2(population, District);! ! explain select name from City where ! District = 'California' and population > 1000000G! *********************** 1. row ***************************! table: City! type: range! possible_keys: comb2! key: comb2! key_len: 4! ref: NULL! rows: 237! Extra: Using where! Uses Index BUT: key_len = 4 (INT) Only population part is used Cardinality = number of unique values
  24. 24. www.percona.com How to Deal with Slow Performance Queries
  25. 25. www.percona.com Complex Slow Queries The World’s Most Popular Open Source Database … Group By … … Order By … Select distinct … Filesort Temporary tables
  26. 26. www.percona.com GROUP BY Queries
  27. 27. www.percona.com mysql> explain select CountryCode, count(*) from City group by CountryCodeG id: 1 select_type: SIMPLE table: City type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 4079 Extra: Using temporary; Using filesort Temporary tables are slow! How many cities in each country?
  28. 28. www.percona.com Temporary Tables: Theory
  29. 29. www.percona.com •  MySQL can create temporary tables when query uses: •  GROUP BY •  Range + ORDER BY •  Some other expressions Main performance issues •  2 types of temporary tables •  MEMORY •  On-disk
  30. 30. www.percona.com •  First, MySQL tries to create temporary table in memory •  MySQL configuration variables: •  tmp_table_size •  maximum size for in Memory temporary tables •  max_heap_table_size •  Sets the maximum size for MEMORY tables
  31. 31. www.percona.com MySQL temp table > tmp_table_ size OR MySQL temp table > max_heap_ table_size convert to MyISAM temporary table on disk
  32. 32. www.percona.com •  MEMORY engine does not support BLOB/TEXT •  select blob_field from table group by field1 •  select concat(...string>512 chars) group by field1 • Create on-disk temporary table right away •  Percona server uses the new MEMORY engine with BLOB/TEXT Support •  BUT: it is not used for the temp tables
  33. 33. www.percona.com Temporary Tables: Practice
  34. 34. www.percona.com 6M rows, ~2G in size CREATE TABLE ontime_2012 ( YearD int(11) DEFAULT NULL, MonthD tinyint(4) DEFAULT NULL, DayofMonth tinyint(4) DEFAULT NULL, DayOfWeek tinyint(4) DEFAULT NULL, Carrier char(2) DEFAULT NULL, Origin char(5) DEFAULT NULL, DepDelayMinutes int(11) DEFAULT NULL, ... ) ENGINE=InnoDB DEFAULT CHARSET=latin1 http://www.transtats.bts.gov/DL_SelectFields.asp? Table_ID=236&DB_Short_Name=On-Time
  35. 35. www.percona.com SELECT max(DepDelayMinutes), carrier, dayofweek FROM ontime_2012 WHERE dayofweek = 7 GROUP BY Carrier •  Find maximum delay for flights on Sunday •  Group by airline
  36. 36. www.percona.com select max(DepDelayMinutes), carrier, dayofweek from ontime_2012 where dayofweek = 7 group by Carrier type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 4833086 Extra: Using where; Using temporary; Using filesort Full table scan! Temporary table!
  37. 37. www.percona.com mysql> alter table ontime_2012 add key (dayofweek); explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2012 where dayofweek =7 group by CarrierG type: ref possible_keys: DayOfWeek key: DayOfWeek key_len: 2 ref: const rows: 817258 Extra: Using where; Using temporary; Using filesort Index is used = better BUT: Large temporary table!
  38. 38. www.percona.com mysql> alter table ontime_2012 add key covered(dayofweek, Carrier, DepDelayMinutes); explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2012 where dayofweek =7 group by CarrierG ... possible_keys: DayOfWeek,covered key: covered key_len: 2 ref: const rows: 905138 Extra: Using where; Using index •  Called “tight index scan” No temporary table! MySQL will only use index
  39. 39. www.percona.com mysql> explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2012 where dayofweek > 3 group by Carrier, dayofweekG ... type: range possible_keys: covered key: covered key_len: 2 ref: NULL rows: 2441781 Extra: Using where; Using index; Using temporary; Using filesort Range scan
  40. 40. www.percona.com (select max(DepDelayMinutes), Carrier, dayofweek from ontime_2012 where dayofweek = 3 group by Carrier, dayofweek) union (select max(DepDelayMinutes), Carrier, dayofweek from ontime_2012 where dayofweek = 4 group by Carrier, dayofweek)
  41. 41. www.percona.com *************************** 1. row *************************** table: ontime_2012 key: covered ... Extra: Using where; Using index *************************** 2. row *************************** table: ontime_2012 key: covered … Extra: Using where; Using index *************************** 3. row *************************** id: NULL select_type: UNION RESULT table: <union1,2> type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: NULL Extra: Using temporary
  42. 42. www.percona.com GROUP BY: Loose index scan •  Loose index scan: •  considers only a fraction of the keys in an index •  Following rules apply: •  The query is over a single table. •  The GROUP BY names only columns that form a leftmost prefix of the index and no other columns. •  The only aggregate functions used in the select list (if any) are MIN() and MAX(), same column
  43. 43. www.percona.com Loose index scan example mysql> alter table ontime_2012 add key loose_index_scan (Carrier, dayofweek, DepDelayMinutes); mysql> explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2012 where dayofweek > 5 group by Carrier, dayofweekG ... table: ontime_2012 type: range possible_keys: NULL key: loose_index_scan key_len: 5 ref: NULL rows: 201 Extra: Using where; Using index for group-by Loose index scan Very fast “Range” works!
  44. 44. www.percona.com GROUP BY: Tight index scan •  Full index scan or a range index scan •  Can work if loose index scan can t be used •  Allow to scan index and AVOID creating tmp table •  Find ALL keys => group •  = Covered index
  45. 45. www.percona.com Loose index scan vs. tight index scan Table: ontime_2012, 6M rows, data: 2G, index: 210M CREATE TABLE ontime_2012 ( YearD int(11) DEFAULT NULL, MonthD tinyint(4) DEFAULT NULL, DayofMonth tinyint(4) DEFAULT NULL, DayOfWeek tinyint(4) DEFAULT NULL, Carrier char(2) DEFAULT NULL, Origin char(5) DEFAULT NULL, DepDelayMinutes int(11) DEFAULT NULL, ... KEY loose_index_scan (Carrier,DayOfWeek,DepDelayMinutes), KEY covered (DayOfWeek,Carrier,DepDelayMinutes) ) ENGINE=InnoDB DEFAULT CHARSET=latin1
  46. 46. www.percona.com Loose index scan vs. tight index scan Loose index scan select max(DepDelayMinutes) as ddm, Carrier, dayofweek from ontime_2012 where dayofweek = 5 group by Carrier, dayofweek table: ontime_2012 type: range possible_keys: covered key: loose_index_scan key_len: 5 ref: NULL rows: 201 Extra: Using where; Using index for group-by mysql> select ... +------+---------+-----------+ | ddm | Carrier | dayofweek | +------+---------+-----------+ | 1606 | AA | 7 | .. 30 rows in set (0.00 sec) Carrier, DayOfWeek, DepDelayMinutes
  47. 47. www.percona.com Loose index scan vs. tight index scan Loose index scan select max(DepDelayMinutes) as ddm, Carrier, dayofweek from ontime_2012 where dayofweek > 5 group by Carrier, dayofweek; table: ontime_2012 type: range possible_keys: covered key: loose_index_scan key_len: 5 ref: NULL rows: 213 Extra: Using where; Using index for group-by; mysql> select ... +------+---------+-----------+ | ddm | Carrier | dayofweek | +------+---------+-----------+ | 1606 | AA | 7 | .. 30 rows in set (0.00 sec) Carrier, DayOfWeek, DepDelayMinutes
  48. 48. www.percona.com Loose index scan vs. tight index scan Loose index scan select max(DepDelayMinutes) as ddm, Carrier, dayofweek from ontime_2012 where dayofweek > 5 group by Carrier, dayofweek order by ddm desc; table: ontime_2012 type: range possible_keys: covered key: loose_index_scan key_len: 5 ref: NULL rows: 213 Extra: Using where; Using index for group-by; Using temporary; Using filesort mysql> select ... +------+---------+-----------+ | ddm | Carrier | dayofweek | +------+---------+-----------+ | 1606 | AA | 7 | .. 30 rows in set (0.00 sec) Carrier, DayOfWeek, DepDelayMinutes
  49. 49. www.percona.com Loose index scan vs. tight index scan Temporary table select max(DepDelayMinutes) as ddm, Carrier, dayofweek from ontime_2012 ignore index(loose_index_scan) where dayofweek > 5 group by Carrier, dayofweek table: ontime_2012 type: range key: covered key_len: 2 ref: NULL rows: 2302412 Extra: Using where; Using index; Using temporary; Using filesort mysql> select ... +------+---------+-----------+ | ddm | Carrier | dayofweek | +------+---------+-----------+ | 1606 | AA | 7 | ... 30 rows in set (1.30 sec) 1.  Range scan 2.  Temp table and filesort of 2M rows
  50. 50. www.percona.com Loose index scan vs. tight index scan Tight Index Scan select max(DepDelayMinutes) as ddm, Carrier, dayofweek from ontime_2012 ignore index(loose_index_scan) where dayofweek = 5 group by Carrier, dayofweek table: ontime_2012 type: ref key: covered key_len: 2 ref: NULL rows: 2302412 Extra: Using where; Using index; mysql> select ... +------+---------+-----------+ | ddm | Carrier | dayofweek | +------+---------+-----------+ | 1606 | AA | 7 | ... 30 rows in set (0.37 sec) 1.  Covered index 2.  No temp table 3.  BUT: have to scan 2M rows DayOfWeek, Carrier, DepDelayMinutes
  51. 51. www.percona.com Loose index scan vs. tight index scan Results 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 Loose index scan Tight Index Scan Temporary table
  52. 52. www.percona.com Where loose index scan is not supported •  AVG() + Group By – loose index scan is not supported mysql> explain select avg(DepDelayMinutes) as ddm, Carrier, dayofweek from ontime_2012 where dayofweek >5 group by Carrier, dayofweek G table: ontime_2012 type: range key: covered key_len: 2 ref: NULL rows: 2961617 Extra: Using where; Using index; Using temporary; Using filesort mysql> select ... +---------+---------+-----------+ | ddm | Carrier | dayofweek | +---------+---------+-----------+ | 10.8564 | AA | 6 | ... 30 rows in set (1.39 sec) 1.  No loose index scan 2.  Filter by key 3.  Group by filesort
  53. 53. www.percona.com ORDER BY and filesort
  54. 54. www.percona.com mysql> explain select district, name, population from City where CountryCode = 'USA' order by population desc limit 10G table: City type: ALL possible_keys: NULL key: NULL key_len: NULL ref: NULL rows: 4079 Extra: Using where; Using filesort
  55. 55. www.percona.com mysql> alter table City add key my_sort2 (CountryCode, population); mysql> explain select district, name, population from City where CountryCode = 'USA' order by population desc limit 10G table: City type: ref key: my_sort2 key_len: 3 ref: const rows: 207 Extra: Using where No filesort
  56. 56. www.percona.com mysql> alter table ontime_2012 add key (DepDelayMinutes); Query OK, 0 rows affected (38.68 sec) mysql> explain select * from ontime_2012 where dayofweek in (6,7) order by DepDelayMinutes desc limit 10G type: index possible_keys: DayOfWeek,covered key: DepDelayMinutes key_len: 5 ref: NULL rows: 24 Extra: Using where 10 rows in set (0.00 sec) 1.  Index is sorted 2.  Scan the whole table in the order of the index 3.  Filter results 4.  Stop after finding 10 rows matching the “where” condition
  57. 57. www.percona.com If Index points to the beginning of the table (physically) = fast As it stops after 10 rows (LIMIT 10)
  58. 58. www.percona.com If Index points to the end of table (physically) or random = slower Much more rows to scan (and skip)
  59. 59. www.percona.com Example: Social Graph
  60. 60. www.percona.com https://blog.twitter.com/2010/introducing-flockdb •  Source_id = <YOU> •  Destination_id = IDs of people following <YOU> •  Position = date (unix_timestamp) when <Destination_id> added <YOU> •  State = {0,1, 2,3} 0 = normal, 2 = removed, 3 = archived (Example)
  61. 61. www.percona.com CREATE TABLE social_graph ( source_id bigint(20) unsigned NOT NULL, destination_id bigint(20) unsigned NOT NULL, position bigint(20) unsigned NOT NULL, state tinyint(4) NOT NULL, ... PRIMARY KEY (source_id,destination_id) ) ENGINE=InnoDB DEFAULT CHARSET=utf8
  62. 62. www.percona.com mysql> explain select * from social_graph where source_id = 927 and state in (0) order by position desc limit 100G select_type: SIMPLE table: social_graph type: ref possible_keys: PRIMARY key: PRIMARY key_len: 8 ref: const rows: 957874 Extra: Using where; Using filesort 1 row in set (0.00 sec) Time: 100 rows in set (0.20 sec) Celebrity! Almost 1M followers! J
  63. 63. www.percona.com mysql> alter table social_graph add key src_state_position(source_id, state, position); mysql> select * from social_graph where source_id = 927 and state in (0) order by position desc limit 100; ... 100 rows in set (0.00 sec)
  64. 64. www.percona.com mysql> explain select * from social_graph where source_id = 927 and state in (0) order by position desc limit 100G table: social_graph type: ref possible_keys: PRIMARY,src_state_position key: src_state_position key_len: 9 ref: const,const rows: 957874 Extra: Using where; Using index No filesort Using Index
  65. 65. www.percona.com mysql> explain select * from where source_id = 927 and state in (0,2) order by position desc limit 100G select_type: SIMPLE table: social_graph type: ref possible_keys: PRIMARY,src_state_position key: src_state_position key_len: 8 ref: const rows: 923251 Extra: Using where; Using filesort 100 rows in set (0.54 sec) In (N,N) = Range Only first field of index is used
  66. 66. www.percona.com mysql> alter table social_graph add key src_position(source_id, position); mysql> explain select * from social_graph where source_id = 927 and state in (0,2) order by position desc limit 100G select_type: SIMPLE table: social_graph type: range possible_keys: PRIMARY,src_state_position,src_position key: src_position key_len: 8 ref: NULL rows: 923922 Extra: Using where 100 rows in set (0.00 sec) Scan by source_id, correct order Filter out “wrong” state(s) Stop after finding 100 rows
  67. 67. www.percona.com mysql> explain select * from social_graph where source_id = 927 and state in (1,2) order by position desc limit 100G select_type: SIMPLE table: social_graph type: range possible_keys: PRIMARY,src_state_position,src_position key: src_state_position key_len: 9 ref: NULL rows: 28 Extra: Using where; Using filesort 100 rows in set (0.00 sec) Use proper index Scan 28 rows is faster even with filesort +-------+----------+ | state | count(*) | +-------+----------+ | 0 | 456156 | | 1 | 27 | | 2 | 1 | +-------+----------+
  68. 68. www.percona.com mysql> explain select * from social_graph ignore index (src_state_position) where source_id = 927 and state in (1,2) order by position desc limit 100G select_type: SIMPLE table: social_graph type: ref possible_keys: PRIMARY,src_position key: src_position key_len: 8 ref: const rows: 923922 Extra: Using where 27 rows in set (0.54 sec) •  Will stop only after “discarding” 400K rows. •  27 rows returned •  filtering + filesort is faster in this case +-------+----------+ | state | count(*) | +-------+----------+ | 0 | 456156 | | 1 | 27 | | 2 | 1 | +-------+----------+
  69. 69. www.percona.com Questions? Thank you!

×