Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

New index features in MySQL 8

95 views

Published on

An introduction to three of the new index features in MySQL 8

Published in: Software
  • Be the first to comment

New index features in MySQL 8

  1. 1. Copyright © 2019 Oracle and/or its affiliates. All rights reserved. New index features in MySQL 8 Erik Frøseth Software Developer MySQL Optimizer Team February 1st, 2019
  2. 2. 2Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  3. 3. 3Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Program Agenda Functional indexes Index skip scan Invisible indexes 1 2 3 4 5 6 7
  4. 4. 4Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Program Agenda Functional indexes Index skip scan Invisible indexes 1 2 3 4 5 6 7
  5. 5. 5Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Terminology # This creates a single index/key with two index/key parts CREATE INDEX my_index ON my_table (column_a, column_b);
  6. 6. 6Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Functional index – what is it? ● A functional index is an index where at least one of the index parts is a function instead of a column. 1) CREATE INDEX my_index ON my_table ((ABS(column_a))); 2) CREATE INDEX my_index ON my_table ((ABS(column_a)), column_b); 3) CREATE TABLE my_table ( column_a VARCHAR(255), INDEX my_index ((UPPER(column_a))) );
  7. 7. 7Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Functional index – how do you use it? ● A functional index is created, used and removed exactly like any other index. mysql> CREATE TABLE my_table ( -> column_a VARCHAR(255), -> INDEX my_index ((UPPER(column_a))) -> ); Query OK, 0 rows affected (0.04 sec) mysql> EXPLAIN SELECT * FROM my_table WHERE UPPER(column_a) = "FOOBAR"; +-------------+----------+------+----------+---------+-------+------+----------+ | select_type | table | type | key | key_len | ref | rows | filtered | +-------------+----------+------+----------+---------+-------+------+----------+ | SIMPLE | my_table | ref | my_index | 1023 | const | 1 | 100.00 | +-------------+----------+------+----------+---------+-------+------+----------+ 1 row in set, 1 warning (0.00 sec)
  8. 8. 8Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Functional index – how do you use it? mysql> CREATE TABLE my_table ( -> column_a VARCHAR(255), -> column_b INT, -> INDEX my_index ((UPPER(column_a)), column_b) -> ); Query OK, 0 rows affected (0.04 sec) mysql> EXPLAIN SELECT * FROM my_table WHERE UPPER(column_a) = "FOOBAR" AND column_b < 20; +-------------+----------+-------+---------------+----------+---------+------+----------+ | select_type | table | type | possible_keys | key | key_len | rows | filtered | +-------------+----------+-------+---------------+----------+---------+------+----------+ | SIMPLE | my_table | range | my_index | my_index | 1028 | 1 | 100.00 | +-------------+----------+-------+---------------+----------+---------+------+----------+ 1 row in set, 1 warning (0.00 sec) ● You can mix functional and non-functional index parts
  9. 9. 9Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Functional index – observability mysql> SHOW INDEX FROM t1; +-------+------------+----------+--------------+-------------+...+-------------------+ | Table | Non_unique | Key_name | Seq_in_index | Column_name |...| Expression | +-------+------------+----------+--------------+-------------+...+-------------------+ | t1 | 1 | idx | 1 | NULL |...| (`col1` + `col1`) | +-------+------------+----------+--------------+-------------+...+-------------------+ 1 row in set (0.01 sec) mysql> SELECT * FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_NAME = "t1"; +...+------------+--------------+-------------+...+-------------------+ |...| INDEX_NAME | SEQ_IN_INDEX | COLUMN_NAME |...| EXPRESSION | +...+------------+--------------+-------------+...+-------------------+ |...| idx | 1 | NULL |...| (`col1` + `col1`) | +...+------------+--------------+-------------+...+-------------------+ 1 row in set (0.00 sec)
  10. 10. 10Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Functional index – observability mysql> SHOW CREATE TABLE t1; +-------+-----------------------------------------------------------+ | Table | Create Table | +-------+-----------------------------------------------------------+ | t1 | CREATE TABLE `t1` ( `col1` int(11) DEFAULT NULL, KEY `idx` (((`col1` + `col1`))) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci | +-------+----------------------------------------------------------+ 1 row in set (0.00 sec)
  11. 11. 11Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Functional index – use cases The main use case is indexing complex type such as geometries and (especially) JSON. CREATE INDEX json_extract_idx ON my_table ((CAST(data->>'$.person.id' AS CHAR(100)))); { “person”: { “first_name”: “Erik”, “last_name”: “Frøseth”, “id”: 8 } }
  12. 12. 12Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Functional index – limitations and other notes ● The primary key cannot be a functional index ● You cannot index a function that returns a BLOB or TEXT ● You can not index non-deterministic functions (RAND(), NOW(), UUID() etc...) ● Each functional index part “uses” one column ● It’s more costly to maintain a functional index than a regular index. Do not overuse it!
  13. 13. 13Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Program Agenda Functional indexes Index skip scan Invisible indexes 1 2 3 4 5 6 7
  14. 14. 14Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Index skip scan ● Based on a contribution from Facebook ● An index can only be used if the first/leading column(s) is referenced in the WHERE-condition CREATE INDEX my_index ON my_table (column1, column2); # Works! SELECT * FROM my_table WHERE column1 = 33; SELECT * FROM my_table WHERE column1 = 33 AND column2 > 3; # Does not work... SELECT * FROM my_table WHERE column2 > 3;
  15. 15. 15Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Index skip scan ● With index skip scan, the first index part can be omitted in the WHERE condition CREATE INDEX my_index ON my_table (column1, column2); # Works! SELECT * FROM my_table WHERE column1 = 33; SELECT * FROM my_table WHERE column1 = 33 AND column2 > 3; # This works too! SELECT * FROM my_table WHERE column2 > 3;
  16. 16. 16Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Index skip scan column1 column2 1 1 1 2 1 3 1 4 2 1 2 2 2 3 2 4 2 5 SELECT * FROM my_table WHERE column2 > 3; column1 = 1 AND column2 > 3 column1 = 2 AND column2 > 3
  17. 17. 17Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Index skip scan mysql> EXPLAIN SELECT * FROM my_table WHERE column2 > 3; +----------+-------+------+------+----------+----------------------------------------+ | table | type | key | rows | filtered | Extra | +----------+-------+------+------+----------+----------------------------------------+ | my_table | range | idx1 | 85 | 100.00 | Using where; Using index for skip scan | +----------+-------+------+------+----------+----------------------------------------+ 1 row in set, 1 warning (0.00 sec)
  18. 18. 18Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Index skip scan ● Usage criterias: – The query must only reference columns that exists in the in the index. – The estimated cost must be the lowest of any alternative (many distinct values in the first key part will most likely results in a high cost).
  19. 19. 19Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Index skip scan Few distinct values "skip_scan_range": { "potential_skip_scan_indexes": [{ "index": "idx2", "tree_travel_cost": 0.45, "num_groups": 12, "rows": 138, "cost": 31.817 } ] } Many distinct values "skip_scan_range": { "potential_skip_scan_indexes": [{ "index": "idx2", "tree_travel_cost": 0.5, "num_groups": 421, "rows": 832, "cost": 546.45 } ] }
  20. 20. 20Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Index skip scan ● Can be controlled by optimizer switch and query hints 1) SET optimizer_switch='skip_scan=off'; 2) SELECT /*+ SKIP_SCAN(t3) */ col1 FROM t3 WHERE col2 < 10; 3) SELECT /*+ SKIP_SCAN(t3 idx1) */ col1 FROM t3 WHERE col2 < 10; 4) SELECT /*+ NO_SKIP_SCAN(t3) */ col1 FROM t3 WHERE col2 < 10;
  21. 21. 21Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Program Agenda Functional indexes Index skip scan Invisible indexes 1 2 3 4 5 6 7
  22. 22. 22Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Invisible indexes – What is it? ● Lets you turn off any index ● Global setting ● Invisible to the optimizer, but it’s still maintained by the storage engine ● Can be turned on/off completely by setting the optimizer switch «use_invisible_indexes» ● Primary keys cannot be made invisible
  23. 23. 23Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Invisible indexes – How is it used? 1) ALTER TABLE t1 ALTER INDEX idx INVISIBLE; 2) CREATE INDEX idx ON my_table (a_column) INVISIBLE; 3) CREATE TABLE t1 ( col1 INT, INDEX idx (col1) INVISIBLE );
  24. 24. 24Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Invisible indexes - Example mysql> EXPLAIN SELECT COUNT(*) FROM lineitem WHERE l_suppkey = 7706; ...+------+---------------+-------------+---------+-------+------+----------+-------------+ ...| type | possible_keys | key | key_len | ref | rows | filtered | Extra | ...+------+---------------+-------------+---------+-------+------+----------+-------------+ ...| ref | i_l_suppkey | i_l_suppkey | 5 | const | 604 | 100.00 | Using index | ...+------+---------------+-------------+---------+-------+------+----------+-------------+ 1 row in set, 1 warning (0.00 sec) mysql> ALTER TABLE lineitem ALTER INDEX i_l_suppkey INVISIBLE; Query OK, 0 rows affected (0.18 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> EXPLAIN SELECT COUNT(*) FROM lineitem WHERE l_suppkey = 7706; ...+------+---------------+------+---------+------+---------+----------+-------------+ ...| type | possible_keys | key | key_len | ref | rows | filtered | Extra | ...+------+---------------+------+---------+------+---------+----------+-------------+ ...| ALL | NULL | NULL | NULL | NULL | 5941264 | 0.01 | Using where | ...+------+---------------+------+---------+------+---------+----------+-------------+ 1 row in set, 1 warning (0.00 sec)
  25. 25. 25Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Invisible indexes - Observability ● See the visibility using INFORMATION_SCHEMA or SHOW INDEXES mysql> SHOW INDEXES FROM my_table; +----------+------------+---------------------+...+---------+ | Table | Non_unique | Key_name |...| Visible | +----------+------------+---------------------+...+---------+ | my_table | 1 | just_to_be_safe_idx |...| NO | +----------+------------+---------------------+...+---------+ mysql> SELECT INDEX_NAME, IS_VISIBLE FROM INFORMATION_SCHEMA.STATISTICS > WHERE TABLE_NAME = "my_table"; +---------------------+------------+ | INDEX_NAME | IS_VISIBLE | +---------------------+------------+ | just_to_be_safe_idx | NO | +---------------------+------------+ 1 row in set (0.01 sec)
  26. 26. 26Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Invisible indexes - Observability mysql> SHOW CREATE TABLE t1; +-------+----------------------------------------------------------+ | Table | Create Table | +-------+----------------------------------------------------------+ | t1 | CREATE TABLE `t1` ( `col1` int(11) DEFAULT NULL, KEY `just_to_be_safe_index` (((`col1` * 2))) /*!80000 INVISIBLE */ ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci | +-------+----------------------------------------------------------+
  27. 27. 27Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Invisible indexes – what you should remember ● See if you get any errors due to index hints referring to the index ● Slow query log ● Performance schema ● Application performance ● Check sys.schema_unused_indexes
  28. 28. 28Copyright © 2019 Oracle and/or its affiliates. All rights reserved.
  29. 29. 29Copyright © 2019 Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

×