MENTOR
Your Indexes
Bill Karwin
Independent Oracle Users Group • 2010-9-21
Me

• Software developer
• C, Java, Perl, PHP, Ruby
• SQL maven
• Author of new book
  SQL Antipatterns
“Whenever any result is sought, the
question will then arise—by what
course of calculation can these results
be arrived at...
Indexes
Common blunders:

• Creating indexes naively
• Executing non-indexable queries
• Rejecting indexes because of overhead
CREATE TABLE Posts (

 PostId
 
 
 
 SERIAL PRIMARY KEY,

 CreationDate
 DATE NOT NULL,

 Title

 
 
 
 VARCHAR(80) NOT NU...
CREATE TABLE Posts (

 PostId
 
 
 
 SERIAL PRIMARY KEY,

 CreationDate
 DATE NOT NULL,

 Title

 
 
 
 VARCHAR(80) NOT NU...
CREATE TABLE Posts (

 PostId
 
 
 
 SERIAL PRIMARY KEY,

 CreationDate
 DATE NOT NULL,

 Title

 
 
 
 VARCHAR(80) NOT NU...
CREATE TABLE Posts (

 PostId
 
 
 
 SERIAL PRIMARY KEY,

 CreationDate
 DATE NOT NULL,

 Title

 
 
 
 VARCHAR(80) NOT NU...
CREATE TABLE Posts (

 PostId
 
 
 
 SERIAL PRIMARY KEY,

 CreationDate
 DATE NOT NULL,

 Title

 
 
 
 VARCHAR(80) NOT NU...
SELECT * FROM Posts
WHERE Title LIKE ‘%crash%’

                    non-leftmost
                    string match
Telephone book analogy:

• Easy to search for Dean Thomas:
                                      uses index
              ...
SELECT * FROM Posts
WHERE MONTH(CreationDate) = 4
              function applied
                 to column
SELECT * FROM Users
WHERE LastName = ‘Thomas’
 OR FirstName = ‘Thomas’
          just like searching
            for first_...
SELECT * FROM Users
ORDER BY FirstName, LastName
                    non-leftmost
                 composite key match
the benefit quickly
                      justifies the overhead




O(n) table scan
O(log n) index scan
Relational         Index
data modeling   optimization
  is derived      is derived
  from data     from queries
MENTOR Your
  Indexes
Measure
 Explain
Nominate
  Test
Optimize
 Repair
Measure
 Explain
Nominate
  Test
Optimize
 Repair
• Profile your code to identify your biggest
  performance costs.
  • MySQL: PROFILER
  • Oracle: TKPROF or Trace Analyzer
...
Measure
 Explain
Nominate
  Test
Optimize
 Repair
• Analyze the database’s optimization plan
    for costly queries.
•   Identify queries that don’t use indexes.
• MySQL: EXPLAIN Query
  • “Explain Output Format”
    http://dev.mysql.com/doc/refman/5.5/en/
    explain-output.html
• Oracle: EXPLAIN PLAN Query
  • “Understanding Explain Plan”
    http://www.orafaq.com/node/1420
Measure
 Explain
Nominate
  Test
Optimize
 Repair
• Which queries need optimization?
  • Frequently used queries
  • Very slow queries
  • Reports
• Which column(s) need indexes?
  • WHERE conditions
  • JOIN conditions
  • ORDER BY criteria
  • MIN() / MAX()
• Automatic tools for nominating indexes:
 • MySQL Enterprise Query Analyzer
 • Oracle Automatic SQL Tuning Advisor
Measure
 Explain
Nominate
  Test
Optimize
 Repair
• After creating index, measure your high-
    priority queries again.
•   Confirm that the new index made a
    difference...
Measure
 Explain
Nominate
  Test
Optimize
 Repair
• Indexes are compact, frequently-used data
    structures.
•   Try to cache indexes in memory.
• Cache indexes in MySQL/InnoDB:
 • Increase innodb_buffer_pool_size
 • Used for both data and indexes
• Cache indexes in ...
• Cache indexes in Oracle:
  ALTER SYSTEM SET DB_32K_CACHE_SIZE = 100m;
  CREATE TABLESPACE INDEX_TS_32K
  BLOCKSIZE 32K;
...
Measure
 Explain
Nominate
  Test
Optimize
 Repair
• Indexes require periodic maintenance.
• Like a filesystem requires periodic
  defragmentation.
• Analyze / rebuild indexes in MySQL:
  • ANALYZE TABLE TableName
  • OPTIMIZE TABLE TableName
• Analyze / rebuild indexes in Oracle:
  • ANALYZE INDEX IndexName
  • ALTER INDEX IndexName REBUILD ...
1. Know Your Data.
2. Know Your Queries.
3. MENTOR Your Indexes.
SQL Antipatterns:
Avoiding the Pitfalls of
Database Programming




http://www.pragprog.com/titles/bksqla/
Copyright 2010 Bill Karwin
        www.slideshare.net/billkarwin
              Released under a Creative Commons 3.0 Licen...
Upcoming SlideShare
Loading in...5
×

Mentor Your Indexes

3,993

Published on

We all know how to define database indexes, but which indexes to define remains a mysterious art for most software developers. This talk will use general principles and specific scenarios to give you practical, step-by-step knowledge to turn a performance bottleneck into an epic win!

Published in: Technology
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,993
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide

Mentor Your Indexes

  1. 1. MENTOR Your Indexes Bill Karwin Independent Oracle Users Group • 2010-9-21
  2. 2. Me • Software developer • C, Java, Perl, PHP, Ruby • SQL maven • Author of new book SQL Antipatterns
  3. 3. “Whenever any result is sought, the question will then arise—by what course of calculation can these results be arrived at by the machine in the shortest time?” — Charles Babbage, Passages from the Life of a Philosopher (1864)
  4. 4. Indexes
  5. 5. Common blunders: • Creating indexes naively • Executing non-indexable queries • Rejecting indexes because of overhead
  6. 6. CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT );
  7. 7. CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT, INDEX (PostId) ); redundant index, already in PK
  8. 8. CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT, INDEX (Title) ); bulky index
  9. 9. CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT, INDEX (Score) ); unnecessary index, we may never query on score
  10. 10. CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT, INDEX (Score, CreationDate, Title) ); unnecessary composite index
  11. 11. SELECT * FROM Posts WHERE Title LIKE ‘%crash%’ non-leftmost string match
  12. 12. Telephone book analogy: • Easy to search for Dean Thomas: uses index to match SELECT * FROM TelephoneBook WHERE full_name LIKE ‘Thomas, %’ • Hard to search for Thomas Riddle: requires full table scan SELECT * FROM TelephoneBook WHERE full_name LIKE ‘%, Thomas’
  13. 13. SELECT * FROM Posts WHERE MONTH(CreationDate) = 4 function applied to column
  14. 14. SELECT * FROM Users WHERE LastName = ‘Thomas’ OR FirstName = ‘Thomas’ just like searching for first_name
  15. 15. SELECT * FROM Users ORDER BY FirstName, LastName non-leftmost composite key match
  16. 16. the benefit quickly justifies the overhead O(n) table scan O(log n) index scan
  17. 17. Relational Index data modeling optimization is derived is derived from data from queries
  18. 18. MENTOR Your Indexes
  19. 19. Measure Explain Nominate Test Optimize Repair
  20. 20. Measure Explain Nominate Test Optimize Repair
  21. 21. • Profile your code to identify your biggest performance costs. • MySQL: PROFILER • Oracle: TKPROF or Trace Analyzer • Application-level profiling
  22. 22. Measure Explain Nominate Test Optimize Repair
  23. 23. • Analyze the database’s optimization plan for costly queries. • Identify queries that don’t use indexes.
  24. 24. • MySQL: EXPLAIN Query • “Explain Output Format” http://dev.mysql.com/doc/refman/5.5/en/ explain-output.html
  25. 25. • Oracle: EXPLAIN PLAN Query • “Understanding Explain Plan” http://www.orafaq.com/node/1420
  26. 26. Measure Explain Nominate Test Optimize Repair
  27. 27. • Which queries need optimization? • Frequently used queries • Very slow queries • Reports
  28. 28. • Which column(s) need indexes? • WHERE conditions • JOIN conditions • ORDER BY criteria • MIN() / MAX()
  29. 29. • Automatic tools for nominating indexes: • MySQL Enterprise Query Analyzer • Oracle Automatic SQL Tuning Advisor
  30. 30. Measure Explain Nominate Test Optimize Repair
  31. 31. • After creating index, measure your high- priority queries again. • Confirm that the new index made a difference to these queries. • Impress your boss/client! “The new index gave us a 127% performance improvement!”
  32. 32. Measure Explain Nominate Test Optimize Repair
  33. 33. • Indexes are compact, frequently-used data structures. • Try to cache indexes in memory.
  34. 34. • Cache indexes in MySQL/InnoDB: • Increase innodb_buffer_pool_size • Used for both data and indexes • Cache indexes in MySQL/MyISAM: • Increase key_buffer_size • LOAD INDEX INTO CACHE TableName [INDEX IndexName];
  35. 35. • Cache indexes in Oracle: ALTER SYSTEM SET DB_32K_CACHE_SIZE = 100m; CREATE TABLESPACE INDEX_TS_32K BLOCKSIZE 32K; ALTER INDEX IndexName REBUILD ONLINE TABLESPACE INDEX_TS_32K; http://www.dba-oracle.com/art_so_optimizer_index_caching.htm
  36. 36. Measure Explain Nominate Test Optimize Repair
  37. 37. • Indexes require periodic maintenance. • Like a filesystem requires periodic defragmentation.
  38. 38. • Analyze / rebuild indexes in MySQL: • ANALYZE TABLE TableName • OPTIMIZE TABLE TableName
  39. 39. • Analyze / rebuild indexes in Oracle: • ANALYZE INDEX IndexName • ALTER INDEX IndexName REBUILD ...
  40. 40. 1. Know Your Data. 2. Know Your Queries. 3. MENTOR Your Indexes.
  41. 41. SQL Antipatterns: Avoiding the Pitfalls of Database Programming http://www.pragprog.com/titles/bksqla/
  42. 42. Copyright 2010 Bill Karwin www.slideshare.net/billkarwin Released under a Creative Commons 3.0 License: http://creativecommons.org/licenses/by-nc-nd/3.0/ You are free to share - to copy, distribute and transmit this work, under the following conditions: Attribution. Noncommercial. No Derivative Works. You must attribute this You may not use this work You may not alter, work to Bill Karwin. for commercial purposes. transform, or build upon this work.

×