Uploaded on

We all know how to define database indexes, but which indexes to define remains a mysterious art for most software developers. This talk will use general principles and specific scenarios to give you …

We all know how to define database indexes, but which indexes to define remains a mysterious art for most software developers. This talk will use general principles and specific scenarios to give you practical, step-by-step knowledge to turn a performance bottleneck into an epic win!

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,726
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
0
Comments
0
Likes
8

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. MENTOR Your Indexes Bill Karwin Independent Oracle Users Group • 2010-9-21
  • 2. Me • Software developer • C, Java, Perl, PHP, Ruby • SQL maven • Author of new book SQL Antipatterns
  • 3. “Whenever any result is sought, the question will then arise—by what course of calculation can these results be arrived at by the machine in the shortest time?” — Charles Babbage, Passages from the Life of a Philosopher (1864)
  • 4. Indexes
  • 5. Common blunders: • Creating indexes naively • Executing non-indexable queries • Rejecting indexes because of overhead
  • 6. CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT );
  • 7. CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT, INDEX (PostId) ); redundant index, already in PK
  • 8. CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT, INDEX (Title) ); bulky index
  • 9. CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT, INDEX (Score) ); unnecessary index, we may never query on score
  • 10. CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT, INDEX (Score, CreationDate, Title) ); unnecessary composite index
  • 11. SELECT * FROM Posts WHERE Title LIKE ‘%crash%’ non-leftmost string match
  • 12. Telephone book analogy: • Easy to search for Dean Thomas: uses index to match SELECT * FROM TelephoneBook WHERE full_name LIKE ‘Thomas, %’ • Hard to search for Thomas Riddle: requires full table scan SELECT * FROM TelephoneBook WHERE full_name LIKE ‘%, Thomas’
  • 13. SELECT * FROM Posts WHERE MONTH(CreationDate) = 4 function applied to column
  • 14. SELECT * FROM Users WHERE LastName = ‘Thomas’ OR FirstName = ‘Thomas’ just like searching for first_name
  • 15. SELECT * FROM Users ORDER BY FirstName, LastName non-leftmost composite key match
  • 16. the benefit quickly justifies the overhead O(n) table scan O(log n) index scan
  • 17. Relational Index data modeling optimization is derived is derived from data from queries
  • 18. MENTOR Your Indexes
  • 19. Measure Explain Nominate Test Optimize Repair
  • 20. Measure Explain Nominate Test Optimize Repair
  • 21. • Profile your code to identify your biggest performance costs. • MySQL: PROFILER • Oracle: TKPROF or Trace Analyzer • Application-level profiling
  • 22. Measure Explain Nominate Test Optimize Repair
  • 23. • Analyze the database’s optimization plan for costly queries. • Identify queries that don’t use indexes.
  • 24. • MySQL: EXPLAIN Query • “Explain Output Format” http://dev.mysql.com/doc/refman/5.5/en/ explain-output.html
  • 25. • Oracle: EXPLAIN PLAN Query • “Understanding Explain Plan” http://www.orafaq.com/node/1420
  • 26. Measure Explain Nominate Test Optimize Repair
  • 27. • Which queries need optimization? • Frequently used queries • Very slow queries • Reports
  • 28. • Which column(s) need indexes? • WHERE conditions • JOIN conditions • ORDER BY criteria • MIN() / MAX()
  • 29. • Automatic tools for nominating indexes: • MySQL Enterprise Query Analyzer • Oracle Automatic SQL Tuning Advisor
  • 30. Measure Explain Nominate Test Optimize Repair
  • 31. • After creating index, measure your high- priority queries again. • Confirm that the new index made a difference to these queries. • Impress your boss/client! “The new index gave us a 127% performance improvement!”
  • 32. Measure Explain Nominate Test Optimize Repair
  • 33. • Indexes are compact, frequently-used data structures. • Try to cache indexes in memory.
  • 34. • Cache indexes in MySQL/InnoDB: • Increase innodb_buffer_pool_size • Used for both data and indexes • Cache indexes in MySQL/MyISAM: • Increase key_buffer_size • LOAD INDEX INTO CACHE TableName [INDEX IndexName];
  • 35. • Cache indexes in Oracle: ALTER SYSTEM SET DB_32K_CACHE_SIZE = 100m; CREATE TABLESPACE INDEX_TS_32K BLOCKSIZE 32K; ALTER INDEX IndexName REBUILD ONLINE TABLESPACE INDEX_TS_32K; http://www.dba-oracle.com/art_so_optimizer_index_caching.htm
  • 36. Measure Explain Nominate Test Optimize Repair
  • 37. • Indexes require periodic maintenance. • Like a filesystem requires periodic defragmentation.
  • 38. • Analyze / rebuild indexes in MySQL: • ANALYZE TABLE TableName • OPTIMIZE TABLE TableName
  • 39. • Analyze / rebuild indexes in Oracle: • ANALYZE INDEX IndexName • ALTER INDEX IndexName REBUILD ...
  • 40. 1. Know Your Data. 2. Know Your Queries. 3. MENTOR Your Indexes.
  • 41. SQL Antipatterns: Avoiding the Pitfalls of Database Programming http://www.pragprog.com/titles/bksqla/
  • 42. Copyright 2010 Bill Karwin www.slideshare.net/billkarwin Released under a Creative Commons 3.0 License: http://creativecommons.org/licenses/by-nc-nd/3.0/ You are free to share - to copy, distribute and transmit this work, under the following conditions: Attribution. Noncommercial. No Derivative Works. You must attribute this You may not use this work You may not alter, work to Bill Karwin. for commercial purposes. transform, or build upon this work.