Mentor Your Indexes

MENTOR Your Indexes Bill Karwin Independent Oracle Users Group • 2010-9-21

Me • Software developer • C, Java, Perl, PHP, Ruby • SQL maven • Author of new book SQL Antipatterns

“Whenever any result is sought, the question will then arise—by what course of calculation can these results be arrived at by the machine in the shortest time?” — Charles Babbage, Passages from the Life of a Philosopher (1864)

Indexes

Common blunders: • Creating indexes naively • Executing non-indexable queries • Rejecting indexes because of overhead

CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT );

CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT, INDEX (PostId) ); redundant index, already in PK

CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT, INDEX (Title) ); bulky index

CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT, INDEX (Score) ); unnecessary index, we may never query on score

CREATE TABLE Posts ( PostId SERIAL PRIMARY KEY, CreationDate DATE NOT NULL, Title VARCHAR(80) NOT NULL, Body TEXT NOT NULL, Score INT, INDEX (Score, CreationDate, Title) ); unnecessary composite index

SELECT * FROM Posts WHERE Title LIKE ‘%crash%’ non-leftmost string match

Telephone book analogy: • Easy to search for Dean Thomas: uses index to match SELECT * FROM TelephoneBook WHERE full_name LIKE ‘Thomas, %’ • Hard to search for Thomas Riddle: requires full table scan SELECT * FROM TelephoneBook WHERE full_name LIKE ‘%, Thomas’

SELECT * FROM Posts WHERE MONTH(CreationDate) = 4 function applied to column

SELECT * FROM Users WHERE LastName = ‘Thomas’ OR FirstName = ‘Thomas’ just like searching for ﬁrst_name

SELECT * FROM Users ORDER BY FirstName, LastName non-leftmost composite key match

the beneﬁt quickly justiﬁes the overhead O(n) table scan O(log n) index scan

Relational Index data modeling optimization is derived is derived from data from queries

MENTOR Your Indexes

Measure Explain Nominate Test Optimize Repair

• Proﬁle your code to identify your biggest performance costs. • MySQL: PROFILER • Oracle: TKPROF or Trace Analyzer • Application-level proﬁling

• Analyze the database’s optimization plan for costly queries. • Identify queries that don’t use indexes.

• MySQL: EXPLAIN Query • “Explain Output Format” http://dev.mysql.com/doc/refman/5.5/en/ explain-output.html

• Oracle: EXPLAIN PLAN Query • “Understanding Explain Plan” http://www.orafaq.com/node/1420

• Which queries need optimization? • Frequently used queries • Very slow queries • Reports

• Which column(s) need indexes? • WHERE conditions • JOIN conditions • ORDER BY criteria • MIN() / MAX()

• Automatic tools for nominating indexes: • MySQL Enterprise Query Analyzer • Oracle Automatic SQL Tuning Advisor

• After creating index, measure your high- priority queries again. • Conﬁrm that the new index made a difference to these queries. • Impress your boss/client! “The new index gave us a 127% performance improvement!”

• Indexes are compact, frequently-used data structures. • Try to cache indexes in memory.

• Cache indexes in MySQL/InnoDB: • Increase innodb_buffer_pool_size • Used for both data and indexes • Cache indexes in MySQL/MyISAM: • Increase key_buffer_size • LOAD INDEX INTO CACHE TableName [INDEX IndexName];

• Cache indexes in Oracle: ALTER SYSTEM SET DB_32K_CACHE_SIZE = 100m; CREATE TABLESPACE INDEX_TS_32K BLOCKSIZE 32K; ALTER INDEX IndexName REBUILD ONLINE TABLESPACE INDEX_TS_32K; http://www.dba-oracle.com/art_so_optimizer_index_caching.htm

• Indexes require periodic maintenance. • Like a ﬁlesystem requires periodic defragmentation.

• Analyze / rebuild indexes in MySQL: • ANALYZE TABLE TableName • OPTIMIZE TABLE TableName

• Analyze / rebuild indexes in Oracle: • ANALYZE INDEX IndexName • ALTER INDEX IndexName REBUILD ...

1. Know Your Data. 2. Know Your Queries. 3. MENTOR Your Indexes.

SQL Antipatterns: Avoiding the Pitfalls of Database Programming http://www.pragprog.com/titles/bksqla/

Copyright 2010 Bill Karwin www.slideshare.net/billkarwin Released under a Creative Commons 3.0 License: http://creativecommons.org/licenses/by-nc-nd/3.0/ You are free to share - to copy, distribute and transmit this work, under the following conditions: Attribution. Noncommercial. No Derivative Works. You must attribute this You may not use this work You may not alter, work to Bill Karwin. for commercial purposes. transform, or build upon this work.

Mentor Your Indexes

by Karwin Software Solutions LLC on Sep 21, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

Mentor Your Indexes — Presentation Transcript