329Preserving Performance through Index Maintenance<br />Tony Bain<br />Director, Tony Bain Group<br />tony@tonybain.com<b...
Overview<br />How fragmentation occurs<br />What fragmentation means<br />External<br />Internal<br />How to see fragmenta...
How Are Indexes Used?<br />Root<br />Index Seek<br />SELECT *<br />FROM MyTable<br />WHERE IDCol=8<br />Intermediate<br />...
How Are Indexes Used?<br />Root<br />Index Scan<br />SELECT *<br />FROM MyTable<br />Intermediate<br />…<br />Leaf<br />
How Are Indexes Used?<br />Root<br />Index Scan<br />Intermediate<br />SELECT *<br />FROM MyTable<br />WHERE IDCOL BETWEEN...
How Fragmentation OccursExternal Fragmentation<br />Modifications can lead to page splits<br />Inserts to a page which is ...
50%<br />50%<br />How Fragmentation OccursExternal Fragmentation<br />Modifications can lead to page splits<br />Inserts t...
How Fragmentation Occurs External Fragmentation<br />Heaps do not have 50/50 Page Splits, only forwarding pointers (no fra...
1<br />2<br />What Fragmentation MeansExternal Fragmentation<br />Logical left to right order of leaf level is NOT physica...
What Fragmentation MeansInternal Fragmentation<br />Free space on the page<br />Desirable for OLTP<br />Free space for new...
How To See Fragmentation<br />Poor query performance over time<br />More disk activity<br />Poor cache utilization<br />Ve...
How To See Fragmentation<br />Manually execute DBCC SHOWCONTIG <br />May take some time on large tables – use FAST to see ...
  demo<br />Fragmentation Impact<br />
DBCC SHOWCONTIG Output<br />External fragmentation<br />Scan density <br />Uses a percentage to show the out of order exte...
DBCC SHOWCONTIG Options<br />DBCC SHOWCONTIG<br />All Tables/Heap Structures Only<br />No Nonclustered Indexes<br />DBCC S...
DBCC SHOWCONTIG WITH TABLERESULTS<br />	Programmatically execute and analyze DBCC SHOWCONTIG results<br />INSERT #ShowCont...
How To Minimize Fragmentation<br />Add internal space on pages to allow fluctuation in row size (due to variable width)<br...
FILLFACTOR<br />Fillfactor impacts ONLY the LEAF level of an index<br />0<br />Default value – special meaning. Leaf level...
FILLFACTOR<br />Estimating an approximate FILLFACTOR<br />What is going to cause splits in this structure?<br />Updates to...
demo<br />Fill Factor<br />
  Best Practice<br />Rebuild your index more frequently with a lower fill factor<br />
Rebuilding An Index<br />Why?<br />When?<br />How?<br />DROP and RECREATE Index<br />DBCC DBREINDEX<br />CREATE with DROP_...
Why?<br />Minimize the number of pages required to store the data<br />Minimize cache requirements<br />Improve OLTP perfo...
When?<br />Depends on the table – OLTP versus OLAP<br />Depends on the usage – large queries, not a lot of cache<br />Perf...
How?<br />Rebuild<br />DROP and re-CREATE<br />DBCC DBREINDEX<br />CREATE with DROP_EXISTING<br />Defrag<br />DBCC INDEXDE...
Drop And Recreate The Index<br />Pros<br />None, really<br />Cons<br />Primary or unique key<br />Cannot drop without drop...
DBCC DBREINDEX<br />Pros<br />Easy syntax, easy to automate<br />Doesn’t require foreign keys to be dropped first<br />SP2...
CREATE With DROP_EXISTING<br />Pros<br />Synonym to DBCC DBREINDEX (same PROs)<br />Can be used to CHANGE the definition o...
DBCC INDEXDEFRAG<br />Pros<br />Does not cause blocking <br />Easy syntax, easy to automate <br />Doesn’t require foreign ...
Most frequent usage for all indexes
Use for quick fix during business hours</li></li></ul><li>Comparisons<br />
Automating Index Rebuilds<br />Use a cursor to walk the tables<br />Programmatically determine the level of fragmentation ...
Index Column List?<br />CREATE TABLE #IndexInfo<br />(<br />	IndexName	sysname,<br />	IndexDesc	varchar(210),<br />	IndexK...
Automating Index Rebuilds<br />sp_RebuildIndexes <br />To Rebuild All Indexes on All Tables for all that have a Scan Densi...
Review<br />How fragmentation occurs<br />What fragmentation means<br />How to see fragmentation<br />How to minimize frag...
Resources…<br />Whitepaper: Microsoft SQL Server 2000 Index Defragmentation Best Practices http://www.microsoft.com/techne...
Upcoming SlideShare
Loading in...5
×

Teched03 Index Maint Tony Bain

654

Published on

Tony Bain's SQL Server Index Maintanence presentation from Tech Ed 2003

Published in: Technology, News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
654
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
29
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Teched03 Index Maint Tony Bain

  1. 1. 329Preserving Performance through Index Maintenance<br />Tony Bain<br />Director, Tony Bain Group<br />tony@tonybain.com<br />
  2. 2. Overview<br />How fragmentation occurs<br />What fragmentation means<br />External<br />Internal<br />How to see fragmentation<br />How to minimize fragmentation<br />Rebuilding an index<br />Why<br />When<br />How<br />Automating index rebuilds<br />
  3. 3. How Are Indexes Used?<br />Root<br />Index Seek<br />SELECT *<br />FROM MyTable<br />WHERE IDCol=8<br />Intermediate<br />…<br />Leaf<br />
  4. 4. How Are Indexes Used?<br />Root<br />Index Scan<br />SELECT *<br />FROM MyTable<br />Intermediate<br />…<br />Leaf<br />
  5. 5. How Are Indexes Used?<br />Root<br />Index Scan<br />Intermediate<br />SELECT *<br />FROM MyTable<br />WHERE IDCOL BETWEEN 4 AND 8<br />…<br />Leaf<br />
  6. 6. How Fragmentation OccursExternal Fragmentation<br />Modifications can lead to page splits<br />Inserts to a page which is already full(there is a special case when the INSERT is greater than the last row on the page)<br />Updates to a variable width column on a page which is already full<br />Data page reviewand scenario<br />INSERT Customer VALUES (‘Atostle’, …)<br />Header 96 Bytes<br />Twenty rows/page8096 bytes for data12 bytes free<br />Albertson<br />Atkinson<br />Bach<br />…<br />Ambers<br />Atonke<br />Badlen<br />Anderson<br />…<br />Barber<br />…<br />Baan<br />…<br />Astronder<br />Bacen<br />Bendron<br />File1, Page 5982<br />File1, Page 5983<br />File1, Page 5984<br />
  7. 7. 50%<br />50%<br />How Fragmentation OccursExternal Fragmentation<br />Modifications can lead to page splits<br />Inserts to a page which is already full(there is a special case when the INSERT is greater than the last row on the page)<br />Updates to a variable width column on a page which is already full<br />Data page reviewand scenario<br />INSERT Customer VALUES (‘Atostle’, …)<br />New page linked in<br />Header 96 Bytes<br />Twenty rows/page8096 bytes for data12 bytes free<br />Albertson<br />Atkinson<br />Bach<br />…<br />…<br />Ambers<br />Atonke<br />Badlen<br />Bann<br />Bacen<br />Anderson<br />…<br />Barber<br />…<br />…<br />Baan<br />Astronder<br />Bendron<br />Bacen<br />File1, Page 5982<br />File1, Page 5983<br />File1, Page 11231<br />File1, Page 5984<br />
  8. 8. How Fragmentation Occurs External Fragmentation<br />Heaps do not have 50/50 Page Splits, only forwarding pointers (no fragmentation, per se); generally, heaps are not recommended<br />Clustered tables<br />INSERTS: can be main problem if Primary Key is not monotonically increasing/decreasing<br />UPDATES: depend on Row Definition and existence of variable width columns<br />Nonclustered indexes<br />INSERTS: very likely to cause splits – unless increasing<br />UPDATES: only when a variable width column in the index changes and/or the change causes the record to change location in the index<br />
  9. 9. 1<br />2<br />What Fragmentation MeansExternal Fragmentation<br />Logical left to right order of leaf level is NOT physically left to right<br />Almost always a concern, but not always!<br />Typically means splits internal fragmentation<br />Can cause disk thrashing for low cache or low selectivity queries<br />1, Griffith, …<br />6, Shen, …<br />19981, Geller, …<br />19979, Kiesow, …<br />19976, Baker, …<br />3, Ambers, …<br />…<br />2, Ulaska, … <br />7, Alberts, …<br />19982, Smith, …<br />19980, Gellock, …<br />19977, Shehy, …<br />4, Johany, …<br />8, Landon, …<br />19983, Jones, …<br />19978, Laws, …<br />5, Smith, …<br />3<br />
  10. 10. What Fragmentation MeansInternal Fragmentation<br />Free space on the page<br />Desirable for OLTP<br />Free space for new records<br />Free space for updates<br />Undesirable OLAP<br />Table requires more pages<br />Large analysis queries require more I/Os<br />1, Griffith, …<br />3, Ambers, …<br />6, Shen, …<br />19981, Geller, …<br />19979, Kiesow, …<br />19976, Baker, …<br />…<br />2, Ulaska, … <br />4, Johany, …<br />7, Alberts, …<br />19982, Smith, …<br />19980, Gellock, …<br />19977, Shehy, …<br />5, Smith, …<br />8, Landon, …<br />19983, Jones, …<br />19978, Laws, …<br />
  11. 11. How To See Fragmentation<br />Poor query performance over time<br />More disk activity<br />Poor cache utilization<br />Verify query I/O<br />SET STATISTICS IO ON<br />Verify scan density<br />DBCC SHOWCONTIG<br /> Manually<br /> Programmatically<br /> Automatically<br />Periodically re-verify Query I/O<br />
  12. 12. How To See Fragmentation<br />Manually execute DBCC SHOWCONTIG <br />May take some time on large tables – use FAST to see if the table is in need of a DEFRAG<br />DBCC SHOWCONTIG scanning &apos;member&apos; table...<br />Table: &apos;member&apos; (2025058250); index ID: 1, database ID: 7<br />TABLE level scan performed.<br />- Pages Scanned................................: 183<br />- Extents Scanned..............................: 24<br />- Extent Switches..............................: 34<br />- Avg. Pages per Extent........................: 7.6<br />- Scan Density [Best Count:Actual Count].......: 65.71% [23:35]<br />- Logical Scan Fragmentation ..................: 3.83%<br />- Extent Scan Fragmentation ...................: 12.50%<br />- Avg. Bytes Free per Page.....................: 1605.0<br />- Avg. Page Density (full).....................: 80.17%<br />DBCC execution completed. If DBCC printed error messages, contact your system administrator.<br />
  13. 13. demo<br />Fragmentation Impact<br />
  14. 14. DBCC SHOWCONTIG Output<br />External fragmentation<br />Scan density <br />Uses a percentage to show the out of order extents<br />Logical scan fragmentation<br />Determines whether or not the pages owned – are contiguous (useless on a HEAP)<br />Extent scan fragmentation<br />Determines whether or not the extents owned have gaps<br />Internal fragmentation<br />Average bytes free per page<br />Amount of free space (if OLTP +, if OLAP -)<br />Average page density (fullness)<br />Shown as a percentage – how FULL are the pages<br />
  15. 15. DBCC SHOWCONTIG Options<br />DBCC SHOWCONTIG<br />All Tables/Heap Structures Only<br />No Nonclustered Indexes<br />DBCC SHOWCONTIG (tablename)<br />Only the Table/Heap Structure for specified tablename<br />DBCC SHOWCONTIG (tablename) WITH ALL_INDEXES<br />Includes all Nonclustered Indexes<br />DBCC SHOWCONTIG (tablename) WITH ALL_INDEXES, TABLERESULTS<br />Returns the results in a tabular result set – excellent for programmatic analysis and automation!<br />
  16. 16. DBCC SHOWCONTIG WITH TABLERESULTS<br /> Programmatically execute and analyze DBCC SHOWCONTIG results<br />INSERT #ShowContigOutput<br /> EXEC (&apos;DBCC SHOWCONTIG WITH TABLERESULTS&apos;)<br />go<br />SELECT ScanDensity FROM #ShowContigOutput <br /> WHERE ObjectName = &apos;TableName&apos; <br /> AND IndexName = &apos;IndexName&apos;<br />go<br /> See script for temp table definition“DBCC ShowContig with Tableresults.sql”<br />
  17. 17. How To Minimize Fragmentation<br />Add internal space on pages to allow fluctuation in row size (due to variable width)<br />Rebuild index structures with appropriate FILLFACTOR setting<br />Use fixed width columns<br />No updates (data dependant)<br />No deletes (data dependant)<br />
  18. 18. FILLFACTOR<br />Fillfactor impacts ONLY the LEAF level of an index<br />0<br />Default value – special meaning. Leaf level is filled to 100%<br />Excellent for OLAP (until it starts splitting)<br />Not ideal for OLTP (starts splitting immediately)<br />100<br />Leaf level is filled to 100% (same as default of 0)<br />Excellent for OLAP (until it starts splitting)<br />Not ideal for OLTP (starts splitting immediately)<br />1-99<br />IDEAL for OLTP – you know your data!<br />But how do you set this?<br />
  19. 19. FILLFACTOR<br />Estimating an approximate FILLFACTOR<br />What is going to cause splits in this structure?<br />Updates to variable width datatypes? <br />How wide? How volatile?<br />The wider and/or more volatile  LOWER FillFactor<br />Inserts?<br />How often? Is the data ever-increasing?<br />The more volatile  LOWER FillFactor<br />How often can you rebuild?<br />The more frequent  HIGHER FillFactor<br />Test your guesstimate!<br />Use DBCC SHOWCONTIG between rebuilds to see how far and how fast the scan density drops<br /> The faster/lower scan density  <br /> LOWER FillFactor <br />or DECREASE the time between rebuilds<br />
  20. 20. demo<br />Fill Factor<br />
  21. 21. Best Practice<br />Rebuild your index more frequently with a lower fill factor<br />
  22. 22. Rebuilding An Index<br />Why?<br />When?<br />How?<br />DROP and RECREATE Index<br />DBCC DBREINDEX<br />CREATE with DROP_EXISTING<br />DBCC INDEXDEFRAG<br />Automation!<br />
  23. 23. Why?<br />Minimize the number of pages required to store the data<br />Minimize cache requirements<br />Improve OLTP performance – less splits<br />Improve OLAP performance – fewer pages to read (condensed)<br />Less resources required<br />
  24. 24. When?<br />Depends on the table – OLTP versus OLAP<br />Depends on the usage – large queries, not a lot of cache<br />Performance gains outweigh the maintenance requirements and potential inaccessibility (due to locking) of the table<br />
  25. 25. How?<br />Rebuild<br />DROP and re-CREATE<br />DBCC DBREINDEX<br />CREATE with DROP_EXISTING<br />Defrag<br />DBCC INDEXDEFRAG<br />Comparisons<br />
  26. 26. Drop And Recreate The Index<br />Pros<br />None, really<br />Cons<br />Primary or unique key<br />Cannot drop without dropping Foreign Key references first <br />Clustered – rebuilds all non-clustered twice<br />Drop causes the Table becomes a HEAP <br /> All non-clustered are rebuilt<br />Recreate causes the table to become clustered again<br />All non-clustered are rebuilt again<br />Not easy to automate<br />Recommendation  do not use<br />
  27. 27. DBCC DBREINDEX<br />Pros<br />Easy syntax, easy to automate<br />Doesn’t require foreign keys to be dropped first<br />SP2+ Only rebuilds the index specified (RTM/SP1 had a bug)<br />Completely rebuilds the LEAF Level and the B-Tree<br />Completely updates statistics<br />Cons<br />Causes locks to be held for the duration of the rebuild<br />Clustered Index Rebuild = eXclusive (X) Table Level Lock<br />Non-clustered Index Rebuild = Shared (S) Table Level Lock<br />Creates a large transaction – cannot be cleared from the log until completion; i.e., lot of log activity during the rebuild<br />Non-clustered indexes are rebuilt if the clustered key is not unique<br /> Recommendation  Excellent/easy to automate!<br />
  28. 28. CREATE With DROP_EXISTING<br />Pros<br />Synonym to DBCC DBREINDEX (same PROs)<br />Can be used to CHANGE the definition of the Clustering Key (the nonclustered indexes will be rebuilt)<br />Cons<br />More complex syntax, harder to automate<br />Non-Clustered indexes are rebuilt if the clustered key is not unique<br />Recommendation <br /> Use when you need to change the Clustering Key<br />
  29. 29. DBCC INDEXDEFRAG<br />Pros<br />Does not cause blocking <br />Easy syntax, easy to automate <br />Doesn’t require foreign keys to be dropped first<br />Runs as small transactions – LOTS of log activity however it can be backed up WHILE this is running (not just a single large tran)<br />Cons<br />Does not update statistics<br />Does not rebuild the non-leaf level(s)<br />Does not move the table for better clustering<br />Recommendations<br /><ul><li>Use when table must be highly available (see KB Q282286 if not using 2000sp1+)
  30. 30. Most frequent usage for all indexes
  31. 31. Use for quick fix during business hours</li></li></ul><li>Comparisons<br />
  32. 32. Automating Index Rebuilds<br />Use a cursor to walk the tables<br />Programmatically determine the level of fragmentation (scan density) for each index<br />Only rebuild those with a S.D. less than desired<br />Rebuild the clustered with DROP_EXISTING<br />Use INSERT with sp_helpindex to get index information (uniqueness and column list)<br />Rebuild the nonclustered with DBCC DBREINDEX<br />Create as a procedure and setup a job!<br />
  33. 33. Index Column List?<br />CREATE TABLE #IndexInfo<br />(<br /> IndexName sysname,<br /> IndexDesc varchar(210),<br /> IndexKeys nvarchar(2126)<br />)<br />GO<br />INSERT #IndexInfo <br /> EXEC sp_helpindex tname<br />GO<br />SELECT * FROM #IndexInfo<br />GO<br />SELECT IndexKeys FROM #IndexInfo WHERE IndexName = indexname<br />go<br />
  34. 34. Automating Index Rebuilds<br />sp_RebuildIndexes <br />To Rebuild All Indexes on All Tables for all that have a Scan Density &lt; 100%<br />sp_RebuildIndexes @ScanDensity = 80<br />To Rebuild All Indexes on All Tables with a Scan Density of &lt; 80%<br />sp_RebuildIndexes &apos;Authors&apos;<br />To Rebuild All Indexes on the authors table - for a Scan Density of &lt; 100%<br />sp_RebuildIndexes &apos;Authors&apos;, 80<br />To Rebuild All Indexes on the authors table - for a Scan Density of &lt; 80%<br />Object Name and ScanDensity are both optional parameters. <br />ScanDensity must be a whole number between 1 and 100<br />
  35. 35. Review<br />How fragmentation occurs<br />What fragmentation means<br />How to see fragmentation<br />How to minimize fragmentation<br />Rebuilding an index<br />Why<br />When<br />How<br />Automating index rebuilds<br />
  36. 36. Resources…<br />Whitepaper: Microsoft SQL Server 2000 Index Defragmentation Best Practices http://www.microsoft.com/technet/treeview/default.asp?url=/technet/prodtechnol/sql/maintain/Optimize/SS2KIDBP.asp<br />
  37. 37. Ask The ExpertsGet Your Questions Answered<br />Talk one-on-one with a community of your peers<br />Community Experts: Microsoft product teams, consultants and Tech*Ed speakers <br />Resources: whiteboards, internet, etc.<br />Location: in the middle of the Exhibit Hall<br />Hours: at least 12-3:30p every day<br />I will be available in the ATE area after this session<br />
  38. 38. © 2003 Microsoft Corporation. All rights reserved.<br />This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×