Successfully reported this slideshow.

Index Reorganization and Rebuilding for Success

0

Share

Loading in …3
×
1 of 32
1 of 32

Index Reorganization and Rebuilding for Success

0

Share

Download to read offline

Description

A process and accompanying tools to make Index reorganization/rebuilding successful. Applicable on small databases and multi-Tb databases with over 20,000 tables (including examples of what can go horribly wrong without the right preparation). Bonus: PowerShell monitoring script included.

Transcript

  1. 1. INDEX REORGANIZATION AND REBUILDING FOR SUCCESS An end-to-end Enterprise re-indexing process
  2. 2. ABSTRACT Dean will introduce a process and accompanying tools to make Index reorganization/rebuilding successful. Applicable on small databases and multi-Tb databases with over 20,000 tables (including examples of what can go horribly wrong without the right preparation). Bonus: PowerShell monitoring script included.
  3. 3. AGENDA • Indexing - What can go wrong? • Process Overview • Pre-requisites (initial setup/configuration) • Pre-index preparation • Running Index optimization • Index progress monitoring (awesome PowerShell script)
  4. 4. WHAT COULD POSSIBLY GO WRONG?
  5. 5. WHAT COULD POSSIBLY GO WRONG? Lose your job BAD • Disk out of space • Tlog or Database files • Tlog backup files • Exponential Tlog growth Temporarily pain • Tlog or database growth initialization causing application timeout
  6. 6. CONSIDERATIONS • Enterprise Edition • Online Index rebuild • Tlog auto-growth settings • % or size • Size planning • Full backups during re-index • Tlog will not truncate during a full backup even if log backups are being taken • Big database (4 hr backup + recovery)  use differentials during re-index? • TODO: Add log_reuse_wait_desc to Powershell monitoring tool when log used percent is > warning threshold. • Re-Index vs. Re-Organize • MaxDOP? • TempDb
  7. 7. PROCESS OVERVIEW
  8. 8. PROCESS OVERVIEW
  9. 9. INITIAL SYSTEM SETUP/CONFIGURATION Pre-requisites to using the proposed (demo) process
  10. 10. OLA HALLENGREN’S INDEX MAINTENANCE SCRIPT • https://ola.hallengren.com/ • Optional, but recommended • CommandLog table
  11. 11. OLA’S INSTALL SCRIPTS • Demo uses a DBA AdminUtility database (instead of master) • @CreateJobs set to Yes • @LogToTable set to Yes
  12. 12. SQL AGENT JOBS • Ola’s scripts will add these jobs • The IndexOptimize – USER_DATABASES job is the one of interest for indexing
  13. 13. ADMINISTRATIVE DATABASE OR TABLES
  14. 14. SP_WHOISACTIVE • Adam Machanic’s sp_WhoIsActive script is a new and improved version of sp_who2 • http://sqlblog.com/blogs/adam_machanic/archive/tags/who+is+active/default.aspx • Shows current connections, % progress of index re-organization, and much more
  15. 15. SCHEDULED TASK
  16. 16. SCHEDULED TASK - TRIGGER • How often to run during indexing
  17. 17. SCHEDULED TASK - ACTION • ExecutionPolicy Bypass
  18. 18. SCHEDULED TASK - RUN
  19. 19. PRE-INDEX PREPARATION
  20. 20. PRE-INDEX PREPARATION FOR VLDB • Tlog backup schedule • Shorten duration • Full backup schedule • Temporarily switch to differential backups • Open monitoring tools or schedule the Sweet Powershell script
  21. 21. INDEX MAINTENANCE
  22. 22. INDEX MAINTENANCE • Ola Hallengren Index Maintenance Solution (recommended) • Queryable maintenance database (read: Sweet PowerShell monitoring) • Post maintenance review (% fragmentation, re-org vs. rebuild, etc.) • Maintenance Plans
  23. 23. PROGRESS MONITORING
  24. 24. INDEX MONITORING – OLD SCHOOL • DMV – Disk Usage • Windows Explorer – .trn backup file sizes • Windows Explorer – tlog backup duration • sp_who2
  25. 25. INDEX MONITORING – LIKE A BOSS • Powershell Script • sp_WhoIsActive (by Adam Mechanic) • Queries • Completed indexes • Current index rebuild progress (%, rows remaining) • Current index reorg progress (%) • Disk space used (Database files and Tlog • Tlog backup size and backup duration • Windows Scheduler • Email
  26. 26. TLOG ABOUT TO GROW • Transactions are paused while TLOG file is growing • Truncation should prevent growth depending on initial size • Autogrowth settings…
  27. 27. DATA FILE ABOUT TO GROW
  28. 28. TLOG ACTIVITY Re-org (~1 Gb/minute) Re-build (~200Kb/minute)
  29. 29. TLOG BACKUP DURATION • Durations shown in the query don’t include the –WithVerify time • I usually use a 3X multiplier for estimating actual tlog backup time • Look at start time of next backup
  30. 30. INDIVIDUAL INDEX PROGRESS Re-Org Re-Build
  31. 31. WARNING SIGNS • VLDB data file about to grow • TLOG file about to grow • TLOG backup duration + Verify taking longer than backup schedule schedule • Log file not truncating (excessive growth) • Wait times on sp_WhoIsActive • File sizes growing larger than available disk space
  32. 32. DEAN WILLSON • Director of Application Development, Aunt Millie’s Bakeries • Long time member of NUFW • Former President of fwPASS • Twitter: @deanwillson • Blogs: dean-o.blogspot.com, practicalhoshin.blogspot.com • LinkedIn: http://linkedin.com/in/deanwillson

Description

A process and accompanying tools to make Index reorganization/rebuilding successful. Applicable on small databases and multi-Tb databases with over 20,000 tables (including examples of what can go horribly wrong without the right preparation). Bonus: PowerShell monitoring script included.

Transcript

  1. 1. INDEX REORGANIZATION AND REBUILDING FOR SUCCESS An end-to-end Enterprise re-indexing process
  2. 2. ABSTRACT Dean will introduce a process and accompanying tools to make Index reorganization/rebuilding successful. Applicable on small databases and multi-Tb databases with over 20,000 tables (including examples of what can go horribly wrong without the right preparation). Bonus: PowerShell monitoring script included.
  3. 3. AGENDA • Indexing - What can go wrong? • Process Overview • Pre-requisites (initial setup/configuration) • Pre-index preparation • Running Index optimization • Index progress monitoring (awesome PowerShell script)
  4. 4. WHAT COULD POSSIBLY GO WRONG?
  5. 5. WHAT COULD POSSIBLY GO WRONG? Lose your job BAD • Disk out of space • Tlog or Database files • Tlog backup files • Exponential Tlog growth Temporarily pain • Tlog or database growth initialization causing application timeout
  6. 6. CONSIDERATIONS • Enterprise Edition • Online Index rebuild • Tlog auto-growth settings • % or size • Size planning • Full backups during re-index • Tlog will not truncate during a full backup even if log backups are being taken • Big database (4 hr backup + recovery)  use differentials during re-index? • TODO: Add log_reuse_wait_desc to Powershell monitoring tool when log used percent is > warning threshold. • Re-Index vs. Re-Organize • MaxDOP? • TempDb
  7. 7. PROCESS OVERVIEW
  8. 8. PROCESS OVERVIEW
  9. 9. INITIAL SYSTEM SETUP/CONFIGURATION Pre-requisites to using the proposed (demo) process
  10. 10. OLA HALLENGREN’S INDEX MAINTENANCE SCRIPT • https://ola.hallengren.com/ • Optional, but recommended • CommandLog table
  11. 11. OLA’S INSTALL SCRIPTS • Demo uses a DBA AdminUtility database (instead of master) • @CreateJobs set to Yes • @LogToTable set to Yes
  12. 12. SQL AGENT JOBS • Ola’s scripts will add these jobs • The IndexOptimize – USER_DATABASES job is the one of interest for indexing
  13. 13. ADMINISTRATIVE DATABASE OR TABLES
  14. 14. SP_WHOISACTIVE • Adam Machanic’s sp_WhoIsActive script is a new and improved version of sp_who2 • http://sqlblog.com/blogs/adam_machanic/archive/tags/who+is+active/default.aspx • Shows current connections, % progress of index re-organization, and much more
  15. 15. SCHEDULED TASK
  16. 16. SCHEDULED TASK - TRIGGER • How often to run during indexing
  17. 17. SCHEDULED TASK - ACTION • ExecutionPolicy Bypass
  18. 18. SCHEDULED TASK - RUN
  19. 19. PRE-INDEX PREPARATION
  20. 20. PRE-INDEX PREPARATION FOR VLDB • Tlog backup schedule • Shorten duration • Full backup schedule • Temporarily switch to differential backups • Open monitoring tools or schedule the Sweet Powershell script
  21. 21. INDEX MAINTENANCE
  22. 22. INDEX MAINTENANCE • Ola Hallengren Index Maintenance Solution (recommended) • Queryable maintenance database (read: Sweet PowerShell monitoring) • Post maintenance review (% fragmentation, re-org vs. rebuild, etc.) • Maintenance Plans
  23. 23. PROGRESS MONITORING
  24. 24. INDEX MONITORING – OLD SCHOOL • DMV – Disk Usage • Windows Explorer – .trn backup file sizes • Windows Explorer – tlog backup duration • sp_who2
  25. 25. INDEX MONITORING – LIKE A BOSS • Powershell Script • sp_WhoIsActive (by Adam Mechanic) • Queries • Completed indexes • Current index rebuild progress (%, rows remaining) • Current index reorg progress (%) • Disk space used (Database files and Tlog • Tlog backup size and backup duration • Windows Scheduler • Email
  26. 26. TLOG ABOUT TO GROW • Transactions are paused while TLOG file is growing • Truncation should prevent growth depending on initial size • Autogrowth settings…
  27. 27. DATA FILE ABOUT TO GROW
  28. 28. TLOG ACTIVITY Re-org (~1 Gb/minute) Re-build (~200Kb/minute)
  29. 29. TLOG BACKUP DURATION • Durations shown in the query don’t include the –WithVerify time • I usually use a 3X multiplier for estimating actual tlog backup time • Look at start time of next backup
  30. 30. INDIVIDUAL INDEX PROGRESS Re-Org Re-Build
  31. 31. WARNING SIGNS • VLDB data file about to grow • TLOG file about to grow • TLOG backup duration + Verify taking longer than backup schedule schedule • Log file not truncating (excessive growth) • Wait times on sp_WhoIsActive • File sizes growing larger than available disk space
  32. 32. DEAN WILLSON • Director of Application Development, Aunt Millie’s Bakeries • Long time member of NUFW • Former President of fwPASS • Twitter: @deanwillson • Blogs: dean-o.blogspot.com, practicalhoshin.blogspot.com • LinkedIn: http://linkedin.com/in/deanwillson

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

×