Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Index Reorganization and Rebuilding for Success

1,465 views

Published on

A process and accompanying tools to make Index reorganization/rebuilding successful. Applicable on small databases and multi-Tb databases with over 20,000 tables (including examples of what can go horribly wrong without the right preparation). Bonus: PowerShell monitoring script included.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Index Reorganization and Rebuilding for Success

  1. 1. INDEX REORGANIZATION AND REBUILDING FOR SUCCESS An end-to-end Enterprise re-indexing process
  2. 2. ABSTRACT Dean will introduce a process and accompanying tools to make Index reorganization/rebuilding successful. Applicable on small databases and multi-Tb databases with over 20,000 tables (including examples of what can go horribly wrong without the right preparation). Bonus: PowerShell monitoring script included.
  3. 3. AGENDA • Indexing - What can go wrong? • Process Overview • Pre-requisites (initial setup/configuration) • Pre-index preparation • Running Index optimization • Index progress monitoring (awesome PowerShell script)
  4. 4. WHAT COULD POSSIBLY GO WRONG?
  5. 5. WHAT COULD POSSIBLY GO WRONG? Lose your job BAD • Disk out of space • Tlog or Database files • Tlog backup files • Exponential Tlog growth Temporarily pain • Tlog or database growth initialization causing application timeout
  6. 6. CONSIDERATIONS • Enterprise Edition • Online Index rebuild • Tlog auto-growth settings • % or size • Size planning • Full backups during re-index • Tlog will not truncate during a full backup even if log backups are being taken • Big database (4 hr backup + recovery)  use differentials during re-index? • TODO: Add log_reuse_wait_desc to Powershell monitoring tool when log used percent is > warning threshold. • Re-Index vs. Re-Organize • MaxDOP? • TempDb
  7. 7. PROCESS OVERVIEW
  8. 8. PROCESS OVERVIEW
  9. 9. INITIAL SYSTEM SETUP/CONFIGURATION Pre-requisites to using the proposed (demo) process
  10. 10. OLA HALLENGREN’S INDEX MAINTENANCE SCRIPT • https://ola.hallengren.com/ • Optional, but recommended • CommandLog table
  11. 11. OLA’S INSTALL SCRIPTS • Demo uses a DBA AdminUtility database (instead of master) • @CreateJobs set to Yes • @LogToTable set to Yes
  12. 12. SQL AGENT JOBS • Ola’s scripts will add these jobs • The IndexOptimize – USER_DATABASES job is the one of interest for indexing
  13. 13. ADMINISTRATIVE DATABASE OR TABLES
  14. 14. SP_WHOISACTIVE • Adam Machanic’s sp_WhoIsActive script is a new and improved version of sp_who2 • http://sqlblog.com/blogs/adam_machanic/archive/tags/who+is+active/default.aspx • Shows current connections, % progress of index re-organization, and much more
  15. 15. SCHEDULED TASK
  16. 16. SCHEDULED TASK - TRIGGER • How often to run during indexing
  17. 17. SCHEDULED TASK - ACTION • ExecutionPolicy Bypass
  18. 18. SCHEDULED TASK - RUN
  19. 19. PRE-INDEX PREPARATION
  20. 20. PRE-INDEX PREPARATION FOR VLDB • Tlog backup schedule • Shorten duration • Full backup schedule • Temporarily switch to differential backups • Open monitoring tools or schedule the Sweet Powershell script
  21. 21. INDEX MAINTENANCE
  22. 22. INDEX MAINTENANCE • Ola Hallengren Index Maintenance Solution (recommended) • Queryable maintenance database (read: Sweet PowerShell monitoring) • Post maintenance review (% fragmentation, re-org vs. rebuild, etc.) • Maintenance Plans
  23. 23. PROGRESS MONITORING
  24. 24. INDEX MONITORING – OLD SCHOOL • DMV – Disk Usage • Windows Explorer – .trn backup file sizes • Windows Explorer – tlog backup duration • sp_who2
  25. 25. INDEX MONITORING – LIKE A BOSS • Powershell Script • sp_WhoIsActive (by Adam Mechanic) • Queries • Completed indexes • Current index rebuild progress (%, rows remaining) • Current index reorg progress (%) • Disk space used (Database files and Tlog • Tlog backup size and backup duration • Windows Scheduler • Email
  26. 26. TLOG ABOUT TO GROW • Transactions are paused while TLOG file is growing • Truncation should prevent growth depending on initial size • Autogrowth settings…
  27. 27. DATA FILE ABOUT TO GROW
  28. 28. TLOG ACTIVITY Re-org (~1 Gb/minute) Re-build (~200Kb/minute)
  29. 29. TLOG BACKUP DURATION • Durations shown in the query don’t include the –WithVerify time • I usually use a 3X multiplier for estimating actual tlog backup time • Look at start time of next backup
  30. 30. INDIVIDUAL INDEX PROGRESS Re-Org Re-Build
  31. 31. WARNING SIGNS • VLDB data file about to grow • TLOG file about to grow • TLOG backup duration + Verify taking longer than backup schedule schedule • Log file not truncating (excessive growth) • Wait times on sp_WhoIsActive • File sizes growing larger than available disk space
  32. 32. DEAN WILLSON • Director of Application Development, Aunt Millie’s Bakeries • Long time member of NUFW • Former President of fwPASS • Twitter: @deanwillson • Blogs: dean-o.blogspot.com, practicalhoshin.blogspot.com • LinkedIn: http://linkedin.com/in/deanwillson

×