Successfully reported this slideshow.

Vld bs conquering the giant


Published on

Published in: Technology, Sports
  • Be the first to comment

  • Be the first to like this

Vld bs conquering the giant

  1. 1. Conquering the Giants:Very Large Databases<br />For audio, call the phone number in your meeting invite.<br />Audio will not come through the computer speakers.<br />
  2. 2. SQL Server Expertfor Quest Software<br />Editor-in-Chief at<br />Former SQL DBA<br />Managed data warehouses,>80tb SAN storage for $7b company<br />
  3. 3. Conquering the Giants: Agenda<br />What’s a VLDB?<br />Ginormous Growth<br />Big Backups<br />Tremendous Tables<br />Slow SQL Statements<br />Your Wingman: Quest<br />Resources<br />Photo Licensed with Creative Commons from<br />
  4. 4. What Makes a VLDB?<br />1 terabyte<br />Billion rows<br />But Not:User Qty,Criticality,Transactions Per Second<br />4<br />Photo Licensed with Creative Commons from<br />
  5. 5. VLDB Space Challenges<br />5<br />
  6. 6. Default File Growth Settings<br />6<br />
  7. 7. Starts Out Small…<br />7<br />
  8. 8. Ginormous Growths<br />8<br />
  9. 9. Instant File Init in Secpol.msc<br />9<br />
  10. 10. VLDBs = Big Backups<br />Time Problem<br />Nightly Load Windows<br />Time to Recover<br />Dev, QA, Test, DR<br />10<br />
  11. 11. Giant Monstrous Backups<br />11<br />
  12. 12. Cutting Them Down to Size<br />12<br />
  13. 13. Next Weapon: SAN Snapshots<br />13<br />
  14. 14. Post-Snapshot<br />14<br />
  15. 15. Making the Snap Available<br />15<br />
  16. 16. Weapon: Backup Compression<br />16<br />Photo Licensed with Creative Commons from<br />
  17. 17. Or – Compress Everything!<br />17<br />
  18. 18. VLDB Problem: Giant Tables<br />18<br />
  19. 19. Hunt Down and Kill Unused Indexes<br />19<br />
  20. 20. Breaking Giants Into Pieces<br />20<br />
  21. 21. The New Way: Partitioning<br />21<br />
  22. 22. Partitioning Basics<br />Create filegroups<br />Create files in the filegroups<br />Design partition function<br />Design partition scheme<br />Alter table’s clustered index<br />22<br />
  23. 23. Sliding Window Loads<br />Partition by day<br />Create newtable similarto Sales table,on same filegroup<br />Load one day’sdata into empty table<br />Swap out a partition for that table<br />23<br />
  24. 24. Giant Drawbacks<br />Not easy to change<br />Requires lots of experimentation<br />Partition elimination doesn’t always work<br />Requires familiarity with queries, data<br />Storage skills help<br />Hardware-dependent<br />24<br />Photo Licensed with Creative Commons from<br />
  25. 25. SQLCAT MAXDOP FAQ<br />25<br />
  26. 26. The New-New Way: 2008 R2<br />26<br />
  27. 27. Slow SQL Statements<br />27<br />
  28. 28. Your Shovel: Resource Governor<br />Slows zombie queries down<br />Throttle CPU, memory<br />Can throttle jobs, backups too<br />Doesn’t throttle storage (yet)<br />28<br />Photo Licensed with Creative Commons from<br />
  29. 29. Implementing The Governator<br />Workload groups:<br />ETL processes<br />Reports<br />AdHoc<br />Resource pool:<br />ETL<br />Reports<br />Classification function based on:<br />Login name<br />Application name<br />29<br />Photo Licensed with Creative Commons from<br />
  30. 30. End Results in SSMS 2008<br />30<br />
  31. 31. Resource Governor Caveats<br />Ask the network team about QoS<br />Get manager consensus on SLAs<br />Keep it absolutely transparent<br />Avoid false alarms about performance problems<br />Teach managers how to “see” it working<br />31<br />
  32. 32. Your Wingman: Quest Software<br />Capacity Manager v3.0<br />Capacity Planning<br />Index Defragmentation<br />Partitioning GUI<br />LiteSpeed v5.1<br />SmartDiffs<br />Throttled Compression<br />32<br />
  33. 33. Showing the Giant Who’s Boss<br />Use Instant File Initialization<br />Consider Differential Backups<br />Compress Data<br />Drop Baggage<br />Partitioning FTW<br />Resource Governator<br />Check Out QuestCapacity Manager<br />Photo Licensed with Creative Commons from<br />
  34. 34. VLDB Resources<br />Presentation Resources:<br />Quest Software:<br />Upcoming Webcasts:<br />