Conquering the Giants:Very Large Databases<br />For audio, call the phone number in your meeting invite.<br />Audio will n...
SQL Server Expertfor Quest Software<br />Editor-in-Chief at SQLServerPedia.com<br />Former SQL DBA<br />Managed data wareh...
Conquering the Giants: Agenda<br />What’s a VLDB?<br />Ginormous Growth<br />Big Backups<br />Tremendous Tables<br />Slow ...
What Makes a VLDB?<br />1 terabyte<br />Billion rows<br />But Not:User Qty,Criticality,Transactions Per Second<br />4<br /...
VLDB Space Challenges<br />5<br />
Default File Growth Settings<br />6<br />
Starts Out Small…<br />7<br />
Ginormous Growths<br />8<br />
Instant File Init in Secpol.msc<br />9<br />
VLDBs = Big Backups<br />Time Problem<br />Nightly Load Windows<br />Time to Recover<br />Dev, QA, Test, DR<br />10<br />
Giant Monstrous Backups<br />11<br />
Cutting Them Down to Size<br />12<br />
Next Weapon: SAN Snapshots<br />13<br />
Post-Snapshot<br />14<br />
Making the Snap Available<br />15<br />
Weapon: Backup Compression<br />16<br />Photo Licensed with Creative Commons from http://www.flickr.com/photos/tcmhitchhik...
Or – Compress Everything!<br />17<br />
VLDB Problem: Giant Tables<br />18<br />
Hunt Down and Kill Unused Indexes<br />19<br />
Breaking Giants Into Pieces<br />20<br />
The New Way: Partitioning<br />21<br />
Partitioning Basics<br />Create filegroups<br />Create files in the filegroups<br />Design partition function<br />Design ...
Sliding Window Loads<br />Partition by day<br />Create newtable similarto Sales table,on same filegroup<br />Load one day’...
Giant Drawbacks<br />Not easy to change<br />Requires lots of experimentation<br />Partition elimination doesn’t always wo...
SQLCAT MAXDOP FAQ<br />25<br />
The New-New Way: 2008 R2<br />26<br />
Slow SQL Statements<br />27<br />
Your Shovel: Resource Governor<br />Slows zombie queries down<br />Throttle CPU, memory<br />Can throttle jobs, backups to...
Implementing The Governator<br />Workload groups:<br />ETL processes<br />Reports<br />AdHoc<br />Resource pool:<br />ETL<...
End Results in SSMS 2008<br />30<br />
Resource Governor Caveats<br />Ask the network team about QoS<br />Get manager consensus on SLAs<br />Keep it absolutely t...
Your Wingman: Quest Software<br />Capacity Manager v3.0<br />Capacity Planning<br />Index Defragmentation<br />Partitionin...
Showing the Giant Who’s Boss<br />Use Instant File Initialization<br />Consider Differential Backups<br />Compress Data<br...
VLDB Resources<br />Presentation Resources:http://sqlserverpedia.com/wiki/VLDB<br />Quest Software:http://www.quest.com/sq...
Upcoming SlideShare
Loading in...5
×

Vld bs conquering the giant

229

Published on

Published in: Technology, Sports
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
229
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Production, Dev, QA, DRWhen files are this big, even the simplest thing is time-consuming – think growing a file.
  • Runs faster tooFrees up nightly windowDoesn’t ease recovery time, though. Makes recovery time LONGER.Let’s look at a solution that can ease recovery time too.
  • Do your backups off the snapshotHold multiple snapshots over timeNot everybody can afford this level of recoverability though.Let’s look at something that Microsoft included with SQL 2008 Enterprise Edition.
  • New in SQL 2008Enterprise Edition onlyInclude DR, development boxesNot configurable
  • New in SQL 2008Enterprise Edition onlyInclude DR, development boxesNot inherited – tables, indexesScript to make it easier at the SSP article
  • Note index size on 2nd rowEvery row has to be backed up, DBCC’d. Harder to get DBCC time on VLDBs.Every index has to be backed up and reorganized
  • Dropping indexes gives us less of them, but they’re still huge.How can we break them into smaller pieces?
  • Easier because it’s transparent to the application.
  • Happens nearly instantaneouslyOnly a metadata change
  • Zombie ad-hoc queries run forever. User closes web browser, but the query runs on headlessly
  • Vld bs conquering the giant

    1. 1. Conquering the Giants:Very Large Databases<br />For audio, call the phone number in your meeting invite.<br />Audio will not come through the computer speakers.<br />
    2. 2. SQL Server Expertfor Quest Software<br />Editor-in-Chief at SQLServerPedia.com<br />Former SQL DBA<br />Managed data warehouses,>80tb SAN storage for $7b company<br />
    3. 3. Conquering the Giants: Agenda<br />What’s a VLDB?<br />Ginormous Growth<br />Big Backups<br />Tremendous Tables<br />Slow SQL Statements<br />Your Wingman: Quest<br />Resources<br />Photo Licensed with Creative Commons from http://www.flickr.com/photos/pagedooley/2172001078/<br />
    4. 4. What Makes a VLDB?<br />1 terabyte<br />Billion rows<br />But Not:User Qty,Criticality,Transactions Per Second<br />4<br />Photo Licensed with Creative Commons from http://www.flickr.com/photos/tcmhitchhiker/1053095394<br />
    5. 5. VLDB Space Challenges<br />5<br />
    6. 6. Default File Growth Settings<br />6<br />
    7. 7. Starts Out Small…<br />7<br />
    8. 8. Ginormous Growths<br />8<br />
    9. 9. Instant File Init in Secpol.msc<br />9<br />
    10. 10. VLDBs = Big Backups<br />Time Problem<br />Nightly Load Windows<br />Time to Recover<br />Dev, QA, Test, DR<br />10<br />
    11. 11. Giant Monstrous Backups<br />11<br />
    12. 12. Cutting Them Down to Size<br />12<br />
    13. 13. Next Weapon: SAN Snapshots<br />13<br />
    14. 14. Post-Snapshot<br />14<br />
    15. 15. Making the Snap Available<br />15<br />
    16. 16. Weapon: Backup Compression<br />16<br />Photo Licensed with Creative Commons from http://www.flickr.com/photos/tcmhitchhiker/1053045338/<br />
    17. 17. Or – Compress Everything!<br />17<br />
    18. 18. VLDB Problem: Giant Tables<br />18<br />
    19. 19. Hunt Down and Kill Unused Indexes<br />19<br />
    20. 20. Breaking Giants Into Pieces<br />20<br />
    21. 21. The New Way: Partitioning<br />21<br />
    22. 22. Partitioning Basics<br />Create filegroups<br />Create files in the filegroups<br />Design partition function<br />Design partition scheme<br />Alter table’s clustered index<br />22<br />
    23. 23. Sliding Window Loads<br />Partition by day<br />Create newtable similarto Sales table,on same filegroup<br />Load one day’sdata into empty table<br />Swap out a partition for that table<br />23<br />
    24. 24. Giant Drawbacks<br />Not easy to change<br />Requires lots of experimentation<br />Partition elimination doesn’t always work<br />Requires familiarity with queries, data<br />Storage skills help<br />Hardware-dependent<br />24<br />Photo Licensed with Creative Commons from http://www.flickr.com/photos/cellphonesusie/3224335140/<br />
    25. 25. SQLCAT MAXDOP FAQ<br />25<br />
    26. 26. The New-New Way: 2008 R2<br />26<br />
    27. 27. Slow SQL Statements<br />27<br />
    28. 28. Your Shovel: Resource Governor<br />Slows zombie queries down<br />Throttle CPU, memory<br />Can throttle jobs, backups too<br />Doesn’t throttle storage (yet)<br />28<br />Photo Licensed with Creative Commons from http://www.flickr.com/photos/archiemcphee/3612278108/<br />
    29. 29. Implementing The Governator<br />Workload groups:<br />ETL processes<br />Reports<br />AdHoc<br />Resource pool:<br />ETL<br />Reports<br />Classification function based on:<br />Login name<br />Application name<br />29<br />Photo Licensed with Creative Commons from http://www.flickr.com/photos/shaunwong/2467881648/<br />
    30. 30. End Results in SSMS 2008<br />30<br />
    31. 31. Resource Governor Caveats<br />Ask the network team about QoS<br />Get manager consensus on SLAs<br />Keep it absolutely transparent<br />Avoid false alarms about performance problems<br />Teach managers how to “see” it working<br />31<br />
    32. 32. Your Wingman: Quest Software<br />Capacity Manager v3.0<br />Capacity Planning<br />Index Defragmentation<br />Partitioning GUI<br />LiteSpeed v5.1<br />SmartDiffs<br />Throttled Compression<br />32<br />
    33. 33. Showing the Giant Who’s Boss<br />Use Instant File Initialization<br />Consider Differential Backups<br />Compress Data<br />Drop Baggage<br />Partitioning FTW<br />Resource Governator<br />Check Out QuestCapacity Manager<br />Photo Licensed with Creative Commons from http://www.flickr.com/photos/keeg/1661401660/<br />
    34. 34. VLDB Resources<br />Presentation Resources:http://sqlserverpedia.com/wiki/VLDB<br />Quest Software:http://www.quest.com/sql-server/<br />Upcoming Webcasts:http://www.quest.com/events<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×