1. Conquering the Giants:Very Large Databases For audio, call the phone number in your meeting invite. Audio will not come through the computer speakers.
2. SQL Server Expertfor Quest Software Editor-in-Chief at SQLServerPedia.com Former SQL DBA Managed data warehouses,>80tb SAN storage for $7b company
3. Conquering the Giants: Agenda What’s a VLDB? Ginormous Growth Big Backups Tremendous Tables Slow SQL Statements Your Wingman: Quest Resources Photo Licensed with Creative Commons from http://www.flickr.com/photos/pagedooley/2172001078/
4. What Makes a VLDB? 1 terabyte Billion rows But Not:User Qty,Criticality,Transactions Per Second 4 Photo Licensed with Creative Commons from http://www.flickr.com/photos/tcmhitchhiker/1053095394
22. Partitioning Basics Create filegroups Create files in the filegroups Design partition function Design partition scheme Alter table’s clustered index 22
23. Sliding Window Loads Partition by day Create newtable similarto Sales table,on same filegroup Load one day’sdata into empty table Swap out a partition for that table 23
24. Giant Drawbacks Not easy to change Requires lots of experimentation Partition elimination doesn’t always work Requires familiarity with queries, data Storage skills help Hardware-dependent 24 Photo Licensed with Creative Commons from http://www.flickr.com/photos/cellphonesusie/3224335140/
28. Your Shovel: Resource Governor Slows zombie queries down Throttle CPU, memory Can throttle jobs, backups too Doesn’t throttle storage (yet) 28 Photo Licensed with Creative Commons from http://www.flickr.com/photos/archiemcphee/3612278108/
29. Implementing The Governator Workload groups: ETL processes Reports AdHoc Resource pool: ETL Reports Classification function based on: Login name Application name 29 Photo Licensed with Creative Commons from http://www.flickr.com/photos/shaunwong/2467881648/
31. Resource Governor Caveats Ask the network team about QoS Get manager consensus on SLAs Keep it absolutely transparent Avoid false alarms about performance problems Teach managers how to “see” it working 31
32. Your Wingman: Quest Software Capacity Manager v3.0 Capacity Planning Index Defragmentation Partitioning GUI LiteSpeed v5.1 SmartDiffs Throttled Compression 32
33. Showing the Giant Who’s Boss Use Instant File Initialization Consider Differential Backups Compress Data Drop Baggage Partitioning FTW Resource Governator Check Out QuestCapacity Manager Photo Licensed with Creative Commons from http://www.flickr.com/photos/keeg/1661401660/
Production, Dev, QA, DRWhen files are this big, even the simplest thing is time-consuming – think growing a file.
Runs faster tooFrees up nightly windowDoesn’t ease recovery time, though. Makes recovery time LONGER.Let’s look at a solution that can ease recovery time too.
Do your backups off the snapshotHold multiple snapshots over timeNot everybody can afford this level of recoverability though.Let’s look at something that Microsoft included with SQL 2008 Enterprise Edition.
New in SQL 2008Enterprise Edition onlyInclude DR, development boxesNot configurable
New in SQL 2008Enterprise Edition onlyInclude DR, development boxesNot inherited – tables, indexesScript to make it easier at the SSP article
Note index size on 2nd rowEvery row has to be backed up, DBCC’d. Harder to get DBCC time on VLDBs.Every index has to be backed up and reorganized
Dropping indexes gives us less of them, but they’re still huge.How can we break them into smaller pieces?
Easier because it’s transparent to the application.
Happens nearly instantaneouslyOnly a metadata change
Zombie ad-hoc queries run forever. User closes web browser, but the query runs on headlessly
