Online Reporting Architectures Behind Load Balancers

Load-Balancing Architectures for Online Reporting DB’s

About the fools in the front…
Crys Manson
No relation to Charlie, though she tries…
Senior DBA with Microsoft Advertising
Started as a DBA in 2004 with Atlas/aQuantive, acquired by MS in 2007
Manages the primary ETL and reporting systems for Microsoft’s Adserving system collecting and reporting on systems that serve over 400,000 ads/sec every day.
Email: crysma@microsoft.com
Twitter: @crysmanson
John Halunen
Senior DBA with Microsoft Advertising
Started working with Crys as a DBA in 2005
Managed various systems in Microsoft Adv, including our Search Mgmt and Atlas Publisher ETL and Reporting DB systems
Email: joha@microsoft.com
Twitter: @sqlmoto

Disclaimer
We are not here representing Microsoft officially
These are solutions that we have used or researched, or plan to use.
Your environment is likely to be different.
Test everything you hear us say in your Production Test environment FIRST.
Basically, Your Mileage May Vary (YMMV), be careful, and Trust, but Verify. 

What we are talking about…
Our journey
Suggested architectures
Advantages/Disadvantages
Lessons learned
Future options

Environmental Background

Webservers/Appservers
Middle Tier servers
Network LoadBalancing / VIP (Virtual IP)

Online Offline
http://www.flickr.com/photos/eterno_retorno/2383602431/sizes/m/in/photostream/
http://www.flickr.com/photos/aussiegall/360422572/sizes/m/in/photostream/

Simple (online/offline)
Paired Reporting db’s on a single server: Refresh/Online
Control table indicating status
Advantages
Disadvantages

Distributed Refresh
http://www.flickr.com/photos/octopushat/2474017307/sizes/z/in/photostream/

Distributed Refresh
Multiple Reporting servers organized in Refresh Groups
Control db
Control jobs
Advantages
Disadvantages

Logshipped
http://www.flickr.com/photos/pinwheel/284410383/

Logshipped
One or more Logshipped Primary (HeadNodes)  refreshes
Ships logs to two or more (groups) of Reporting servers.
Use db snapshots to present the data.
Standalone server (for emergency situations)
Advantages
Disadvantages

SCALABLE SHARED DATABASES
http://www.flickr.com/photos/jonsphotos/218704764/sizes/m/in/photostream/
http://www.flickr.com/photos/froboy/2761162751/sizes/m/in/photostream/
http://www.flickr.com/photos/timothymorgan/75593157/sizes/o/in/photostream/

Scalable Shared Disks
Maximizes use of server resources
Requires less total SAN space but it must be fast
MSDN recommends a max of 8 servers per db
Requires at least two sets of disk (R, and R/W)
Maybe 4 if maintenance needs are big

Some other options

A brief mention of Some Other Options
Federation
Lots of good resources online
Complicated (split databases horizontally to allow processing queries on multiple servers)
AlwaysOn
Mirroring with up to 4 readable replicas
Not really a scaling solution, more of a great way to use your BCP hardware.
Next PNWSQL meeting is this topic

Mgmt Considerations(for the various Architectures)http://www.flickr.com/photos/ksukalac/5050125511/in/photostream/

Management Considerations

Suggestions…http://www.flickr.com/photos/1001words/3604142099/

Suggestions
Get monitoring/maintenance included in your initial design
Build logs to record the report generation sqlstatements, and exceptions
Having those logs means you can rerun the actual reporting load to test or troubleshoot your db systems!
Build logs to record refresh process success/failure
Build a refresh Verify process
Code Rollback is a crucial safety net
Test this in your Production Test Env. or its not a rollback
Testing in Production can be very useful
Must use Parallel chains or Pin servers to minimize risk
Consider your future needs carefully

Conclusionhttp://www.flickr.com/photos/snowpeak/4152675183/sizes/m/in/photostream/

References
Our Presentation
http://www.crysmanson.com/go/LoadBalancing
How to use Mirroring and Snapshots http://www.sqlservercentral.com/articles/Reporting+System/61688/
Scalable Shared Disks
http://technet.microsoft.com/en-us/library/ms345450.aspx
Deploying a Scalable Shared Database
http://msdn.microsoft.com/en-us/library/ms345584.aspx
SQLVelocity Blog on Scalable Shared Db’s
http://sqlvelocity.typepad.com/blog/2010/09/scalable-shared-data-base-part-1.html
Ola Hallengren’s Maintenance scripts
http://ola.hallengren.com/
Dev Nambi’s blog on Stats creation scripts for Read Only Report
http://devnambi.com/archive/2011/01/read-only-databases-prevents-stats/
Jeremiah Peschka on Federated DB’s generically
http://facility9.com/2011/03/29/building-our-own-federated-database
PWNSQL “Offloading reporting workload to active replica(s) in SQL Server 2011”
http://www.pnwsql.orgby Sunil Agarwal, Principal Program Manager, Microsoft, SQL Server Product Group
April 13th, 2011

Vendors
Network Load Balancers
F5
Citrix
Cisco

http://www.crysmanson.com	207
url_unknown	2
http://translate.googleusercontent.com	1

Online Reporting Architectures Behind Load Balancers

by Crystal Foor Manson on Apr 09, 2011

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 210

Statistics

Online Reporting Architectures Behind Load Balancers Presentation Transcript