Building SSRS 2008 large scale solutions

5,251
-1

Published on

Building SSRS 2008 Large Scale Solutions as per SQLPASS 2008 Summit w/ Lukasz Pawlowski and Denny Lee

1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total Views
5,251
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
130
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide
  • Front ends connected to same cluster of databasesContent switch allows for automatic failover of SSRS servers (IP address remapping)Mirrored databases on disaster recovery site asynchronously (some metadata loss is okay)Manual failover from primary to disaster recovery siteDatabase Instance names are the same (e.g. REDMOND\\sql4, BAY\\sql4)
  • By default, RS uses Snapshot data stored in RSTempDB to render reportsTo be efficient, data is spread across over small logical divisions of dataBy default, server must query RSTempDB to get a snapshot chunkAs user load increases, perf degradationSolution: FS SnapshotsCreate file system chunks as cache for snapshot chunksi.e. hit the RS server file system for data instead of always hitting RSTempDBNote, recommended load balancing solution has affinity (e.g. cookie persistence) user sessions to RS node to access FS chunks
  • Building SSRS 2008 large scale solutions

    1. 1. Denny Lee, Lukasz PawlowskiSQL Customer Advisory TeamSQL Server Reporting ServicesBuilding SSRS 2008 LargeScale Solutions PASS Community Summit 2008 November 18 – 21, 2008 Seattle WA
    2. 2. SQL Server Customer Advisory Team (SQLCAT) Works on the largest, most complex SQL Server projects worldwide – US: NASDAQ, Progressive, Premier Bankcard, Hilton Hotels – Europe: Barclays Capital, Danske Bank, McLaren, Bwin – Asia/Pacific: Korea Telecom, GMarket, Japan Railways East, China Mobile – LATAM: Banco Itau, Oi – Strategic ISVs: SAP, Siebel, JDE, PeopleSoft, GE Healthcare, SunGard, Siemens, Dynamics and more Drives product requirements back into SQL Server from our customers and ISVs Shares deep technical content with SQL Server community – SQLCAT.com – http://blogs.msdn.com/sqlcat – http://blogs.msdn.com/mssqlisv – http://technet.microsoft.com/en-us/sqlserver/bb331794.aspx PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 2
    3. 3. SQL Server Design Win Program  Target the Most Challenging and Innovative Applications on SQL Server  Investing in Large Scale, Referenceable SQL Server Projects Across the World – Provide SQLCAT technical & project experience – Conduct architecture and design reviews covering performance, operation, scalability and availability aspects – Offer use of HW lab in Redmond with direct access to SQL Server development team  Work with Marketing Team Developing Case StudiesPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 3
    4. 4. Session Objectives and Takeaways Session Objective(s): – Provide guidance on how to scale out your Reporting Services environment – Provide RS best practices on RS catalogs, scale out deployment, and performance optimizations Agenda: – Reporting Services Scale Out Architecture – Report Catalog Best Practices – Scale Out Deployment Best Practices – Performance Optimization ConfigurationsPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 4
    5. 5. Reporting Services Scale Out ArchitecturePASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 5
    6. 6. Scale Out Architecture: Overall Architecture Report Server RS Scale Out Deployment Clients RS Server Report Catalog Reporting Data Flat Files, OLE DB, ODBC NLBClients RS Server RSDB SQL, AS, DB2, Oracle, RS Server Teradata, etc. Clients PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 6
    7. 7. Scale Out Architecture Read the manuals! A lot of documentation on SSRS available online Many mistakes in implementation could have been avoided Read these: – Planning for Scalability and Performance with Reporting Services – Upgrading Reporting Services (SQL Books Online) – Configuring a Report Server Scale-Out Deployment On sqlcat.com – Building and Deploying Large Scale SQL Server Reporting Services Environments SeriesPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 7
    8. 8. Scale Out Architecture: Enterprise Rent-A-Car Customer Scenario Report Server RS Server RS Scale Out Deployment AS Server Report Catalog Reporting Data RS Server Teradata RSDB RS Server AS Server RS ServerPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 8
    9. 9. Scale Out Architecture: Enterprise Rent-A-Car Customer Scenario Build test system that accurately represented production – Goal: 1800 concurrent users  using VS test  10s think time  Mean 33-36s txn time – Testing allowed them to identify blocking issues  drop down parameter lists of thousands of rows for areas and branches  Developed accurate workload representation (e.g. Proclarity and SSRS clients) Currently in production This presentation incorporates lessons learned from this and other customersPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 9
    10. 10. Scale Out Architecture Importance of Performance Testing Need to understand your scenarios and reports – Scenarios are defined by user personas & usage patterns – Reports are either test reports or actual reports – Tests should isolate Report Server from other systems Need tools to automate the testing – See white paper: Using Visual Studio 2005 to Perform Load Testing on a SQL Server 2005 Reporting Services Report Server – Make single incremental changes between tests – Do not use SQL trace inside VSTEPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 10
    11. 11. Scale Out Architecture Customer Performance Testing RS User Mean Tx Mean Servers s Time CPU% Max # of Concurrent Users 1 Server 608 36.9 99 2500 2 servers 1218 36.8 96 2300 2000 4 servers 2300 30.5 80 1500  8GB RAM, 2 dual core RS 1218 1000 servers, Windows 2003 500  Graph is max # of users 608 reached for sustained time 0 period (>=15 min) 1 server 2 servers 4 servers  2x RAM and CPU core, only Users 1/3 increase in loadPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 11
    12. 12. Scale Out Architecture RS 2008 vs. RS 2005: Lessons Learned RS 2008 Front-End Server Scales Up Much Better than RS 2005 – Able to respond to 3–4 times the total number of users and their requests without errors on the same hardware for all renderers – RS 2008 consistently outperformed RS 2005 with the PDF and XLS renderers on the four-processor, quad-core hardware platform See: Scaling Up RS 2008 vs. RS 2005: Lessoned LearnedPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 12
    13. 13. Scale Out Architecture RS 2008 vs. RS 2005: Lessoned Learned Avg. Response Time (lower is better) 250 4x4 2008 Mix 4x4 2005 Mix 200 4x2 2008 Mix 4x2 2005 Mix 2x2 2008 Mix 150Avg. Response Time (ms) 2x2 2005 Mix 100 50 0 User Load PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 13
    14. 14. Scale Out Architecture Scaling Up and Scaling Out with RS 2008 (cont) RS 2008 – Scale up front-end server to four-processor, quad-core servers for performance – Scale out to a two-node deployment for high availability – Optimize disk I/O subsystem on all RS 2008 boxes for maximum performancePASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 14
    15. 15. Reporting Catalog Best PracticesPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 15
    16. 16. Report Catalog Best Practices Report Server RS Scale Out Deployment Clients RS Server Report Catalog Reporting Data Flat Files, OLE DB, ODBC NLBClients RS Server RSDB SQL, AS, DB2, Oracle, RS Server Teradata, etc. Clients PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 16
    17. 17. Report Catalog Best Practices Report Server Catalog Breakdown Report Server Catalog (RSDB) Stores all report metadata including report Report Catalog definitions, report / history snapshots, scheduling, etc. RSDB RS Temp DB Stores temporary snapshots while running reports  These databases can be a bottleneck  Optimize by applying standard SQL DB techniques  Catalog has a lot of I/O and transactions – RS2005: Many inserts to ChunkData, SnapshotData, and SessionData tables – RS2008: Many inserts Segment; takes majority of transactions of RSTempDBPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 17
    18. 18. Report Catalog Best Practices Use a dedicated server Common scenarios  Same server as SSRS Server – Great for small environmentsRSDB – In enterprise environments, too much resource contention  Same server as data source database – SQL resource contention (TempDB, plan cache, memory buffer pool) between data source and RS catalogs – As load increases need to monitor CPU, I/O, network resources, and buffer pool  Reduce resource contention by having a dedicated RS catalog server you can tune.  Apply standard high availability and disaster recovery (e.g. clustering, mirroring, log shipping) to protect the RSDBPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 18
    19. 19. Report Catalog Best Practices High Performance Disk  Check out Predeployment I/O Best Practices  Have more smaller size disks with faster rotation speeds (>=15k RPM) vs. fewer larger disks with slower rotationsRSDB  Maximize/balance I/O across ALL available spindles  Separate disks between RSDB and RSTempDB – RSDB a lot of small transactions (report metadata) – RSTempDB has more (not as many) larger transactions  Pre-grow your databases  Stripe dB files to number of cores (0.25 – 1.0) – Minimize allocation contention – Easier to rebalance database when new LUNs are available  Use RAID 10, not RAID 5PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 19
    20. 20. Report Catalog Best Practices Operations Best Practices  Data in RSTempDB is highly volatile – Report lifetime policy of data = SessionTimeout value (10min) – CleanupCycleMinutes guides background cleanup threadRSDB – Once session timeout reached, cleanup temporary snapshot from tempDB – This is done every CleanupCycleMinutes  Data is RSDB is long lived; should be backed up – Backing Up and Restore Databases in SQL Server – Optimizing Backup and Restore Performance in SQL Server – Backing Up and Restore Encryption Keys  Maintain your RS catalogs – Remember, these are SQL databses – E.g. Re-indexing catalog tables or updating stats may improve query performancePASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 20
    21. 21. Report Catalog Best Practices Report Catalog Sizing  RSDB database size – Varies by number of reports published and number of history snapshots – General rule of thumb:  Moderate size report definition takes 100-200KB of disk spaceRSDB  This is larger than the actual RDL  SSRS persists both RDL and compiled binary  Assume 5:1 compression ratio (e.g. 10MB of data, snapshot is 2MB in size)  RSTempDB database size – Varies by number of users whom are concurrently using the Report Servers – Each live report execution generates report snapshot persisted in the RSTempDB – General rule of thumb:  10-20% concurrency of user base  E.g. 1000 users, then max 200 concurrent users.  If most users are accessing 10MB reports, then you will need 400MB of storage – 200 users x 10MB reports / 5:1 compression ratio= 400MB  Want to calculate for the maximum number of concurrent usersPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 21
    22. 22. Disaster Recovery Primary Data Center Content Switch Automatic Failover SSRS SSRS Manual FailoverFailover Cluster RSDB Async RSDB RSDB Mirroring
    23. 23. Scale Out Deployment Best PracticesPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 23
    24. 24. Scale Out Deployment Best Practices Report Server RS Scale Out Deployment Clients RS Server Report Catalog Reporting Data Flat Files, OLE DB, ODBC NLBClients RS Server RSDB SQL, AS, DB2, Oracle, RS Server Teradata, etc. Clients PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 24
    25. 25. Scale Out Deployment Best Practices RS 2005: File System Snapshots  RS TempDB has a lot of transactions to keep report consistency (i.e. cached reports)  Reduce RS Catalog I/O with File SystemRS Server Snapshots – It will store data on file system – Unlike RS/IIS setup, will require more disk spaceRS Server  To enable, update RSReportServer.config file: – <Add Key="WebServiceUseFileShareStorage" Value="true" />RS Server – <WindowsServiceUseFileShareStorage>True</Wi ndowsServiceUseFileShareStorage>PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 25
    26. 26. Scale Out Deployment Best Practices RS 2005: File System Snapshots  RS TempDB has a lot of transactions to keep report consistency (i.e. cached reports)  Reduce RS Catalog I/O with File SystemRS Server Snapshots – It will store data on file system – Unlike RS/IIS setup, will require more disk spaceRS Server  To enable, update RSReportServer.config file: – <Add Key="WebServiceUseFileShareStorage" Value="true" />RS Server – <WindowsServiceUseFileShareStorage>True</Wi ndowsServiceUseFileShareStorage>PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 26
    27. 27. Scale Out Deployment Best Practices RS 2008: Why not File System Snapshots?  SSRS 2005 – Advantage for SSRS 2005 because enabling feature allowed less hits to RSTempDBRS Server – Entire report was calculated when requesting first page  SSRS 2008 caches a lot of this data into memory – Data continually persisted in report catalogsRS Server – Local file system acts as a write-through cache – Does not pre-calculate everything on initial request – On-demand engine retrieves all of the data and places into RSTempDB for consistencyRS Server – But many calculations are done on-demand as needed vs. pre-calculated and stored.  Still want to test in your environmentPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 27
    28. 28. Scale Out Deployment Best Practices Cache Execution  Recurring theme on effective user of memory and minimal I/O  To help reduce I/O further, enable cache executionRS Server on your reports.  By default, reports are live execution  Turn on cache execution for each report so theRS Server report is stored in memory (thus reduced disk I/O)  E.g. Even if you cache report every 5 minutes, potentially a 80% reduction in I/ORS Server – If report is hit every minute, now only I/O hit every 5 minutes, i.e. 20% of the time  No global setting for cache executionPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 28
    29. 29. Scale Out Deployment Best Practices Load Balance your Network  Load balancing important for many client connections to RS servers RS Scale Out Deployment Clients  Recommend: Use cookie RS Server persistence to preserve SSRS-to- client connection – IP affinity can work but may be NLB overload for browser-basedClients connections RS Server – Makes use of SSRS file cache – Keep round-robin for initial connections RS Server  Recommend: dual NIC for RS Clients – Split browser and AS/DB traffic PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 29
    30. 30. Scale Out Deployment Best Practices Isolate your workloads Report Server Interaction RS Scale Out Deployment NLB Clients RS Server Report Catalog Reporting Data SchedulingBenefits: RSDBPredictable Workloads RS Scale Out DeploymentHelps with Security ModelIsolate Performance Issues RS Server PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 30
    31. 31. Scale Out Deployment Best Practices Report Data Performance Considerations  Scale out works for RS but may not work for underlying Report Data (data source)  Reporting loads Report Data, limit impact of large numbers of users Reporting Data Flat Files, OLE DB, – Limit data set size using report filters ODBC – SSIS limited data from Operational data sources – Do not let all users access all of the reports – E.g. Report Builder against Analysis Services results in many queries being executed. SQL, AS, DB2, Oracle,Teradata, etc.PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 31
    32. 32. Scale Out Deployment Best Practices Report Data Performance Considerations Additional resources to scale out SQL and SSAS  Deploying a Scalable Shared Database  SQL Server Replication: Providing High Availability using Database Mirroring Reporting Data Flat Files, OLE DB, ODBC  Database Mirroring and Log Shipping  SQL Server Replication Features  Scale-Out Querying with Analysis Services SQL, AS,  Scale-Out Querying with Analysis Services Using DB2, Oracle,Teradata, etc. SAN Snapshots  Scaling out an Analysis Services SolutionPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 32
    33. 33. Performance Optimization ConfigurationsPASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 33
    34. 34. Performance Optimization Handling Large Workloads  Control Size of reports – Do you need them? – Is this really a data feed?RS Server – Aggregate reports and remove unused columns  Recommendations – Cache Execution – Report Execution Timeouts – Scheduled snapshots for large reports with data processing bottlenecks – Delivered Rendered reports for non-browser formats – Pre-populate report cache using data driven subscriptions PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 34
    35. 35. Performance Optimization Large Workload Tuning  Analyze your reports – Use ExecutionLog2 View  Back to Report CatalogsRS Server – Increase size of your report catalogs to store more snapshot data  Tune the web service – SSRS 2005: Tune IIS – SSRS 2008: Tune HTTP.sys  Windows 2003  Windows 2008 PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 35
    36. 36. Performance Optimization ExecutionLog2 Analysis Checklist Sort by ElapsedSec or RowCount  Sort by Instance for long running reports – Determine if NLB is handling request in – TimeDataRetrieval: If high, need to balanced fashion optimize data source  Sort by Report Path & Timestart – High RowCount: A lot of data to determine report pattern aggregated by SSRS, have SQL do this – E.g. Expensive report (takes 5 minutes to run) running every 10 minutes Sort by Request Type  Sort by Status – A lot of subscriptions, can determine bottlenecks and stagger reports – Failures occur before (e.g. incorrect RDL) or after (e.g. subscription delivery Sort by source error) report is processed – To determine if live data or snapshot – Outdated information or settings (e.g. – If report can be snapshot (e.g. last expired passwords, missing week’s report), create snapshot to subreports, etc.) avoid query execution, report  Data Driven Subscriptions processing, and rendering – Errors > 5% – Continual Scale ModePASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 36
    37. 37. Monitoring Reporting Services by theExecutionLog2 View
    38. 38. Performance Optimization Should we use 64-bit? Yes!  Report Server Catalog – Standard Database techniques for optimizationRS Server – Since SQL 2005, database written natively for 64-bit  Report Server Service – Most reports memory intensive – Note, some workloads (e.g. many small reports) 32-bit can execute faster – Handle more concurrent report users or more large reports – Able to more effectively use memory in SSRS 2008 – Will spill to file system if hits memory pressure – Exceptions:  Certain data provides not available for 64-bit PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 38
    39. 39. Performance Optimization SSRS 2008 Memory Configurations  Uses memory more efficiently; under intensive workload pressure, it uses the file system cache. E.g., small requests will continue to stay in memory while long running request will go to disk  Therefore, before looking at the file system, check these memoryRS Server configurations: – WorkingSetMinimum / WorkingSetMaximum:  Minimum / Maximum amount of physical memory that RS will make available to perform its task;  KB value within RSReportServer.config  Increase value to process more requests in memory  After WorkingSetMaximum is reached and exceeded for a period of time, recycle app domains to reduce memory utlization – MemorySafetyMargin:  Defines boundary between low/medium pressure scenarios  Default 80% value in RSReportServer.config – MemoryThreshold:  Defines boundary between medium/high pressure scenarios  Default 90% value in RSReportServer.config PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 39
    40. 40. Performance Optimization SSRS 2005 Memory Configurations Recall, SSRS 2005 does not scale up as well as SSRS 2008  MemoryLimit Configuration • Default 60% of physical memoryRS Server • Increase help process more requests • Once threshold hit, no new requests are accepted  MaximumMemoryLimit Configuration • Default 80% of physical memory • If this threshold is met, processing is aborted  Changing values may solve RS only to bring up other contentions  Recommendation: If constantly hitting memory thresholds, consider scaling up and then scaling out PASS Community Summit 2008 BI-401-A Building SSRS 2008 Large Scale Solutions 40
    41. 41. Thank youfor attending this session and thePASS Community Summit 2008 PASS Community Summit 2008 November 18 – 21, 2008 Seattle WA
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×