SharePoint Server 2013 Farm Architecture and Performance by Ben Curry - SPTechCon


Published on

Technical Class:

SharePoint Server 2013 Farm Architecture and Performance by Ben Curry - SPTechCon

  2. 2. BEN CURRY / MANAGING PARTNER • Founding Partner of Summit 7 Systems, Inc. • Summit 7 Systems Lead Architect • Six time Microsoft® MVP for SharePoint Server • CISSP, Microsoft Certified Trainer, CCNA, MCITP: SharePoint Server • Author of several SharePoint books by Microsoft PRESSS • Master SCUBA Diver Trainer • curry/
  3. 3. OUR BACKGROUNDSummit 7 systems is a premier provider of consulting and implementation services specializing on theMicrosoft SharePoint Platform and FAST Enterprise Search. » Summit 7 Systems was chosen by KMWorld Magazine as one of the top 100 Companies that Matter in Knowledge Management along with companies such as Microsoft, Oracle and IBM. » Summit 7 Systems was named to the 2011 and 2012 CRN Next-Gen 250 List as a company bringing innovative processes, methodologies and models to the solution provider industry. » Top 1% of Microsoft Partners Worldwide » Summit 7 Systems was named #6 on the 2012 CRN Fast Growth 100 based on our 2009 – 2011 growth of over 930% per year. » ~ 50% of Technical Staff hold US Government SECRET Clearances. » Service Disabled Veteran Owned Small Business (SDVOSB).
  4. 4. PLATFORMS, SOLUTIONS & SERVICESSOFTWARE PLATFORMS SERVICES FAST Enterprise Search SharePoint QuickStart SharePoint 2007 Information Architecture and Governance Development SharePoint 2010 Upgrade and Migration SharePoint 2013 Branding and Design (User Experience) Office Platform Web Content Management Design and Deployment Sitecore SharePoint Search Custom Workflow or Web Part DevelopmentSOLUTION AREAS InfoPath Forms and Workflows SharePoint Platform Solutions Performance Baselines and Best Practices Optimization Enterprise Search Mapping Business Process to Software Platforms Enterprise Content Management Cloud Services Design and Provisioning Internet / Web Content Management Remote Support Contracts Extranet Solutions Compliance Quickstart Intranet Solutions Business Process Management Enterprise Project Management Exchange Server
  5. 5. COMMERCIAL PAST PERFORMANCE MEDICAL/PHARMA FINANCE ENERGY/CHEMICAL EDUCATION MANUFACTURINGUniversity of Utah Medical OneWest Bank Otter Tail Power Kentucky Community and Hawker-Beechcraft Center Technical College System BBVA Compass Nova Chemicals Kimberly-ClarkSt Jude Children’s Research Texas A&M University, Hospital WellPoint Champion Technologies Qatar Coca Cola Bottling Consolidated Nektar Therapeutic Bank Independent QuikTrip UNC-Charlotte SC Johnson Pfizer Kraton Escola de Sao Paulo, Macau, China Nucor Steel
  6. 6. STATE & LOCAL PAST PERFORMANCE CITY OF MINNESOTA COUNTY OF ALABAMA KENTUCKY COMMUNITY & GALVESTON CHARLOTTE, DEPARTMENT OF SANTA CLARA, MEDICAID TECHNICAL COLLEGE SYSTEM COUNTY, TEXAS NORTHTRANSPORTATION CALIFORNIA CAROLINA 1 Year 14 Months Strategy and Governance 18 Months Employee Portal 4 Months 8 Months 902TS Professional AvePoint DocAve SharePoint 2007 toServices Contract Holder Architecture, SharePoint 2007 to SharePoint 2010 SharePoint 2010 Migration SharePoint 2010 SharePoint 2010 Implementation and Migration Extranet Requirements, Requirements, Design SharePoint 2007 to Support SharePoint 2010 Public Planning and Design and Implementation BPOS-Dedicated SharePoint 2010 to SharePoint 2013 Farm Branding, Custom SharePoint 2010 Operations and Migration Development and Website Migration Maintenance through Buildout AnchorPoint Business Process Automation Business Data Catalog Operations and Development Project Management Office Maintenance through AnchorPoint SiteCore Web Content Management Operations and Maintenance through AnchorPoint
  7. 7. FEDERAL PAST PERFORMANCE SPACE AND NATO MEADS DEPARTMENT OF REDSTONE TEST MISSILE DEFENSE INTERNAL MISSILE DEFENSE MANAGEMENT VETERANS CENTER AGENCY REVENUE SERVICE COMMAND AGENCY AFFAIRS Sub to ERC and Bering Sub to Boeing, SRA Sub to ASD Sub to KAI Global Sub to Dell / Quest Sub to Dell / Quest Straits Information International and Software Software Technology Trident Group 2 Months 5 Months 9 Months 9 Months 22 Months 10 Months Custom Workflow SharePoint and K2 Quest SharePoint SharePoint 2007 toInformation Architecture Information Architecture Development Workflow Development Toolset Requirements, SharePoint 2010 Development Development / Troubleshooting Design and Migration SharePoint Branding and Implementation System Stabilization SharePoint Logical and UX SharePoint 2010 Physical Architecture Migration Dashboarding and Business Intelligence
  8. 8. AGENDASharePoint Server 2010 FarmArchitectureCapacity PlanningTesting Tools 9
  9. 9. SHAREPOINT SERVER 2010• Enterprise Search • Project Server• PerformancePoint • FAST for SharePoint• Excel Calc Services • Sandboxed Solutions• BCS • Office Web Apps• InfoPath • Web Applications• Word Conversions • Usage Logging• Visio Graphics • Access Services• Managed Metadata • Secure Store• User Profiles• Web Analytics
  10. 10. OLD SCHOOL 2007 Web Web Query Query Incoming Email Incoming Email Index Server Excel Services Document Conversions SQL Server SQL Server
  11. 11. SHAREPOINT SERVER 2013 FARM Web Web QueryIndex QueryIndex Incoming Email Incoming Email Viso Graphics Service Viso Graphics Service BCS BCS Managed Metadata Managed Metadata Crawler Crawler Excel Services Excel Services Document Conversions Document Conversions Project Server Project Server Secure Store Secure Store PerformancePoint PerformancePoint User Profiles User Profile Sync Web Analytics User Profiles SQL Server SQL Server Reporting Services Reporting Services Analysis Services Analysis Services
  12. 12. WEB APPLICATION ARCHITECTURE Site Site Site Site Collection Collection Collection Collection Content Content Content Database Database Database
  13. 13. WEB APPLICATION ARCHITECTURE Site Site Site Site Collection Collection Collection Collection Application SQL Content Content Content Database Pool Server Database Database
  14. 14. SERVICE APPLICATION ARCHITECTURE Web Application Service Application Proxy Service Service Service Application Machine Machine Instance Instance Service Instance Service Instance
  16. 16. SCALING SEARCHCrawl Server (component) › Temp Indexing › Connects to Crawl DatabaseQuery Server (component) › Index Partition and Query Component › Can be Primary or FailoverProperty DatabaseCrawl DatabaseCrawling Target Server (dedicated host for crawling)
  17. 17. SEARCH ARCHITECTURE Web Web Web QueryIndex QueryIndex - Crawl Target Misc Apps Misc Apps - Crawler - Crawler SQL Server SQL Server - SearchDB - SearchDB - PropertyDB - PropertyDB
  18. 18. CAPACITY & PERFORMANCE PLANNINGResearch and Study › your targetsCreate Logical ArchitectureDesign Physical ArchitectureTest and validateLog and MonitorAdjust architecture as needed
  19. 19. CACHING & COMPRESSIONHTTP CompressionBLOB CachingOutput CachingObject Caching
  20. 20. MY TOOLBOXInternet Explorer Developer ToolbarVisual Round-trip Analyzer › Free Download › Requires Netmon 2.xFiddlerDev DashboardVisual Studio Team Suite & Ultimate versions
  21. 21. DEVELOPER DASHBOARD$DevDashboardSettings =[Microsoft.SharePoint.Administration.SPWebService]::ContentService.DeveloperDashboardSettings;$DevDashboardSettings.DisplayLevel = OnDemand;$DevDashboardSettings.RequiredPermissions = EmptyMask;$DevDashboardSettings.TraceEnabled = $true;$DevDashboardsettings.Update()
  22. 22. WEB APPLICATION BOUNDARIES LIMIT MAXIMUM VALUE LIMIT TYPE NOTES Administrative operations such as creating a new site collection will experience decrease in performance.Content database 300 per Web application Supported Use Windows PowerShell to manage the Web application when a large number of content databases are present. The number of zones defined for a farm is hard-coded Zone 5 per Web application Boundary to 5. Managed paths are cached on the Web server, and Managed path 20 per Web application Supported CPU resources are used to process incoming requests against the managed path list.Application Pools 10 per Web Server Supported RAM and Workload
  23. 23. CONTENT DATABASE BOUNDARIES +200GB requires disk I/O of .25 IOPS per GB, 200 GB per content database. but 2 IOPS per GB is recommended.Content database size Supported **4TB w/ restrictions Robust DR strategy Strongly recommend limiting the number ofSite collections per content 2,000 recommended site collections in a content database to 2,000. Supporteddatabase 5,000 maximum *Up to 5,000 site collections in a database are supported. Time to first byte of any From the time that SharePoint Server 2010Remote BLOB Storage (RBS) response from the NAS cannot requests a BLOB, until it receives the first bytestorage subsystem on Network Boundary exceed 20 milliseconds from the NAS, no more than 20 millisecondsAttached Storage (NAS) can pass.
  24. 24. VERY LARGE SCALE DEMO14.4TB content database failed over in 45 seconds…
  25. 25. Document Size # ofDocuments
  26. 26. LISTS AND LIBRARIES BOUNDARIESFile size 2 GB Boundary The default maximum file size is 50 MB.Documents 30,000,000 per library Supported Varies depending on organization, type and size. This value may vary depending on the number ofItems 30,000,000 per list Supported columns in the list and the usage of the list. Maximum number of items that a database operationList view threshold 5,000 Threshold can simultantouesly process (outside the daily time window) Navigation and management performance willSubsite 2,000 per site view Threshold decrease significantly as the number of subsites grows.
  27. 27. SEARCH BOUNDARIESSharePoint search service 20 per farm Supported ..that’s more than you need!applications *10 crawl databases per search serviceCrawl databases and database The supported limit is 10 crawl databases application ThresholdItems per SharePoint Search service application. *25 million items per crawl database The total number of crawl components perCrawl components 16 per search service application Threshold server must be less than 128/(total query components) . 20 per search service application; 128 The boundary for the total number of indexIndex partitions Threshold total partitions is 128. 100 million per search serviceIndexed items application; 10 million per index Supported partitionCrawl log entries 100 million per search application SupportedProperty databases 10 per search service application ThresholdQuery components 128 per search application ThresholdConcurrent crawls 20 per search application Threshold
  28. 28. MINE 2007 USAGE CONTENTUsers › Browser Read • Operations › Browser Write – Indexing › Outlook RSS › Outlook Sync – Search Crawling › OneNote – Backup/Restore › Offline Clients › Collab – CooP › Management – Shared Services
  29. 29. DOCUMENT CURRENT CONTENTDatabase SizesNumber of content databasesNumber of site collections › Current Quotas!Number of Web appsSearch Index and DB sizeUser Profiles
  30. 30. DEFINE TARGETSServer availabilityWeb application availabilityService Application availabilityResponse TimesFour ‘9s’ is achievable › (but it is expensive…)
  31. 31. VISUAL STUDIO TESTING FOR IT PROS Visual Studio 2010 isn’t just for developers!
  32. 32. OVERVIEW OF INFRASTRUCTURE TESTINGTest single server or modify affinity/stickyFirewall ports from controller and agents tocontroller/agents/serversRemote Registry permissions to tested servers
  33. 33. WHY TEST?Ensure adequate resourcesTest performance impact of Web partsTest any major configuration change DON’T GUESS. KNOW!
  34. 34. MY FARM SETUP SharePoint Server 2010 FE01 FE02 SQL Server 2008 R2 (Web + Query) (Web + Query) Visual Studio Ultimate 2010 Windows Server 2008 R2 App01 APP01 (Applications) SQL Server
  35. 35. PREPARE VISUAL STUDIOSetup Results Database on SQL Server 2005 or 2008Remove default tests if desiredSetup the test controller and agents
  36. 36. BUILD A TESTING RIG’Requires Visual Studio 2010 Ultimate, Test, or Team Suite Editions › If you don’t see ‘Test’ as an option, you have the wrong versionMust have one controller › Use multiple agents for better real-world loadCreate new Project using the ‘Test Project’ templateCreate Load test results store (DB)Set loadtest database as the repository
  37. 37. CREATE A TEST PROJECT AND WIRE UP THE REPOSITORYTip: Use loadtestresultsrepository.sql to create the repository database: C:Program Files (x86)Microsoft Visual Studio 10.0Common7IDE’
  38. 38. TEST WEB PART IMPACTRecord Web test for the hosting pageBaseline load time › Change run settings to test browser version and WAN clientsLoad Web partTest response timeShould not be used for production testing, but will give you agood idea of Web part impact
  39. 39. CREATE LOAD TESTCreate load test using one or more Web tests › Choose % mix for your environment › For new implementations or complete unknowns, use the posted mix on TechNetDocument Load Test Settings › Be sure to use the same settings when comparing baselines and changesCreate Peak and off-peak baselines
  40. 40. READING THE RESULTSProcessor time and Disk I/O are criticalUse Systems Center Operations Manager management packdocumentation for performance counters to monitor during tests › Or rule author viewLoad Test Summary is usefulAdd custom counters for Search, etc.
  41. 41. LAST PERFORMANCE SLIDEAnalyze the ResultsIdentify bottlebecksOptimizeRinse, Lather, & Repeat
  42. 42. LOGGINGEvent LogsTrace LogsUsage LogsWeb AnalyticsDeveloper DashboardSCOM
  43. 43. MONITORINGHealth ReportsWeb AnalyticsAdministrative ReportsTimer Job StatusSQLIO Benchmark Tool
  44. 44. AGGREGATE PERFORMANCEAdd-SPDiagnosticsPerformanceCounter -Category"Processor" -Counter "% Processor Time" -Instance "_Total" -WebFrontEnd
  45. 45. WHAT ARE WE LOOKING FOR?Processor Upgrade processorProcessor - % Processor Time Over 75-85% Increase number of processors Add additional server(s)Disk Increase number or speed of disks Gradually increasing, system not in a steady state andAvg. Disk Queue Length Change array configuration to stripe queue is backing up Move some data to an alternative server Increase number of disks% Idle Time Greater than 90% Move data to an alternative disk or server Increase number of disks% Free Space Less than 30% Move data to an alternative disk or serverMemory Add memory. Note:Available Mbytes Less than 2GB on a Web server. SQL server available memory will be low, by design, and does not always indicate a problem. Add memoryCache Faults/sec Greater than 1 Increase cache speed or size if possible Move data to an alternative disk or serverPages/sec Greater than 10 Add memory
  46. 46. WHAT ARE WE LOOKING FOR?Paging File The server paging file, sometimes called the swap file, holds "virtual" memory addresses on disk. Page faults occur when a process has to stop and wait while% Used and % Used Peak Add memory required "virtual" resources are retrieved from disk into memory. These will be more frequent if the physical memory is inadequate.NIC Investigate further by monitoring Bytes received/sec and Bytes Sent/sec. Over 40-50% of network capacity. This is the rate at which data is sent andTotal Bytes/sec Reassess network interface card speed received via the network interface card. Check number, size, and usage of memory buffersProcessWorking Set Greater than 80% of total memory Add memory Increase number of processors% Processor Time Over 75-85%. Redistribute workload to additional serversASP.NET Make sure that you have not implemented settings that automatically recycle the applicationApplication Pool Recycles Several per day, causing intermittent slowness. pool unnecessarily throughout the day. Implement additional Web serversRequests Queued Hundreds or thousands of requests queued. The default maximum for this counter is 5,000, and you can change this setting in the Machine.config file As the number of wait events increases, users will experience degraded pageRequest Wait Time Implement additional Web servers rendering performance.Requests Rejected Greater than 0 Implement additional Web servers
  47. 47. PROCESSOR & PROCESSESProcessor This shows processor usage over a period of time. If this is consistently too high, you may find performance is adversely affected. Remember to count "Total" in% Processor Time multiprocessor systems. You can measure the utilization on each processor as well, to ensure balanced performance between cores.Process This counter indicates the current size (in bytes) of the working set for a given- Working Set process. This memory is reserved for the process, even if it is not in use. This counter indicates the percentage of processor time that is used by a given- % Processor Time process.Thread Count (_Total) The current number of threads.
  48. 48. PERFORMANCE COUNTERS You should monitor processor performance to ensure that all processor usage does not remain consistently high (over 80 percent) as this indicates that the system would not be able to handle any sudden surges of activity. And that in the common state, you willProcessor not see a domino effect if one component failure will bring the remaining components to a malfunctioning state. For example – if you have three Web servers, you should make sure the average CPU across all servers is under 60% so that if one fails, there is still room for the other two to pick up the extra load. Monitor the rate at which data is sent and received via the network interface card. This should remain below 50 percent ofNetwork Interface network capacity. There are a number of logical disk options that you should monitor regularly. The available disk space is essential in any capacity study, but you should also review the time that the disk is idle. Dependent on the types of applications or services you are runningDisks and Cache on your servers, you may review disk read and write times. Extended queuing for write or read function will affect performance. The cache has a major impact on read and write operations. You must monitor for increased cache failures. Monitor the amount of physical memory available for allocation. Insufficient memory will lead to excessive use of the page file andMemory and Paging File an increase in the number of page faults per second.
  49. 49. DISK COUNTERS This shows the average number of both read and write requests that were queued for the selected disk during the sample interval. A bigger disk queue length may not be a problem as long as diskAvg. Disk Queue Length reads/writes are not suffering and the system is working in a steady state without expanding queuing.Avg. Disk Read Queue Length The average number of read requests that are queued.Avg. Disk Write Queue Length The average number of write requests that are queued.Disk Reads/sec The number of reads to disk per second.Disk Writes/sec The number of writes to disk per second.
  50. 50. MEMORY COUNTERS This shows the amount of physical memory available for allocation. Insufficient memory will lead to- Available Mbytes excessive use of the page file and an increase in the number of page faults per second. This counter shows the rate at which faults occur when a page is sought in the file system cache and is not found. This may be a soft fault, when the page is found in memory, or a hard fault, when the page is on disk.- Cache Faults/sec The effective use of the cache for read and write operations can have a significant effect on server performance. You must monitor for increased cache failures, indicated by a reduction in the Async Fast Reads/sec or Read Aheads/sec. This counter shows the rate at which pages are read from or written to disk to resolve hard page faults. If- Pages/sec this rises, it indicates system-wide performance problems.Paging File The server paging file, sometimes called the swap file, holds "virtual" memory addresses on disk. Page- % Used and % Used Peak faults occur when a process has to stop and wait while required "virtual" resources are retrieved from disk into memory. These will be more frequent if the physical memory is inadequate.
  51. 51. ASP.NETRequests Total The total number of requests since the service was started. Microsoft SharePoint Foundation 2010 provides the building blocks for HTML pages that are rendered in theRequests Queued user browser over HTTP. This counter shows the number of requests waiting to be processed. The number of milliseconds that the most recent request waited in the queue for processing. As theRequest Wait Time number of wait events increases, users will experience degraded page-rendering performance. The total number of requests not executed because of insufficient server resources to process them. ThisRequests Rejected counter represents the number of requests that return a 503 HTTP status code, indicating that the server is too busy.Requests Executing (_Total) The number of requests currently executing. The number of requests executed per second. This represents the current throughput of the application.Requests/Sec (_Total) Under constant load, this number should remain within a certain range, barring other server work (such as garbage collection, cache cleanup thread, external server tools, and so on).
  52. 52. SQL SERVER This object provides counters to monitor general server-wide activity, such as the number of current connections and the number of usersGeneral Statistics connecting and disconnecting per second from computers running an instance of SQL Server. This counter shows the amount of user connections on your instance of SQL Server. If you see this number rise by 500 percent from yourUser Connections baseline, you may see a performance reduction. This object provides counters to monitor bulk copy operations, backup and restore throughput, and transaction log activities. Monitor transactions and the transaction log to determine how much user activity is occurring in the database and how full the transaction log isDatabases becoming. The amount of user activity can determine the performance of the database and affect log size, locking, and replication. Monitoring low-level log activity to gauge user activity and resource usage can help you to identify performance bottlenecks. This counter shows the amount of transactions on a given database or on the entire SQL Server instance per second. This number is to helpTransactions/sec you create a baseline and to help you troubleshoot issues.Locks This object provides information about SQL Server locks on individual resource types.Number of Deadlocks/sec This counter shows the number of deadlocks on the SQL Server per second. This should normally be 0.Average Wait Time (ms) This counter shows the average amount of wait time for each lock request that resulted in a wait.Lock Wait Time (ms) This counter shows the total wait time for locks in the last second.Lock Waits/sec This counter shows the number of locks per second that could not be satisfied immediately and had to wait for resources. This object provides counters to monitor compilation and the type of requests sent to an instance of SQL Server. Monitoring the number ofSQL Statistics query compilations and recompilations and the number of batches received by an instance of SQL Server gives you an indication of how quickly SQL Server is processing user queries and how effectively the query optimizer is processing the queries. This object provides counters to monitor how SQL Server uses memory to store data pages, internal data structures, and the procedureBuffer Cache cache, as well as counters to monitor the physical I/O as SQL Server reads and writes database pages.
  53. 53. FOR MORE FROM SUMMIT 7SYSTEMS… @summit7systems summit 7 systems summit7systems