Bca1931 final

BCA1931Design, Deploy, and OptimizeSharePoint 2010 on VMwareItzik Reich, EMC CorporationAlex Fontana, VMware, Inc.

Disclaimer This session may contain product features that are currently under development. This session/overview of the new technology represents no commitment from VMware to deliver these features in any generally available product. Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. Technical feasibility and market demand will affect final delivery. Pricing and packaging for any new technologies or features discussed or presented have not been determined.2

Agenda Introduction and Benefits SharePoint on vSphere Performance SharePoint on vSphere Capacity Planning • Workload Modeling and Architectural Design • SQL Server Capacity and Performance • Deploying to ESX/ESXi SharePoint on vSphere Availability and Recovery • High Availability • Disaster Recovery • Backup and Recovery Customer Case Study with EMC More Information3

SharePoint – What is it? Mostly an intranet portal for collaboration It can be customized for all sorts of uses Varies from insignificant to critical http://www.topsharepoint.com4

Key Benefits – Virtualizing SharePoint Consolidation  Achieve 2-10x consolidation ratio, especially for larger deployments Performance  Improved front end performance with more, smaller WFEs rather than few large WFEs Availability  VM-based protection for SharePoint provides homogeneous high availability (VMware HA) Business Continuity  Simplified DR management with vCenter Site Recovery Manager Maintenance  Live migration of SharePoint virtual machines (VMware vMotion) Load Balancing  Maximized overall performance with balanced HW utilization across the farm (VMware DRS)5

Virtualizing Server Roles in SharePoint Server Roles/Priority What to Consider 1st Web Front End / Query CPU – User concurrency, Search requests Scaling out is more efficient Network – segment vNICs and vSwitches 2ndApplication (Excel, Doc Conv, etc) CPU – Application dependent Scaling out is more efficient 3rd  Redundant (Non redundant in MOSS 2007) Index/Crawl  CPU – Crawling, indexing (depends on content type/size)  Scale out (Up only with MOSS 2007)  Memory intensive CPU – Document updates, Search, Backup 4th VMFS/RDM SQL Scale up/out (VMware ≤32 vCPU) Failover Clustering, Mirroring, VMware HA Understanding your existing workload is better than any best practice!!! 6

Maximum Scalability and Performance With vSphere 5 Application’s Performance Requirements 95% of Apps VMware Inf. VMware VMware ESX 1 ESX 2 Require 3.0/3.5 vSphere 4 vSphere 5% of Applications CPU 1 to 2 CPUs 1 VCPUs 2 VCPUs 4 VCPUs 8 VCPUs 32 VCPUs Memory < 4 GB at peak 2 GB per VM 3.6 GB per VM 16/64 GB per VM 256 GB per VM 1,000 GB per VM Network <2.4 Mb/s <.5Gb/s .9 Gb/s 9 Gb/s 30 Gb/s >36Gb/s IOPS < 10,000 <5,000 7,000 100,000 300,000 1,000,000 8

SharePoint Performance – The User Experience Domain Controller SQL Server Authentication Document Storage Web Front End Request − Content/Metadata − Search Type Of Acceptable user − System Examples Server operation Network Client response time • Browsing to the home page ‒ CPU Common <3 seconds • Browsing to a document library BLOB ‒ Memory • Creating a subsite Creating a list Uncommon ‒ HBA/CNA • <5 seconds Storage ‒ NIC Uploading a document to a document library • Backing up a site (Optional) Rare • Creating a site collection <7 seconds BLOB http://technet.microsoft.com/en-us/library/cc287790(office.12).aspx Retrieval/Creation9

SharePoint 2010 Performance Test Logical Architecture Workload • Root portal configured with collaboration template • 260GB content, approximately 600K items in 10 site collections • Incremental crawl every four hours and weekly full crawl VSTS load generator – Zero think time (per Microsoft guidelines) Transaction mix – 80-10-10 read-write-search Real world settings – IIS logging on, SharePoint caching disabled10

Physical versus Virtual Study Physical versus virtual Web front end (WFE) comparison shows the overall request per second (RPS) differs very little, even at higher CPU saturation levels Physical Versus Virtual WFE CPU Comparison 50 45 Physical Virtual 40 35 RPS Physical 30 Virtual 25 Physical 20 Virtual 15 10 5 0 1 CPU (95%+ Saturation) 2 CPU (75-90% Saturation)11

Scaling Out the SQL Server Back-End 60-20-20 Mix Test compares one SQL instance versus two SQL instances Scaling out SQL Server provides better throughput! Test with your workload for best SQL server scale out throughput12

Performance Monitoring vSphere Client: • GUI interface, primary tool for observing one or more ESX/ESXi hosts • Does not require high levels of privilege Resxtop/Esxtop • Gives access to detailed performance data of a single ESX/ESXi host • Provides fast access to a large number of performance metrics • Requires root-level access • Runs in interactive, batch, or replay mode In-guest Monitoring tools • SQL Server: Perfmon, Profiler, Dynamic Manage Views • Use ESX Counters in PerfMon for more accurate results - http://vpivot.com/2009/09/17/using-perfmon-for-accurate-esx-performance-counters/13

Capacity Planning Process Summary Estimate User Activity Select a starting point architecture Map out resource requirements by server role (virtual machine requirements) Plan the ESX/ESXi host hardware configuration (ESX/ESXi host requirements) Perform initial placement exercise to verify resource allocations and failover headroom15

Estimating User Activity Upgrading from SharePoint 2007 • Mine IIS logs and utilize Microsoft or 3rd party testing tools • SharePoint 2010 Load Testing Kit http://technet.microsoft.com/en-us/library/ff823736.aspx. • Visual Studio 2008 Team System http://www.microsoft.com/downloads/en/details.aspx?FamilyID=d95598d7-aa6e-4f24- 82e3-81570c5384cb&DisplayLang=en. • Visual Studio 2008 Service Pack 1 http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=10986 New installation Enterprise Intranet Collaboration Environment Technical Case Study Example http://technet.microsoft.com/en-us/library/ff758650.aspx • Requests per second Workload Characteristics Value Average daily RPS 157 • Concurrent users Average RPS at peak time 350 • Total daily users Total number of unique users per day 69,702 • Total daily requests Average daily concurrent users 420 Peak concurrent users at peak time 1,433 Total number of requests per day 18,866,52716

Selecting a Starting Point Architecture SharePoint 2010 topologies SharePoint 2010 Medium Topology Example • Published Microsoft topologies from small to large enterprise farms • Use recommended role requirements to plan resource allocation for VMs • http://technet.microsoft.com /en-us/library/cc263044.aspx SharePoint Server 2010 technical case studies • Published Microsoft technical case studies illustrating existing production environments • Select the case study that is most applicable to your organizations expected usage patterns http://technet.microsoft.com/en-us/library/cc261716.aspx.17

SharePoint Farm Topologies Small le out approach = More servers ? Sca Medium LargeWeb Web Servers Groups MOSS SP H/W 2007 2010 RAM >2GB 8GB >3.0GHz >2.5GHz CPU Web/Query Web User requests Crawling/Admin Dual Quad Features that impact SQL Server Sizing • The size of content databasesApplication App Servers Groups MOSS •SP The addition of service applications or features H/W 2007 2010 into the environment RAM 4GB 8GB • The use of SQL Server mirroring >2.5GHz >2.5GHz App Query/Crawl App Query Crawl Central Admin CPU Dual • Quad The frequent use of files larger than 15MB /Office/OtherDatabase MOSS SP H/W 2007 2010 RAM >2GB 8 - 64GB >2.5GHz CPU >2.0GHz All DBs Quad Search DBs SharePoint DBs Search DBs SharePoint DBs Content DBs 18

SQL Server Capacity and Performance19

A Day in the life of SharePoint…SQL Server CPU The majority of load comes from systematic operations…20

A Day in the life of SharePoint…SQL Server Storage I/O Plan for user load peaks, not systematic peaks…21

Database Sizing Central Administration • 2GB capacity; minimal disk throughput required Configuration Database • 2GB capacity; minimal disk throughput required • Can slowly grow beyond 1GB; approx. 40MB for every 50K site collections • Transaction logs can be large; change recovery model from full to simple unless mirroring Content Databases Database size = ((D × V) × S) + (10KB × (L + (V × D))) • D = the number of documents you expect to host • S = the average size of each document • L = the number of list items in the environment; start with 3 X D and adjust • V = the approximate number of document versions22

SQL Server Storage Best Practices Plan for performance in addition to capacity Search database and temp database are the most demanding for disk I/O. Search database is write intensive when crawling. When possible, place SQL transaction log and database files on physically separate disk pools Place transaction log files on RAID1/0 volumes/pools for high write performance and faster rebuilds Most of SharePoint data (content databases) can use RAID5 volumes • RAID5 for more read intensive workloads (common, mainly publishing farms) • RAID1/0 for higher random write workloads (heavy collaboration, tempdb, search) • RAID 6 usually for higher availability with large amount of drives (Virtual Pools)23

Deploying to ESX/ESXi24

Virtual Machine Resource Allocation Virtual CPUs • Allocate the minimum requirement and adjust as needed; use HotAdd. • If overcommitting processors, monitor %RDY, %MLMTD, and %CSTP • Keep NUMA node size in mind with sizing virtual machines Virtual Memory • “Right-size” memory allocations for efficient use of host memory • Use vSphere 4.1 to take advantage of memory compression • If overcommitting memory, monitor SWAP /MB: r/s, w/s and MCTLSZ Storage • Understand I/O requirements for each application tier to avoid performance degradation due to under-provisioned storage • Use redundant paths to storage – Dual host-bus adapters or teamed network interface cards connected to separate switching infrastructures • Avoid partition misalignment by creating VMFS partitions from within the vSphere client – If creating VMFS from the CLI use fdisk to align25

Sample Architecture on vSphere Based on Microsoft’s departmental collaboration environment technical case study (SharePoint Server 2010) at http://technet.microsoft.com/en-us/library/ff758649.aspx The SQL Server configuration exceeded vSphere 4.1 limitations for vCPU allocation (maximum of 8 in vSphere 4.1), this large SQL Server was split into two SQL Server virtual machines with 8vCPUs eachDepartmental Collaboration Environment Technical Case Study on vSphere Sample Architecture26

SharePoint 2010 Availability What to protect? (Service Level Agreements) • Recovery Point Objectives (RPO) • Recovery Time Objectives (RTO) • Recovery Level Objectives (RLO) How to protect? • Tools and technologies available from SharePoint 2010 natively • VMware vSphere additions28

High Availability29

SharePoint 2010 with Application-Aware HA Protects all SharePoint server roles from hardware and application failure Does not require complex cluster setup or standby resources Fully integrated with VMware HA and vCenter30

VMware HA and High Availability Database Mirroring SharePoint 2010 is mirroring-aware Provides redundancy for SharePoint 2010 databases Protection against HW/SW failures and DB corruption Storage flexibility (FC, iSCSI, NFS) RTO in few seconds VMware HA + Database Mirroring • Seamless integration, virtual machines rejoin mirroring session after VMware HA recovery • Can shorten time that database is in unprotected state • Reduces synchronization time after virtual machine recovery31

Disaster Recovery32

Disaster Recovery with Site Recovery Manager (SRM) Relies on storage replication Allows creation, maintenance, and execution of automated process to facilitate site recovery Safe testing without impacting production environment Improves hardware utilization with co-located test/dev with DR Self-documenting33

Backup and Recovery34

What to Backup SharePoint 2010 Server Farm Servers Front End, Application, Index, Search, SQL SQL Server Databases Web Applications Site Collections SitesConfiguration, Search, Serv ices, and so on Content Databases Lists (document libraries, events, contacts, and the like) Documents and Items35

VMware Data Recovery (VDR) Quick, simple, and complete data protection for your SharePoint VMs with VDR, a disk-based backup and recovery solution Integrated with vCenter to enable centralized and efficient management of backup jobs Useful for small environments Can be used for SQL Server if the service is STOPPED36

SharePoint 2010 Backup using EMC Replication Manager37

Summary vSphere provides the foundation for high performance SharePoint environments Virtualized SharePoint instances perform very well compared to equally sized physical instances Tests of both Web front-end and SQL virtual machines show scaling out can provide increased throughput Monitoring virtualized SharePoint remains the same as a physical deployment with additional visibility into the underlying infrastructure Use VMware HA to protect SharePoint from downtime; for higher availability, consider: • Symantec Application HA for more granular control at the service level • Combining VMware HA with SQL Server Mirroring Use SRM for site recovery; co-locate test/dev and recovery VMs38

Customer Use Case A Global company – 50,000 Employees Is among the top 15 companies in it’s segmentation Designed and Implemented fully virtualized SharePoint Solution for 120,00 Seats based on VMware vSphere & EMC Technologies Solution Building Blocks Replication Manager SRM 4.1 vSphere 4.1 EMC VMAX40

SAN Layout Considerations • Dedicated FA ports for Production, UAT & Test environments • Dedicated FA ports for replication (mount hosts access) V-MAX • Shared/dedicated SRDF ports • ESX ports connectivity to SAN as redundant conf. • All FA/SRDF ports connected to high- speed SAN ports as redundant conf.41

Thin Provisioning in VMware vSphere Environments Decisions, Decisions, Decisions….. vSphere provides native thin provisioning VMAX provided array thin provisioning (Virtual Provisioning) • Either one can be used, no performance penalty in any of them • Both features can be used simultaneously but doing so increases risk • We debated where to implement it.. • Decided on the Array level…why? VMAX Virtual Provisioning simplifies drive and DA workload distribution • Provides additional benefits besides optimizing storage use • Ensure enough paths and TDEVs to support the workload VMAX Virtual Provisioning provides additional benefits • Zero Reclaim and Rebalancing42

Storage Layout Decisions Traditional Array Design Revolutionary Array (RAID 5 / 10 / SSD Design (Storage Tiering) Proven Fairly New Very costly Hugh Savings Not Flexible Highly Dynamic We decided to go with Storage Tiering but why? Saving money is an important thing but wasn’t the success criteria here.. The dynamic nature of the customer SharePoint environment was!43

EMC Symmetrix FAST VP – overview  Automatic storage tiering for Virtual Provisioning thin pools  Analysis and data movement at sub-LUN level: • Spreads data from a single thin device across multiple pools • Places very active parts of a LUN on high- EFD performing EFDs • Places less active parts of a LUN on higher- FC capacity, more cost-effective FC or SATA drives • Moves data at the extent group level - 7,680 KB  Moves data based on user-defined policies and SATA application performance needs  Data movement is automatic and nondisruptive 44 44

EMC FAST in Action EMC Storage with an active ESX Cluster VMware VMware VMware VMware All Fibre Channel Disk Drives Disk Resources are ~80% Busy45

EMC FAST in Action Add Flash Drives and Apply FAST Policy VMware VMware VMware VMware Tiered Storage 5% Flash Drives 65% FC Drives 30% SATA 68% Less Disk I/O Contention 2.5X Faster Disk Response Time46

Where did the customer use FAST VP Is VMAX FAST VP a good fit for SharePoint 2010? • Yes, but for maximum efficiency, it depends on which storage role • Search Index component ? No • Highly changing, throw-away data • Search Query component ? Yes • Highly-read data with small burst write changes • TempDB? Yes • The same blocks are re-used on disk and performance of TEMPDB directly affects SharePoint performance request - TempDB is used in every SharePoint request • Helps to handle unanticipated performance requirements47

Balancing the I/O OK, so the storage will balance itself but Path Fault with PowerPath/VE what about the Queues up until we reach vSphere server the storage subsystem layer.. The Customer evaluated PowerPath/VE PowerPath/VE We were able to achieve up to X 2.5 the performance compared with RR HBA HBA Why? FC, iSCSI, FCoE FC, iSCSI, FCoE Because it does a predictive load balancing.. Storage Storage Port Port48

Set it and Forget it It also helped the customer to find out about a flaky fiber cable using the VSI (Virtual Storage Integrator) Plugin VMware vCenter Server 49

This customer loves to take replicas on their SharePointenvironment But how do you take 40 Replicas? Yes, 40…50

Enabling the SharePoint Team Self-provision and refresh dev/test environments • Single console simplifies replica management • Wizards step through the process for replica management Replica 1 Replica 2 Replica 3 Replica 4 HIGHEST • User roles facilitate self-sufficiency Replication Manager Administrator • PRIVILEGES USER ROLE Power Database Administrator Integrate pre- and post-processes Database Administrator • Schedule jobs or run ad hoc Power User • Auto-expire replicas based Operator on retention policies LOWEST51

Selecting A 3rd Party BLOB Software…Your SharePoint SQLServer is NOT a FileServer!52

Selecting A 3rd Party BLOB Software…So let’s redirect theseFiles OUTSIDE of theSQL Database53

Selecting A 3rd Party BLOB Software… Together with the customer, we evaluated some 3rd party BLOB software and decided on Metalogix StoragePoint We reduced the content DB by 90%54

SharePoint 2010 DR: Virtualized Farm (Full Replication) SharePoint Farm A Primary Site Secondary Site • Resource/Protection Group VMs VMs level granularity • Active/Active (Sync distances) or Active/Passive (Async VMs distances) VMs • Failover automation: VMs VMs • VMware Site Recovery Manager (SRM) Protection VMs Groups for all server roles VMs Databases Databases BLOB Store SRDF/S BLOB Store 55 E

Automated Site Failback with VSI 456

To Cluster or Not to Cluster…. Microsoft Clustering VMware HA (MSCS) Node Resource Failover Full VM Failover Application Specific Application / OS agnostic Complex to set up VERY easy to set up Costly Cheap We decided to go with VMware HA but why? Ease of use was the key decision making..57

Key Takeaways SharePoint is more than just SQL… • Leverage EMC Proven solutions and Best Practices for SharePoint storage, networking and compute design • FAST, FAST Cache, VP improve efficiency & performance but require proper planning • Use RBS to improve scalability and TCO and in some cases, performance Full farm virtualization has great advantages over physical/hybrid configurations • Horizontal scaling is more efficient • The best FULL farm protection when Integrated with EMC replication • Simplifies, accelerates and automates SharePoint DR! (SRM) EMC’s SharePoint VSS based replication can significantly accelerate replication and recovery of SharePoint • A must for large deployments (TBs) • Protects all farm components, not just content (solution dependent) • Fast and simple Item level recovery while integrating with EMC partners (e.g. Kroll)58

Resources • Visit us on the Web to learn more about specific apps http://www.vmware.com/solutions/business-critical-apps/ • Best practices, reference architectures, and case studies • Microsoft Apps (Exchange, SQL, SharePoint) • Oracle • SAP • Check out the VMTN user communities • Email (Exchange, Lotus, BlackBerry) http://communities.vmware.com/community/vmtn/general/emailapps • EMC SharePoint solutions http://www.emc.com/solutions/application- environment/microsoft/solutions-for-microsoft-office-system- sharepoint.htm • Metalogix • http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStud yID=400001075960

Introducing: Business Production Bundle Maximize the benefits of virtualizing business critical applications NewAutomated Disaster Recovery vCenter Site Recovery Manager Business Production BundleDynamic Security vShield App with Data Security vCenter Site Recovery Manager vShield App with Data SecurityAutomated Operations vCenter Operations Advanced vCenter Operations Advanced vSphere vSphere vSphere 61

Questions62

BCA1931Design, Deploy, and OptimizeSharePoint 2010 on VMware

vSphere Performance Enhancements Feature Description VMware ESX®/VMware ESXi™ attempts to keep a virtual machine assigned to its home NUMA Support NUMA-node. Because memory for the virtual machine is allocated from the home node memory access is local and provides the best performance possible Transparent Page Page sharing allows the hypervisor to reclaim the redundant copies of memory created Sharing when multiple virtual machines run the same operating system and applications The balloon driver allows the hypervisor to reclaim host physical memory if memory Memory Ballooning resources are under contention. This is done with little to no impact to the performance of the application Pages elected to be swapped that can be compressed are stored in a compression cache Memory Compression in main memory. When required, pages are decompressed from compression cache versus paging out from disk Applications that can benefit from large pages on native systems, such as MS SQL, can Large Memory achieve similar performance improvement on a virtual machine backed with large Page Support memory pages Para-virtualized Network High-performance virtual I/O adapters that can provide greater throughput while requiring and Storage Controllers lower CPU utilization Distributed Resource As resource utilization fluctuates within a VMware vSphere® cluster, workloads are Scheduler (DRS migrated with no impact to performance or uptime using VMware vSphere® vMotion® and vMotion)65

Estimating User Activity (cont.)Enterprise Intranet Collaboration Environment Technical Case Study Example Workload Characteristics Value Average daily RPS 157 Average RPS at peak time 350 Total number of unique users per day 69,702 Average daily concurrent users 420 Peak concurrent users at peak time 1,433 Total number of requests per day 18,866,527SharePoint 2007 User Loads from Microsoft TechNet User Load Request Rate Requests Per Second Per User Light 20 requests per hour. An active user generates a .006 request every 180 seconds Typical 36 requests per hour. An active user generates a .010 request every 100 seconds Heavy 60 requests per hour. An active user generates a .017 request every 60 seconds Extreme 120 requests per hour. An active user generates .034 a request every 30 seconds66

Key Metrics to Monitor for ESX/ESXi Host /Resource Metric Description VM %USED Both CPU used over the collection interval (%)CPU %RDY VM CPU time spent in ready state %SYS Both Percentage of time spent in the ESX host VMkernel Memory ESX/ESXi host swaps in/out from/to disk (per Swapin, Swapout Both virtual machine, or cumulative over host)Memory Amount of memory reclaimed from resource pool by MCTLSZ (MB) Both way of ballooning READs per second, Both Reads and writes issued in the collection interval WRITEs per second DAVG/cmd Both Average latency (ms) of the device (LUN)Disk Average latency (ms) in the VMkernel, also known as KAVG/cmd Both “queuing time” Average latency (ms) in the guest. GAVG = DAVG + GAVG/cmd Both KAVG MbRX/s, MbTX/s Both Amount of data transmitted per second PKTRX/s, PKTTX/s Both Packets transmitted per secondNetwork %DRPRX, Both Drop packets per second %DRPTX 67

Key Metrics for SharePointResource Metric Description Processor usage over a period of time. Consistently high utilization can adversely affect performance. Remember to count "Total" inCPU % Processor Time multiprocessor systems. Maintain balanced performance between cores by also measuring individual core utilization Physical memory available for allocation. Insufficient memory leads to Available Mbytes excessive use of page file and increase in page faults Rate at which faults occur when a page is sought in the file system Cache Faults/sec cache and is not found. Effective use of cache for read and writeMemory operations can have a significant effect on performance Rate at which pages are read from or written to disk to resolve hard Pages/sec page faults. Increases in page faults indicate system-wide performance degradation High page file utilization can mean an increase in hard page faults, % UsedPaging File monitor this counter along with Pages/sec and Available Mbytes to % Used Peak determine if allocated memory is inadequate Disk Reads/sec Number of disk reads and writes per second Disk Writes/secDisk Avg. Disk sec/Read Average latency (seconds) of reads and writes of data from disk Avg. Disk sec/WriteNetwork Total Bytes/sec Rate at which data is sent and received through the network interface 68

VMware HA and Failover Clustering Supports two-node cluster Failover cluster nodes can be physical or virtual or any combination of the two Host attach (FC) or in-guest (iSCSI) Supports RDM only VMware HA + failover clustering • Seamless integration, virtual machines rejoin clustering session after VMware HA recovery • Can shorten time that database is in unprotected state Failover clustering now supported with VMware HA with vSphere v4.1 http://kb.vmware.com/selfservice/microsites/search.do?language=en _US&cmd=displayKC&externalId=103795969

http://www.vm4.ru	2741
http://itzikr.wordpress.com	1399
http://hghltd.yandex.net	6
http://webcache.googleusercontent.com	2
http://www.techgig.com	1
http://translate.googleusercontent.com	1
http://www.linkedin.com	1

Bca1931 final

by Itzik Reich

Accessibility

Categories

Upload Details

Usage Rights

7 Embeds 4,151

Statistics

Bca1931 final Presentation Transcript