Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hyper-V: Best Practices

9,508 views

Published on

Slides from my session on KulenDayz 2016, about best practices in Hyper-V deplyoments.

Published in: Technology

Hyper-V: Best Practices

  1. 1. Tomica Kaniški tomica@kaniski.eu | http://blog.kaniski.eu/ HYPER-V: BEST PRACTICES
  2. 2. OUR SPONSORS MADE THIS POSSIBLE! THANK YOU!
  3. 3. TAKE A DEEP BREATH… "Design/planning phase" is critical!
  4. 4. HOW MANY… • … hosts? • … CPUs? • … RAM? • … bandwidth? • … ? • Microsoft Assessment and Planning Toolkit (MAP Toolkit) – https://www.microsoft.com/en- us/download/details.aspx?id=7826 – DEMO
  5. 5. HOST (HARDWARE) • use standardized hardware (easier to manage) • use Windows Server certified hardware (https://www.windowsservercatalog.com/) • CPU – server virtualization (Intel VT/AMD-V) and Data Execution Prevention (XD/NX) - Enabled – SLAT-capable with large cache – don’t mix vendors (mixed CPU vendors = NO Live Migration) – Hyper-Threading – enabled – „compatible” • performance impact if not using same instruction set – „best buy” (12-cores per socket?) • RAM – ECC-capable – the more, the better (how many VMs can go down in case of disaster? 30? ) • HDD (local) – RAID-1 (of two SSDs or HDDs)
  6. 6. POWER OPTIONS • full power or green IT?  • power settings: – BIOS/UEFI level – host OS level – guest OS level • easy to set: – POWERCFG.EXE /S SCHEME_MIN (High performance) – POWERCFG.EXE /S SCHEME_BALANCED (Balanced (recommended))
  7. 7. HOST (SOFTWARE) (1) • Hyper-V host is a Hyper-V host (and nothing else)! • install the latest drivers & firmware versions (that work) – use driver/firmware bundles (like SPP for HP servers) to standardize BIOS, NIC, storage controllers and HBA versions in your environment • http://h17007.www1.hpe.com/us/en/enterprise/servers/products/service_pack/spp/index.aspx • install only the necessary (Hyper-V role, Failover Clustering, MPIO + DSM) – use MinShell or Core (or even Nano ) installation options – or full GUI, configure and then „downgrade” to Core – or just full GUI…  • updates & hotfixes – Cluster – https://support.microsoft.com/en-us/kb/2920151 – Hyper-V – https://support.microsoft.com/en-us/kb/3135020 – support will first ask if you’ve installed all the latest and greatest… 
  8. 8. HOST (SOFTWARE) (2) • install backup and monitoring agents • install antivirus if necessary (company policy) – don’t forget to set exclusions! • don’t forget to modify Hyper-V settings after the installation – default paths are on the system drive! • Set-VMHost -ComputerName localhost -VirtualHardDiskPath 'D:VMs' • Set-VMHost -ComputerName localhost -VirtualMachinePath 'D:VMs' • sign out from Hyper-V hosts when finished working – we are admins… we love PowerShell… we don’t need RDP sessions to hosts  • remember to close remote PS sessions as well 
  9. 9. ANTIVIRUS EXCLUSIONS • antivirus exclusions for Hyper-V hosts & clusters are: – all directories that contain virtual machines, their disks, snapshots, … – processes: • %systemroot%System32vmms.exe • %systemroot%System32vmwp.exe • %systemroot%Clusterclussvc.exe • %systemroot%Clusterrhs.exe – folders: • %programfiles%MicrosoftHyper-V* • %public%DocumentsHyper-V* • %systemdrive%ClusterStorage* • %systemdrive%ProgramDataMicrosoftWindowsHyper-V* • %systemroot%Cluster* • (witness disk) – https://support.microsoft.com/en-us/kb/3105657 & https://support.microsoft.com/en-us/kb/961804
  10. 10. INSTALLATION • manual or automatic installation • speed-up the deployment – unattend.xml (one-time prepare, many-time use ) – DEMO – bare-metal host deployment (SCVMM) – MDT/SCCM deployments • which version? – Standard, Datacenter (AVMA) • which option? – full GUI, MinShell, Core, Nano? – full GUI then „downgrade” to Core? • Remove-WindowsFeature Server-Gui-Shell, Server-Gui-Mgmt-Infra
  11. 11. MANAGEMENT TOOLS • although „real (wo)men” use PowerShell for everything, there are some other tools that can help you: () – Server Manager console • Best Practices Analyzer (BPA) – useful tool! – Hyper-V console • don’t use this one for managing clustered VMs… please  – Failover Cluster console • Cluster Validation Wizard – VERY useful tool! (ask Microsoft Support! ) – System Center VMM • it works… just fine… but not every time… () – 3rd party solutions (5nine, SysInternals, …)
  12. 12. VIRTUAL MACHINES (1) • use/convert to Generation 2 VMs where possible – https://blogs.technet.microsoft.com/jhoward/2013/11/1 4/hyper-v-generation-2-virtual-machines-part-10/ – Convert-VMGeneration tool • https://code.msdn.microsoft.com/ConvertVMGeneration • use only supported guest OSes/versions – https://technet.microsoft.com/en- us/library/dn792027(v=ws.11).aspx • update Integration Components regularly – some OS hotfixes bring new versions of ICs as well!
  13. 13. VIRTUAL MACHINES (2) • don’t use screen savers inside VMs and sign out of VMs • templates – use the „smarter” sysprep: – %systemroot%SysprepSysprep.exe /OOBE /Generalize /Shutdown /Mode:VM – faster sysprep, a lot less hardware recognizing – don’t use this templates for multiple virtualization platforms – Convert-WindowsImage • https://gallery.technet.microsoft.com/scriptcenter/Conv ert-WindowsImageps1-0fe23a8f – offline patching – Apply-WindowsUpdate • https://gallery.technet.microsoft.com/Offline-Servicing- of-VHDs-df776bda
  14. 14. VIRTUAL MACHINES (3) • devices: – don’t use Legacy Network Adapter if it’s not absolutely necessary – the same goes for virtual HBAs – remove devices you’re not using • Automatic Stop Action – do not leave „Save” – change to either Shut Down or Turn Off • disable unnecessary background „things”: – SuperFetch – Windows Search – Scheduled Tasks (disk defragmentation, …) – Aero (optimize for performance)
  15. 15. ACTIVE DIRECTORY/GPO • should I join Hyper-V hosts to Active Directory domain? – YES (existing or the separate „management” domain – little added security, but huge management overhead; in highly-secured environments) – Hyper-V will work if domain is not available (but Live Migration won’t!) • should I disable Windows Firewall? – NO (fine-tune it rather!) – Group Policy is a great tool, and your host is domain-joined already, so… • GPO – disable RDP Printer Mapping – Computer Confguration | Policies | Administrative Templates | Windows Components | Remote Desktop Services | Remote Desktop Session Host | Printer Redirection | Do not allow client printer redirection | Enable
  16. 16. VIRTUAL MEMORY/PAGING FILE (PAGEFILE.SYS) • size – leave it on (automatic) at host level • move it to separate disk – for VM – SCSI disk – exclude this disk from Hyper-V Replica • you can use the following: $computer = Get-WmiObject Win32_ComputerSystem -EnableAllPrivileges $computer.AutomaticManagedPagefile = $false $computer.Put() $CurrentPageFile = Get-WmiObject -Query "select * from Win32_PageFileSetting where name='C:pagefile.sys'" $CurrentPageFile.delete() Set-WMIInstance -Class Win32_PageFileSetting -Arguments @{name="D:pagefile.sys";InitialSize = 0; MaximumSize = 0}
  17. 17. CLUSTER • Hyper-V cluster is a Hyper-V cluster (and nothing else)! • start with at least 3 nodes • separate network communication – Corporate, Storage, LiveMigration, vSwitch, Backup • use teaming where it seems appropriate • rename cluster resources (networks, disks) and set network „roles” & priority in cluster • Cluster Aware Updating – use it if it makes sense and don’t use it with SCCM – http://www.altaro.com/hyper-v/cluster-aware-updating- hyper-v-basics/
  18. 18. NETWORKING (1) • (re)name your NICs (and NIC teams and vSwitches) – „Local Area Connection 23” is a great name, but…  • use teaming (it’s free and it works!) – should you use Active/Actives? • it depends (on underlying network… switches and connections among them – possible link saturation) • using 1 NIC in Stand-by is just dumb… or not?  – Dynamic and Switch Independent is the way to go! • if you have issues with a load balancer, go with Hyper-V port • which architecture should you use? (converged or not?) – http://www.hyper-v.nu/archives/hvredevoort/2014/02/definitive- guide-to-hyper-v-r2-network-architectures/
  19. 19. NETWORKING (2) • no need for a dedicated heartbeat network in cluster – all cluster networks communicate „heartbeat” signals • don’t disable VMQ… if you don’t have problems with it (Broadcom?) – http://www.dell.com/support/article/us/en/04/SLN132131 • create virtual switches – … and name them the same (use the convention that works for you, but same on all hosts)!  – how many vSwitches do I need? • smallest number that makes sense (use VLANs!) • (how many NICs? teamed or not? )
  20. 20. NETWORKING (3) • networking optimizations: – set the right networking order – Corporate network first (ncpa.cpl – Advanced – Advanced Settings – Adapters and Bindings) – disable unnecessary network services • Live Migration – leave only Client for Microsoft Networks, File and Printer Sharing, TCP/IPv4 and TCP/IPv6 (NetBIOS is disabled) • iSCSI – leave only TCP/IPv4 and TCP/IPv6 (NetBIOS is disabled) – choose which networks will register in DNS (Corporate only) – set cluster network priorities and roles – choose Live Migration network(s) – iSCSI network – use MPIO, not teaming… on separate NICs
  21. 21. NETWORKING (4) • DHCP guard (filtering unsolicited DHCP Server offers) – enable for all VMs, and disable for the ones that need it – small performance impact • Router Guard (filtering ICMP Router Advertisements and redirect messages) – not used so much • implement IPAM for addressing – instead of those old, boring Excel files  • do not share adapter with physical OS • enable Jumbo Frames for CSV, iSCSI and LM networks
  22. 22. STORAGE (1) • single LUN per VM? – no need… CSVs work just fine • SAN or SOFS/SMB3? – if there is SAN in your environment (and it’s supported for Hyper-V), use it – if you want a „cheaper” solution (and there is no reason against it), try using SOFS/SMB3 – or a combo (SOFS for SMB3 on SAN LUNs)? • use 4K native disks and 64K allocation unit size for drives hosting VHD(X) • avoid vIDE for data disks (Generation 1 VMs) • use virtual HBAs only if needed • use Storage QoS if you need it
  23. 23. STORAGE (2) • should I use the „witness” disk in cluster? Yes. • install MPIO, latest DSMs (usually requires restart) and drivers • rename the CSV „mount point” (C:ClusterStorageVolume1)” • put VHD(X)s on CSV into folders! – access-related errors if there are VHD(X)s in a CSV root! • NTFS or ReFS? – NTFS – 64k for VHD(X)s for optimal performance – ReFS – with Windows Server 2016 • measure IOPS – http://blog.workinghardinit.work/2014/01/08/how-to-measure-iops- of-a-virtual-machine-with-resource-metering-and-measurevm/
  24. 24. VIRTUAL DISKS • use/convert to VHDX • fixed/dynamic/differencing/pass-through? – VHDX – dynamic – VHD – fixed-size • don’t forget to ensure enough free space for dynamic disks! – the same goes to using thin-provisioned LUNs! • one-liner: – Convert-VHD -Path D:VM01.vhd -DestinationPath D:VM01.vhdx -VHDType Dynamic • don’t use pass-through disks (no point anymore – dynamic disks are as performant as it gets, and their mobility/manageability is not questionable • convert pass-through disk to VHDX – New-VHD -Path "D:VMSConverted.vhdx" -Dynamic –SourceDisk 5
  25. 25. CLUSTER SHARED VOLUMES (1) • how many and how big? – minimum of 2 (to utilize two different storage controllers, …) – < 8 cluster nodes = 1 CSV per 1 node – > 8 cluster nodes = 1 CSV per 2-4 nodes • how many VMs per CSV? – up to 50 (server VMs) – up to 100 (client VMs in VDI environment) • IOPS are all that matters! • enable CSV cache to improve read operations – (Get-Cluster).BlockCacheSize = 2048 • metadata changes can only occur on the CSV coordinator (owner node) • please… rename CSVs before you use them! – C:ClusterStorageVolume1 and C:ClusterStorageVolume2 are great names, but…
  26. 26. CLUSTER SHARED VOLUMES (2) • set the automatic stop action to anything other than „Save” • fill up your CSVs with a maximum of 75 percent of its capacity to allow growth of all these files • how much space is available on your CSVs? – http://www.powershellmagazine.com/2014/02/28/reporting-cluster- shared-volume-csv-disk-space-utilization/ • encrypt CSV with BitLocker – performance impact of 20-30% • network adapters used for CSVs should have the Client for Microsoft Networks and File and Printer Sharing for Microsoft Networks enabled – activate Microsoft Failover Cluster Virtual Adapter Performance Filter (not if you are using Guest Cluster in your virtual machine – then it should be disabled on the host level)
  27. 27. LIVE MIGRATION (1) • don't forget to set the live migration network and options • use Kerberos constrained delegation instead of CredSSP (need to log on to the server to start live migration) • enabling KCD for two node cluster is quite easy (): – HYP1 delegates credentials to HYP2 • $hvhost = "HYP2" • $domain = "sun.tklabs.eu" • Get-ADComputer HYP1 | Set-ADObject -Add @{"msDS- AllowedToDelegateTo"="Microsoft Virtual System Migration Service/$hvhost.$domain", "cifs/$hvhost.$domain","Microsoft Virtual System Migration Service/$hvhost", "cifs/$hvhost"}
  28. 28. LIVE MIGRATION (2) – HYP2 delegates credentials to HYP1 • $hvhost = "HYP1" • $domain = "sun.tklabs.eu" • Get-ADComputer HYP2 | Set-ADObject -Add @{"msDS- AllowedToDelegateTo"="Microsoft Virtual System Migration Service/$hvhost.$domain", "cifs/$hvhost.$domain","Microsoft Virtual System Migration Service/$hvhost", "cifs/$hvhost"} • enable live migration – Enable-VMMigration -Computername HYP1, HYP2 – Set-VMHost -Computername HYP1, HYP2 - VirtualMachineMigrationAuthenticationType Kerberos • yes, it can be done through GUI as well! 
  29. 29. GUEST CLUSTERING • use Anti-Affinity rules to make sure nodes don’t end up on the same physical host – either VMM or PowerShell • not available through Failover Clustering console! • (Get-ClusterGroup KULEN1).AntiAffinityClassNames = "GuestClusterKULEN" • (Get-ClusterGroup KULEN2).AntiAffinityClassNames = "GuestClusterKULEN" • Get-ClusterGroup KULEN1 | fl Anti* • change default failover-triggering heartbeat times (to allow live migrations on host, without failover actions in guest – default is 10 seconds) – (Get-Cluster).CrossSubnetThreshold = 25 – (Get-Cluster).SameSubnetThreshold = 25
  30. 30. CHECKPOINTS • Windows Server 2012 R2 – don’t use checkpoints (snapshots) in production • Windows Server 2016 – use Production checkpoints  • tips: – use them as rarely as possible – checkpoints are no substitution for backups – delete checkpoints as soon as possible – don’t delete a checkpoint file on the file level! • console/PowerShell is the only way to go – use them with caution on domain controllers and database servers
  31. 31. BACKUP AND DISASTER RECOVERY • backup – poor man’s backup solution – Windows Server Backup – other solutions like SC DPM, Veeam, Altaro, … • no real need to backup Hyper-V hosts – except maybe for faster recovery, but… • use Hyper-V Replica if you need disaster recovery – it’s included anyway – exclude Paging file disks from replication
  32. 32. PERFORMANCE (PERFMON.EXE) (1) • storage – it’s all about the IOPS! – counters: • Logical Disk(*)Avg. Disk sec/Read • Logical Disk(*)Avg. Disk sec/Write – thresholds: • up to 15 ms should be OK • 15-25 ms may cause negative impact on workloads • > 25 ms will cause negative impact on workloads
  33. 33. PERFORMANCE (PERFMON.EXE) (2) • memory – there has to be enough memory available inside a VM (or a Hyper-V host)! – counters: • MemoryAvailable Mbytes – RAM available for running the active processes – 15% (or more) is OK • MemoryPages/sec – how often the disk is accessed to resolve hard page faults – < 500 pages per second is OK – > 500 pages per second – machine just needs more RAM (don’t be cheap!) 
  34. 34. PERFORMANCE (PERFMON.EXE) (3) • network – counters • Network Interface(*)Bytes Total/sec – shows current network utilization – 20% (or more) free is OK • Network Interface(*)Output Queue Length – shows latency in sending network packets (threads that wait on the NIC) – 0 is OK – 1 or more is a sign of degraded network performance • Hyper-V Virtual Network Adapter(*)Bytes/sec – shows which virtual network adapters are consuming the most bandwidth
  35. 35. PERFORMANCE (PERFMON.EXE) (4) • processor – counters • Processor(*)% Processor Time – overall CPU utilization (at host level) – < 80% is always OK () • Hyper-V Hypervisor Logical Processor(_Total)% Total Run Time – at host level, but to evaluate guest utilization • Hyper-V Hypervisor Root Virtual Processor% Total Run Time – CPU of the root partition • Hyper-V Hypervisor Virtual Processor(_Total)% Total Run Time – total time of the virtual processors • Hyper-V Hypervisor Virtual Processor(*)%Guest Run Time – CPU of the guest partitions – always measure the CPU usage of the physical system by using the Hyper-V Hypervisor Processor performance counters
  36. 36. PAL IS YOUR NEW (BEST) PAL! • PAL (a.k.a. Performance Analysis of Logs Tool) – reads a performance monitor counter logs and analyzes them using known thresholds – for Hyper-V, SQL, IIS, Exchange, SharePoint, … – you can get it at https://pal.codeplex.com/ – free! – (P.S. use en-US regional formatting – (P.P.S. Set-Culture en-US ) – DEMO
  37. 37. A FEW MORE THINGS... • vCPU allocation (Microsoft recommendation) – no more than 8 virtual CPUs per physical CPU core for server workloads – no more than 12 virtual CPUs per physical CPU core for VDI workloads • be careful with thin-provisioned disks (running out of storage space), dynamic memory (don’t use it for SQL or Exchange) • Generation 1 VMs – create bigger VHD(X) and smaller partition – you don’t need to shutdown VM to resize  • don’t forget common tools like defrag and chkdsk • set up a naming convention (and stick to it ) • cluster log is your friend… when things go bad (increase sensitivity logging level) – (Get-Cluster).ClusterLogLevel = 5 – Get-ClusterLog -Destination C:Logs -UseLocalTime • DOCUMENT EVERYTHING.
  38. 38. RESOURCES • checklists: – https://blogs.technet.microsoft.com/askpfeplat/2013/11/03/windows-server-2012-r2-hyper- v-best-practices-in-easy-checklist-form/ – https://blogs.technet.microsoft.com/askpfeplat/2013/03/10/windows-server-2012-hyper-v- best-practices-in-easy-checklist-form/ • book: – https://www.packtpub.com/virtualization-and-cloud/hyper-v-best-practices • other: – http://www.showit.sk/ShowIT/media/ShowIT/prezentacie/2015/marek_jan_Microsoft-Hyper- V-performance-tuning.pdf – http://www.altaro.com/hyper-v/23-best-practices-improve-hyper-v-vm-performance/ – http://www.altaro.com/hyper-v/19-best-practices-hyper-v-cluster/ – http://www.altaro.com/hyper-v/common-hyper-v-deployment-mistakes/ – http://www.altaro.com/hyper-v/hyper-v-virtual-cpus-explained/
  39. 39. THANK YOU!

×