PlovDev 2016: Application Performance in Virtualized Environments by Todor Tsankov
•Download as PPTX, PDF•
0 likes•383 views
Application Performance in Virtualized Environments
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (7)
Similar to PlovDev 2016: Application Performance in Virtualized Environments by Todor Tsankov
Similar to PlovDev 2016: Application Performance in Virtualized Environments by Todor Tsankov (20)
More from PlovDev Conference
More from PlovDev Conference (7)
Recently uploaded
Recently uploaded (20)
PlovDev 2016: Application Performance in Virtualized Environments by Todor Tsankov
- 1. Application Performance in Virtualized Environments Todor Tsankov, Cloud Service Engineer
- 2. Agenda 1 CPU Resources 2 Memory Resources 3 Storage Resources 4 Network Resources 2
- 4. CPU - Features • Fair proportional scheduling: – Shares – Limit – Reservation • vSMP / Co-scheduling • Hypethreading • Intel VT-x / AMD-V • NUMA / prefer-HT • vNUMA 4
- 5. CPU - NUMA 5
- 7. CPU – NUMA and Hyperthreading 7
- 8. CPU - Monitoring • Ready (%RDY) – % time a vCPU was ready to be scheduled on a physical processor but couldn’t due to processor contention – Investigation Threshold: 10% per vCPU • Co-Stop (%CSTP) – % time a vCPU in an SMP virtual machines is “stopped” from executing, so that another vCPU in the same virtual machine could run to “catch-up” and make sure the skew between the two virtual processors doesn’t grow too large – Investigation Threshold: 3% 8
- 9. CPU – Best Practices • Do not over-allocate vCPUs • Create single vCPU VMs whenever possible • Enable Hyperthreading • Right Size the VM – vCPU count should be less or equal to the number of cores in a single physical CPU (single NUMA node) 9
- 11. Memory - Features • Allow memory over-commitment • Fair proportional memory scheduling – Scheduling parameters: • Shares • Limit • Reservation • Support for large pages – Performance increased by 10 to 30% • Intel EPT / AMD RVI 11
- 12. Memory - Mapping • Three types of memory address spaces – Virtual memory – Physical memory – Machine memory 12
- 13. Memory - Mapping • Hardware accelerated virtualization (Intel EPT / AMD RVI) – Handle shadow mapping in the hardware – Tagged Translation Look-aside Buffers (TLB) 13
- 14. Memory - Reclamation • Transparent page sharing – most efficient • Memory Ballooning – always install latest version of VMware tools • Memory Compression – may sound strange, but this is much faster than swapping • Virtual Machines Swap – Hypervisor swap, not to be confused with OS swap file/partition 14
- 15. Memory - Reclamation - Transparent Page Sharing (TPS) Background process for removing duplicate memory pages 15
- 16. Memory – Reclamation - Ballooning “Pushes” memory pressure from ESX host into VM 16
- 17. Memory - Reclamation - Compression Essentially “zips” memory instead of swapping it so that it uses less space in RAM 17
- 18. Memory - Reclamation - Swapping Writes VM memory from physical RAM out to disk 18
- 19. Memory - Monitoring • Balloon driver size (MCTLSZ) – The total amount of guest physical memory reclaimed by the balloon driver – Investigation Threshold: 1 • Swapping (SWCUR) – The current amount of guest physical memory that is swapped out to the ESX kernel VM swap file – Investigation Threshold: 1 • Swap Reads/sec (SWR/s) – The rate at which machine memory is swapped in from disk – Investigation Threshold: 1 • Swap Writes/sec (SWW/s) – the rate at which machine memory is swapped out to disk – Investigation Threshold: 1 19
- 20. Memory – Best Practices • Do not overcommit memory • Configure swap in your Guest Operating System – Size it to be at least equal to the configured vRAM for the VM – Put the swapping partition or swap file (for Windows) in separate virtual disk • Install VMware tools – This enables the ballooning driver and enables the VMkernel to use the best memory reclamation technique • Enable Intel EPT / AMD RVI in the ESX host BIOS • Use large memory pages in guest OS – Minimizes the TLB misses 20
- 23. Storage – Monitoring • Kernel Latency Average (KAVG) – This counter tracks the latencies of IO passing thru the Kernel – Investigation Threshold: 1ms • Device Latency Average (DAVG) – This is the latency seen at the device driver level. It includes the roundtrip thime between the HBA and the storage – Investigation Threshold: 15-20ms, lower is better, some spikes are okay • Abort (ABRT/s) – The number of commands aborted per second – Investigation Threshold: 1 23
- 24. Storage – Best Practices • Separate VM disk on different physical disks if needed • Do not oversize VM disks – VM disk can be expanded, but it is difficult to shrink • Preprovision VM disks – Don’t use thin provisioned disk for mission critical applications • Install VMware Tools – Installs optimized, specific OS drivers for the SCSI controllers • Align guest OS disks – Most modern OS does this automatically 24
- 27. Network - Monitoring • Transmit Dropped Packets (%DRPTX) – The percentage of transmit packets dropped – Investigation Threshold: 1ms • Receive Dropped Packets (%DRPRX) – The percentage of received packets dropped – Investigation Threshold: 1ms 27
- 28. Network - Best Practices • Load balance on vSwitch level, not inside VM – Allow the Hypervisor to do the network teaming • Install VMware Tools – Installs optimized, specific OS drivers for the NIC adapters • Use VMXNET3 vNIC adapters when possible – Support for most modern OS 28
- 29. Thank You Questions & Answers
Editor's Notes
- Shares are configuring the relative priority of this VM with regards to other VMs in the same resource pool. Remember shares is only handled when we have contention! Limit: additionally limits the VM machine CPU time regardless of its shares and current condition of the host. In other words, it is possible that a VM will not run even when the host idle and there are plenty of resource, just because of the VM’s limit set. Reservation: On cpu the reservation is a guarantee for clock cycles. You define the reservation in MHZ. If you give a virtual machine a reservation it means the VMkernel CPU scheduler will give it at least that amount of resources. If a virtual machine is not using its resources the CPU cycles are not wasted on the physical host. Other machines can use it. What you do with CPU reservations is making sure that a VM will always get access to physical CPU in a committed environment. Hierarchy: the total amount of shares for all worlds running in current resource pool are the shares of this resource pool and every world receives CPU time proportional to its shares. The same principle applies recursively for the parent pool of the current resource pool.
- Shares are configuring the relative priority of this VM with regards to other VMs in the same resource pool. Remember shares is only handled when we have contention! Limit: additionally limits the VM machine CPU time regardless of its shares and current condition of the host. In other words, it is possible that a VM will not run even when the host idle and there are plenty of resource, just because of the VM’s limit set. Reservation: On cpu the reservation is a guarantee for clock cycles. You define the reservation in MHZ. If you give a virtual machine a reservation it means the VMkernel CPU scheduler will give it at least that amount of resources. If a virtual machine is not using its resources the CPU cycles are not wasted on the physical host. Other machines can use it. What you do with CPU reservations is making sure that a VM will always get access to physical CPU in a committed environment. Hierarchy: the total amount of shares for all worlds running in current resource pool are the shares of this resource pool and every world receives CPU time proportional to its shares. The same principle applies recursively for the parent pool of the current resource pool.
- Shares are configuring the relative priority of this VM with regards to other VMs in the same resource pool. Remember shares is only handled when we have contention! Limit: additionally limits the VM machine CPU time regardless of its shares and current condition of the host. In other words, it is possible that a VM will not run even when the host idle and there are plenty of resource, just because of the VM’s limit set. Reservation: On cpu the reservation is a guarantee for clock cycles. You define the reservation in MHZ. If you give a virtual machine a reservation it means the VMkernel CPU scheduler will give it at least that amount of resources. If a virtual machine is not using its resources the CPU cycles are not wasted on the physical host. Other machines can use it. What you do with CPU reservations is making sure that a VM will always get access to physical CPU in a committed environment. Hierarchy: the total amount of shares for all worlds running in current resource pool are the shares of this resource pool and every world receives CPU time proportional to its shares. The same principle applies recursively for the parent pool of the current resource pool.
- Shares are configuring the relative priority of this VM with regards to other VMs in the same resource pool. Remember shares is only handled when we have contention! Limit: additionally limits the VM machine CPU time regardless of its shares and current condition of the host. In other words, it is possible that a VM will not run even when the host idle and there are plenty of resource, just because of the VM’s limit set. Reservation: On cpu the reservation is a guarantee for clock cycles. You define the reservation in MHZ. If you give a virtual machine a reservation it means the VMkernel CPU scheduler will give it at least that amount of resources. If a virtual machine is not using its resources the CPU cycles are not wasted on the physical host. Other machines can use it. What you do with CPU reservations is making sure that a VM will always get access to physical CPU in a committed environment. Hierarchy: the total amount of shares for all worlds running in current resource pool are the shares of this resource pool and every world receives CPU time proportional to its shares. The same principle applies recursively for the parent pool of the current resource pool.
- Demand = how much memory the VM wants to actively use Entitlement = how much physical RAM a VM can get VM will achieve best performance when entitlement >= demand The exact entitlement is based on each VM’s relative priority with other VMs – based on shares, idle memory and other VMs running on the host.
- Second level of page mapping handled by the VMM and VMkernel. Traditionally this was handled in the software, but now the Hardware accelerated virtualization (VT-x for Intel and AMD-V) provides handling of shadow pages mapping on hardware level.
- Second level of page mapping handled by the VMM and VMkernel. Traditionally this was handled in the software, but now the Hardware accelerated virtualization (VT-x for Intel and AMD-V) provides handling of shadow pages mapping on hardware level.
- The Guest Operating System sees the Virtual Memory and “thinks” can use everything. This way we are not able to achieve very good memory consolidation ratio. The swap file size is defined as (swap file = configured memory – memory reservation)
- Not configurable – always running. Very effective, especially for zero pages. Completely transparent for the Guest Operating System. Very good results in the case of Vmware View application.
- Uses Vmware tools vmctl Always install Vmware tools if possible.
- Background process, looking for inactive memory pages. The page is compressed only if a 1:2 (50%) compression ratio can be achieved. The idea is to put at least two pages in one physical page. These pages (containing compressed memory) are never swapped, of course.
- Nothing too much to say here – this is traditional virtual memory handling for almost every operating system out there. More interesting is that Vmkernel memory manager supports swapping in multiple files. Every VM have at least two swap files: vRAM and VMX. The first one is sized to contain the entire vRAM, configured to the VM (nothing more, nothing less). This way it supports suspending the machine. The second one is for the VMX process, managing this VM. Part of the VMX process is never swapped. The locations of both files can be configured. In addition, Vmkernel supports one more, big and common for all user processes running on the host swap file (location also configurable). The idea is that there are a lot of UW running (management agents, daemons, etc.), but no always required in the memory. By swapping them, the Vmkernel can achieve even bigger VM consolidation ratio.
- Demand = how much memory the VM wants to actively use Entitlement = how much physical RAM a VM can get VM will achieve best performance when entitlement >= demand The exact entitlement is based on each VM’s relative priority with other VMs – based on shares, idle memory and other VMs running on the host.
- Demand = how much memory the VM wants to actively use Entitlement = how much physical RAM a VM can get VM will achieve best performance when entitlement >= demand The exact entitlement is based on each VM’s relative priority with other VMs – based on shares, idle memory and other VMs running on the host.
- Demand = how much memory the VM wants to actively use Entitlement = how much physical RAM a VM can get VM will achieve best performance when entitlement >= demand The exact entitlement is based on each VM’s relative priority with other VMs – based on shares, idle memory and other VMs running on the host.
- Demand = how much memory the VM wants to actively use Entitlement = how much physical RAM a VM can get VM will achieve best performance when entitlement >= demand The exact entitlement is based on each VM’s relative priority with other VMs – based on shares, idle memory and other VMs running on the host.
- With this I would like to finish the presentation and leave the rest of the time for question and aswers.