Federal VMUG - March - Virtual machine sizing considerations in a numa environment v3


Published on

Federal VMUG - March 2011
"Virtual machine sizing considerations in a numa environment"
- Jason Shiplett, Netstar Systems International

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Federal VMUG - March - Virtual machine sizing considerations in a numa environment v3

  1. 1. Virtual Machine Sizing Considerations in NUMA Architectures<br />Jason Shiplett – NetStar Systems<br />Federal VMUG 3/23/11<br />
  2. 2. Non-Uniform Memory Access<br /> A computer memory design used in multiprocessors, where the memory access time depends on the memory location relative to a processor. Under NUMA, a processor can access its own local memory faster than non-local memory, that is, memory local to another processor or memory shared between processors.<br />
  3. 3. History and Modern Implementations<br />Commercially developed in the 1990s by companies such as Burroughs (later Unisys), Convex Computer (later Hewlett-Packard), and Silicon Graphics.<br />Modern implementations of cache-coherent NUMA (ccNUMA) include:<br />AMD Opteron multiprocessor systems<br />Intel Nehalem (x86) and Tukwila (IA-64)<br />With Nehalem, Intel introduced Quick-Path Interconnect (QPI), which is a very high bandwidth point-to-point interconnect used to connect CPU sockets, ergo NUMA nodes, relieving some of the hit when accessing remote memory<br />AMD uses a HyperTransport bus to connect CPU sockets, which competes with Intel’s QPI.<br />
  4. 4. How NUMA support in vSphere affects you!<br />NUMA ensures memory locality<br />Memory locality lowers latency to access of physical memory pages<br />Memory access within a NUMA node provides a higher bandwidth connection to memory via on-die memory controller<br />Lower latency + higher bandwidth = better performance<br />
  5. 5. NUMA Nodes<br />
  6. 6. Transparent Page Sharing and NUMA<br />Transparent Page Sharing (TPS) is restricted to within NUMA nodes<br />This means memory pages will only be shared within a single NUMA node.<br />Like pages in disparate NUMA nodes will be duplicated.<br />
  7. 7. Wide-VM NUMA Support in vSphere 4.1<br />A wide-VM is one which does not fit within a single NUMA node, e.g. 8 vCPU VM in a 4-socket quad-core server. <br />In vSphere 4.1, a wide-VM is split into smaller NUMA clients, which then occupy the fewest possible NUMA nodes. <br />Memory is interleaved between the NUMA nodes occupied by the individual NUMA clients. <br />Interleaving the memory in an equitable manner increases the percentage of memory locality over heuristic methods. <br />Implemented to improve performance of wide-VMs on large, e.g. quad socket, servers. Both CPU and memory performance do not differ much on two-node NUMA systems.<br />
  8. 8. What Wide-VM Support Means for You!<br />Few VMs will span multiple NUMA nodes<br />Very large, tier 1 applications<br />Only makes a real difference in large, e.g. 4+ socket, systems<br />In large systems, memory interleaving (equitable distribution of memory pages) can make a significant performance increase<br />The performance increase is more noticeable in memory-intensive applications.<br />
  9. 9. Wide-VM NUMA Support<br />
  10. 10. Sizing guidelines and considerations<br />As always, it depends on your workload, servers, desired consolidation, etc.<br />As a rule of thumb, keep vCPUs to a minimum, and if you can, keep it within a single NUMA node to ensure memory locality.<br />Wide-VM NUMA support in vSphere 4.1 gives greater flexibility when spanning NUMA nodes is necessary.<br />Memory intensive workloads benefit more from NUMA support<br />
  11. 11. Questions?<br />
  12. 12. References and links<br />1. "Non-Uniform Memory Access." Wikipedia, The Free Encyclopedia. Wikimedia Foundation, Inc. 22 Feb 2011. Web. 16 Mar 2011.<br />2. “Sizing VMS and NUMA Nodes.” Frank Denneman. 3 Feb 2010. Web. 16 Mar 2011<br />3. “VMware vSphere: The CPU Scheduler in ESX 4.1.” VMware. 2010. Web. 16 Mar 2011<br />4. “ESX 4.1 NUMA Scheduling.” Frank Denneman. 13 Sep 2010. Web. 16 Mar 2011<br />http://en.wikipedia.org/wiki/Non-Uniform_Memory_Access<br />http://www.vmware.com/files/pdf/techpaper/VMW_vSphere41_cpu_schedule_ESX.pdf<br />http://frankdenneman.nl<br />
  13. 13. Thanks!<br />My blog – http://blog.shiplett.org<br />Follow me on Twitter - @jshiplett<br />