1. NUMA and Scale Up
Zhongke Chen
July 2012

2. SMP VS NUMA (NON-UNIFORM MEMORY
ARCHITECTURE)
SMP NUMA (!=MPP)
Technology Education Confidential and Proprietary 2

3. HOW LINUX HANDLES NUMA
Linux detects NUMA and deals it differently from SMP
Default Policy: One process is fixed to one node.
Confidential and Proprietary 3

4. SCALE UP PROBLEM
What happens if one process uses large memory?
(larger than the memory in one node)
Confidential and Proprietary 4

5. WHAT HAPPENS
Only memory on one node is used.
The system starts to swap heavily.
Confidential and Proprietary 5

6. WORKAROUNDS
numactl --interleave=all
(inefficient to access remote memory)
Confidential and Proprietary 6

7. TRUE SOLUTIONS
1. Multiple Smaller Processes
2. Control Policy via libnuma API
3. Another workaround: Virtualization
Confidential and Proprietary 7

8. REFERENCE
• http://en.wikipedia.org/wiki/Non-Uniform_Memory_Access
• http://developer.amd.com/assets/LibNUMA-WP-fv1.pdf
• http://www.kernel.org/doc/Documentation/vm/numa_memor
y_policy.txt
• http://linux.die.net/man/8/numactl
• http://linux.die.net/man/3/numa
Confidential and Proprietary 8