vNUMA in Xen 
Wei Liu & Elena U
mtseva 
Chicago { August 18, 2014
Agenda Motives History and status Design Problems Benchmark Future Work 
Agenda 
I Motives 
I History and status 
I Design...
Agenda Motives History and status Design Problems Benchmark Future Work 
Motives 
I Cross NUMA node memory access is expen...
Agenda Motives History and status Design Problems Benchmark Future Work 
History and status 
I PV vNUMA presented in Xen S...
mtseva has been working on upstreamable PV 
vNUMA since 2013 
Chicago { August 18, 2014 vNUMA in Xen 4 / 19
Agenda Motives History and status Design Problems Benchmark Future Work 
Design: PV and PVH 
I Toolstack puts enlightenmen...
Agenda Motives History and status Design Problems Benchmark Future Work 
Design: PV and PVH 
Enlightenment information str...
Agenda Motives History and status Design Problems Benchmark Future Work 
Design: HVM 
I Toolstack puts enlightenment infor...
Agenda Motives History and status Design Problems Benchmark Future Work 
Problems: vNUMA and other features 
PV PVH HVM 
B...
Agenda Motives History and status Design Problems Benchmark Future Work 
Problems: CPU topology 
Chicago { August 18, 2014...
Agenda Motives History and status Design Problems Benchmark Future Work 
Benchmark 
I Host 
I 2 sockets, 12 PCPUs, HT disa...
Agenda Motives History and status Design Problems Benchmark Future Work 
Benchmark: Autonuma 
Chicago { August 18, 2014 vN...
Agenda Motives History and status Design Problems Benchmark Future Work 
Benchmark: Autonuma 
Chicago { August 18, 2014 vN...
Agenda Motives History and status Design Problems Benchmark Future Work 
Benchmark: Autonuma 
Chicago { August 18, 2014 vN...
Agenda Motives History and status Design Problems Benchmark Future Work 
Benchmark: SPECJBB 
Chicago { August 18, 2014 vNU...
Agenda Motives History and status Design Problems Benchmark Future Work 
Benchmark: SPECJBB 
Chicago { August 18, 2014 vNU...
Agenda Motives History and status Design Problems Benchmark Future Work 
Benchmark: STREAM 
Chicago { August 18, 2014 vNUM...
Agenda Motives History and status Design Problems Benchmark Future Work 
Benchmark: Conclusion 
I vNUMA improves performan...
Upcoming SlideShare
Loading in...5
×

XPDS14 - vNUMA in Xen - Wei Liu, Citrix

824

Published on

vNUMA (virtual NUMA) is a memory optimisation technology that makes virtual machine aware of the NUMA topology of the underlying physical server topology, which is very important for some specific kind of workload like HPC. Some significant work on vNUMA on PV has been done by Elena Ufimtseva, vNUMA on HVM was also posted by other developers, but none of them was merged upstreamed. This talk will cover the history, design and implementation of vNUMA, and possibly with some number to back up the importance of this feature.

Published in: Technology
1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total Views
824
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
25
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

XPDS14 - vNUMA in Xen - Wei Liu, Citrix

  1. 1. vNUMA in Xen Wei Liu & Elena U
  2. 2. mtseva Chicago { August 18, 2014
  3. 3. Agenda Motives History and status Design Problems Benchmark Future Work Agenda I Motives I History and status I Design I Problems I Preliminary benchmark results I Future work Chicago { August 18, 2014 vNUMA in Xen 2 / 19
  4. 4. Agenda Motives History and status Design Problems Benchmark Future Work Motives I Cross NUMA node memory access is expensive I Need to avoid cross node memory access I Xen is NUMA aware I NUMA aware scheduling I NUMA aware guest memory placement I Operating system like Linux is NUMA aware I NUMA aware scheduling I NUMA aware memory allocation / migration I The missing bits I Memory layout information I CPU topology Chicago { August 18, 2014 vNUMA in Xen 3 / 19
  5. 5. Agenda Motives History and status Design Problems Benchmark Future Work History and status I PV vNUMA presented in Xen Summit 2010 by Dulloor Rao http://slidesha.re/1AXsFbu I HVM vNUMA patches posted by Andre Przywara circa 2010 I Elena U
  6. 6. mtseva has been working on upstreamable PV vNUMA since 2013 Chicago { August 18, 2014 vNUMA in Xen 4 / 19
  7. 7. Agenda Motives History and status Design Problems Benchmark Future Work Design: PV and PVH I Toolstack puts enlightenment information in hypervisor I Guest memory allocation in accordance with enlightenment information I Guest retrieves enlightenment information via hypercall during boot up Chicago { August 18, 2014 vNUMA in Xen 5 / 19
  8. 8. Agenda Motives History and status Design Problems Benchmark Future Work Design: PV and PVH Enlightenment information structure struct vnuma_info { nr_vnodes; vdistance[nr_vnodes * nr_vnodes]; vcpu_to_vnode[nr_vnodes]; vnode_to_pnode[nr_vnodes]; vmemrange[nr_vnodes]; } Chicago { August 18, 2014 vNUMA in Xen 6 / 19
  9. 9. Agenda Motives History and status Design Problems Benchmark Future Work Design: HVM I Toolstack puts enlightenment information in hypervisor I Toolstack arranges ACPI tables I Guest memory allocation in accordance with enlightenment information I Guest retrieves layout information via ACPI tables during boot up Chicago { August 18, 2014 vNUMA in Xen 7 / 19
  10. 10. Agenda Motives History and status Design Problems Benchmark Future Work Problems: vNUMA and other features PV PVH HVM Ballooning Y Y* N PoD N/A ? N Chicago { August 18, 2014 vNUMA in Xen 8 / 19
  11. 11. Agenda Motives History and status Design Problems Benchmark Future Work Problems: CPU topology Chicago { August 18, 2014 vNUMA in Xen 9 / 19
  12. 12. Agenda Motives History and status Design Problems Benchmark Future Work Benchmark I Host I 2 sockets, 12 PCPUs, HT disabled I 36GB RAM, 2 NUMA nodes I NUMA balancing enabled I Guest I 12 VCPUs I 16GB RAM, 2 virtual NUMA nodes I vnodes mapped to dierent pnodes, vcpu pinned to pnode I NUMA balancing enabled I Benchmarks to run I Autonuma I SPECJBB I STREAM Chicago { August 18, 2014 vNUMA in Xen 10 / 19
  13. 13. Agenda Motives History and status Design Problems Benchmark Future Work Benchmark: Autonuma Chicago { August 18, 2014 vNUMA in Xen 11 / 19
  14. 14. Agenda Motives History and status Design Problems Benchmark Future Work Benchmark: Autonuma Chicago { August 18, 2014 vNUMA in Xen 12 / 19
  15. 15. Agenda Motives History and status Design Problems Benchmark Future Work Benchmark: Autonuma Chicago { August 18, 2014 vNUMA in Xen 13 / 19
  16. 16. Agenda Motives History and status Design Problems Benchmark Future Work Benchmark: SPECJBB Chicago { August 18, 2014 vNUMA in Xen 14 / 19
  17. 17. Agenda Motives History and status Design Problems Benchmark Future Work Benchmark: SPECJBB Chicago { August 18, 2014 vNUMA in Xen 15 / 19
  18. 18. Agenda Motives History and status Design Problems Benchmark Future Work Benchmark: STREAM Chicago { August 18, 2014 vNUMA in Xen 16 / 19
  19. 19. Agenda Motives History and status Design Problems Benchmark Future Work Benchmark: Conclusion I vNUMA improves performance for PV guest I vNUMA has weird result in SPECJBB for PVH guest, but other two benchmarks have good results Chicago { August 18, 2014 vNUMA in Xen 17 / 19
  20. 20. Agenda Motives History and status Design Problems Benchmark Future Work Future Work I basic vNUMA support for all guest types I Dom0 vNUMA I address vNUMA compatibility issues with PoD and ballooning for HVM guest I address performance issue for PVH Chicago { August 18, 2014 vNUMA in Xen 18 / 19
  21. 21. Thank you! Chicago { August 18, 2014 vNUMA in Xen 19 / 19
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×