Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

XPDDS18: Memory Overcommitment in XEN - Huang Zhichao, Huawei


Published on

This talk will introduce our practice on Memory Overcommitment in XEN, and share some findings and lessons. i.e.: The best practice of POD(Populate On Demand), including live migration of the POD pages; Introduce the mem-shr, a memory-saving de-duplication feature, to merge the samepages; Xenpaging optimizing, including some policy enhancements; Scalability investigation and enhancements on Memory Overcommitment; What fields Memory Overcommitment benefits Huawei Cloud, and some performance data

Published in: Technology
  • Be the first to comment

  • Be the first to like this

XPDDS18: Memory Overcommitment in XEN - Huang Zhichao, Huawei

  1. 1. HISILICON SEMICONDUCTORHuawei Confidential Page 1 HUAWEI TECHNOLOGIES CO., LTD. Memory Overcommitment in XEN June, 2018 Zhichao Huang <>
  2. 2. Agenda • Use cases • Issues & Proposals • Summary
  3. 3. What is memory overcomitment • Allows users to power on virtual machines (VMs) with a total configured memory that exceeds the memory available on the physical machine. Currently, we use it for: – More than 1000 key customers, 100,000 + VMs – More then 150% overcommitment ratio – More than one year without any bugs
  4. 4. Use cases • Higher consolidation ratio – Traditional business clouding – Old physical machine and same memory size – server virtualization – All VMs with large memory size, to avoid serious waste of resource – scenario under low loads – VDI, Web service, improve resource utilization • Dynamic resource scheduling – high availability – distributed power management – Software/Hardware upgrades - increase the concurrency of live migration
  5. 5. How to do it • Memory Popluate on Demand – Basic technique – Not useful only by itself – POD reclaim in Hypervisor is too expensive • Memory Ballooning – Requires a driver inside VMs – Must use it carefully • Memory Sharing – Stable and effective – Works well most of the time Memory Ballooning VM1 VM2 Balloon Free Used Used Free Memory Sharing VM1 VM2 VM3 Physical Memory
  6. 6. How to do it • Memory Swapping – Guaranteed, in an emergency – Disk I/O speed is the key point • Memory Compression – A nice toy • Memory Overcommitment Policy – When, What, How – Memory Qos (Reservation, shares, limit) Memory Swapping Disk VM VM
  7. 7. Memory Overcommitment scalability • Lots of optimization in upstream – Use unlocked p2m lookups in hvmemul_rep_movs – Defer the invalidation until the p2m lock is released – Use per-cpu lock instead of global shr-lock – …… • Some other optimization from Huawei – Frequent ipi to invalidate ept on related pCPUs – Global calllock -> per-cpu lock – Ticket lock -> mcs lock(scalable lock) – ……
  8. 8. Evaluation Results With the optimization, the number of lock wait and each wait consuming are both reduced significantly
  9. 9. With other feature • With migration / Snapshot / Hiberate – Time consumption is reduced to 1/3 or less • With Guest Dump – Time consumption is reduced to 1/2 or less • With Device Passthrough – Full memory reservation automaticly • ……
  10. 10. Summary • Use cases • Stability • Performance • Scalability • Compatibility
  11. 11. Thank you