Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

XPDDS18: Performance tuning on Xen platform - Bo Zhang & Yifei Jiang, Huawei

56 views

Published on

Huawei Cloud applies xen platform to many customer scenarios. This talk will introduce our optimizations on the xen platform to solve problems occuring in these scenarios.
E.g
1. Redesign the implementation of kernel locks to improve the scalability of the Xen platform in large-scale server scenarios.
2. Develop LazyFPU and L3 cache affinity features to improve virtual machine performance in SAP HANA database service scenarios.
3. Develop HostNUMA and GuestNUMA features to enhance virtual machine performance in specvirt test and desktop cloud scenarios.
4. Shorten the time cost of concurrent life-cycle operations for large scales of virtual machines, to achieve quick change of classes in the cloud classroom.

Published in: Sports
  • Be the first to comment

  • Be the first to like this

XPDDS18: Performance tuning on Xen platform - Bo Zhang & Yifei Jiang, Huawei

  1. 1. HUAWEI TECHNOLOGIES CO., LTD. www.huawei.com Huawei Confidential Security Level:2018/7/2 Bo Zhang <oscar.zhangbo@Huawei.com> Yifei Jiang <jiangyifei@Huawei.com> Performance tuning on Xen platform
  2. 2. Agenda ◼ Application Scenarios of Huawei Cloud ◼ Scenario 1: SAP HANA ◼ Scenario 2: Desktop Cloud ◼ Scenario 3: Cloud Classroom
  3. 3. HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Application Scenarios of Huawei Cloud Page 3 • Huawei UVP supports 100 cloud services under 14 catagories • Public cloud • Private cloud • NFV • Usage scenarios that have performance demands for virtualization platform • SAP HANA • Desktop cloud(VDI) • Cloud classroom • ……
  4. 4. HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Scenario 1: SAP HANA Page 4 • scenario • Large-sized servers & virtual machines • Un-overcommitted CPU & memory • Performance demands • Meet the SAP HANA virtualization certification’s performance index. • Performance Barriers:  Single-vm:the performance of a vm should be lower no more than 10% compared to the physical machine of the same size.  Multi-vm: no more than 5% performance degradation when other guests are under pressure, compared with that other guests are idle.
  5. 5. HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Scenario 1: SAP HANA Page 5 • Approach • Keep vm’s hardware attributes consitent with physical machines • Cpu topology & L3 cache affinity • NUMA • Virtualization platform cost • Save the cost of Vmexit/vmentry • Kernel lock optimization • Resource isolation of CPU, memory and cache • Dom0 and DomU • DomU and DomU
  6. 6. HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Cpu topology & L3 cache affinity Page 6 • Problem • the server & client thread inside the guest that communicate via a socket share the same VCPU. • Thus both the server thread & client thread could not obtain the computing power of a single pcpu. • Root cause • The linux scheduler inside guest senses no MC scheduling domain. • The cache-affinity threads are scheduled on the same VCPU. • Technical solution • Present CPU topology information to guests. • Make L3 cache shared among vcpus that is under the same socket of the guest.
  7. 7. HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Save the cost of vmexit/vmentry Page 7 • Problem • Large amount of CLTS instructions cause massive vmexit cost • PLE may casue vmexit when vcpu paused for too long • Technical solution • Reduce vmexit times when CLTS instruction is executed. • Disable PLE.
  8. 8. HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Kernel lock optimization Page 8 • Problem • Xen uses ticket spinlock by default. Ticket spinlock is quite performace consuming for many core.  Spin on global shared variable, among large amounts of pcpus  Expensive cache entries invalidation • Technical solution • MCS lock is scalable.  Spin on local variable  Generate a constant number of cache misses per acquisition, avoid the performance collapse with many cores.
  9. 9. HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Resource isolation Page 9 • Problem • On multi-vm scenario, other domU and dom0 may have large pressure, and may affect the tested domU. • Technical solution • Isolation on domU and domU • Isolation on domU and dom0
  10. 10. HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Scenario 2: VDI cloud Page 10 • Desktop cloud demands: • Large amont of small-sized virtual machines • Performance demands:VSI density test • Approach • PLE enabled • Guests’ memory and cpu location optimization • Qemu resources limited: rm unused devices&slowdown devices’timer’s frequency • Hyperv to guests • ……
  11. 11. HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Scenario 3: Cloud Classroom Page 11 • Scenario introduction • Cloud classroom customers requires rapid completion of server power-up to a large numbers guests get online. • Fast virtual machines’shutdown-startup-image replacement operations during class change. • Approach • Virtual machines’ concurrent lifecycle operation timecost optimization • Optimize the poweron/poweroff timecost inside guests
  12. 12. HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential lifecycle operation timecost optimization Page 12 • Problem • Xenstore has poor concurrent write permance • Approach • Use serialized xenstore write to avoid atomic breakup during guests concurrent startup. • Weak write that do not disturb transactions
  13. 13. HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Timecost optimization inside guests Page 13 • Problem • When a large scale of virtual machines start/stop concurrently, the power on/off timecost inside guests takes long too. • Root cause • Lots of IO r/w operation inside virtual machine during power on/off • Technical solution • Use memory disks for virtual machines • Expand the number of grant-table to improve disk IO r/w performance • Destroy virtual machines, rather than shutdown.
  14. 14. Thank you www.huawei.com

×