This document discusses boosting compute performance for virtual machines (VMs) in an OpenStack cloud. It covers VM placement in OpenStack, which impacts both VM performance and efficient resource usage. Specifically, it focuses on NUMA-aware scheduling and CPU pinning to improve performance for high-performance workloads. A demo is provided of reserving dedicated CPU cores for host processes and VMs in OpenStack Kilo to enable these techniques. Benchmark results show performance improvements for VMs using these methods.

1. Boost Compute Performance for VMs in a Cloud
Thang Man
1
Viet OpenStack Meetup 7th
@Saigon 2015

2. $ cat /me.info
★ Open Source Cloud Computing Follower
mvthang
manvanthang
manthang.wordpress.com
2

3. Agenda
● Motivation
● VM placement in OpenStack
● Boost compute performance for VMs under KVM
○ NUMA-aware scheduling
○ CPU Pinning
● Demo
3

4. Motivation
When / Why we consider running high performance VMs in
a cloud?
And the challenges facing us?
4

5. Popular HPC Workloads
5

6. HPC on Cloud: The benefits of cost
6
On-Premises Cloud Services

7. HPC on Cloud: The benefits of elasticity
7

8. HPC on Cloud: The key challenges
8
The performance issues facing us…
Compute (processor, memory) (*)
Network I/O
Disk I/O
“Noisy neighbours” effect
etc.
(*) Our focus today :).

9. VM Placement in OpenStack
VM or cloud instance placement is one of the most crucial
components of most clouds. It has not only impact on the
performance of VMs, but also the efficient resource usage
of the cloud.
9

10. Instance Creation Flow
10

11. Nova Scheduler
11
Using Filter Scheduler by default
● NUMATopologyFilter
● AggregateInstanceExtraSpecsFilter

12. Why NUMA?
12
NUMA helps to surpass the scalability, bandwidth and latency limits of the SMP
(symmetric multiprocessing) architecture, e.g. UMA.

13. NUMA Placement
13
Non-Optimal NUMA PlacementOptimal NUMA Placement

14. NUMA Downsides
14
Good: Low latency
(local memory access)
Bad: High latency
(remote memory access)

15. Some KVM-NUMA Suggestions
15
● Pinning guest vCPUs to pCPU cores
○ No share those pinned cores with other VMs or Host.
○ Help avoid CPU cache misses.
● Don't assign over compute resources to guests
○ Guest vCPUs <= available pCPU cores per node.
○ Guest memory < available memory per node.
⇒ Say “NO” to overcommit!
● In OpenStack, using Host Aggregates to separate VMs with dedicated
resourcing requirements from those without dedicated resources.

16. Demo in OpenStack Kilo (1)
16
● Reserving dedicated cores for host processes.
# cat /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash isolcpus=4-23"
● Reserving dedicated cores for VMs.
● Enabling the required scheduler filters.
# cat /etc/nova/nova.conf
[DEFAULT]
vcpu_pin_set=4-23
scheduler_default_filters=NUMATopologyFilter,AggregateInstanceExtraSpecsFilter,...

17. Demo in OpenStack Kilo (2)
17
● Creating a Host Aggregate to add all hosts configured for CPU pinning to.
● Creating a performance focused Flavor to target this Host Aggregate.
● Launching an instance with above performance focused Flavor!

18. Benchmark Results
18
CERN: Tuning hypervisors for High Throughput Computing
http://openstack-in-production.blogspot.com/2015/08/kvm-and-hyper-v-comparison-for-high.html

19. Reference
19
● http://docs.openstack.org/
● http://redhatstackblog.redhat.com/2015/05/05/cpu-pinning-and-numa-
topology-awareness-in-openstack-compute/
● https://moinakg.wordpress.com/2013/06/05/findings-by-google-on-numa-
performance/
● http://frankdenneman.nl/2015/02/27/memory-deep-dive-numa-data-locality/
● http://blogs.technet.com/b/windowsserver/archive/2012/04/05/windows-
server-8-beta-hyper-v-amp-scale-up-virtual-machines-part-1.aspx
● http://blog.platform9.com/virtual-machine-placement-openstack/
● http://www.slideshare.net/AmazonWebServices/intro-to-high-performance-
computing-in-the-aws-cloud

20. Thank You!
Q&A
20