This document provides an overview and configuration tips for Nova, the compute component of OpenStack. It discusses Nova architecture, customizing Nova by choosing backends like hypervisor, database and message queue. It also summarizes new features in Folsom like scheduler and virtualization options, and provides comparisons of RPC backends, databases, and virtualization options available in Nova. Configuration options in Nova are categorized by type and new options introduced in Folsom release are highlighted. Examples of customizing compute manager and scheduler options are also provided.

1. Pimp My Cloud
Nova Configuration Hints and Tricks
Joe Gordon
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 1
Wednesday, October 17, 12

2. About Me
• Engineer at Cloudscaling
• Contributor
• Deployer
• Folsom Contributions
o Top 10 developer
(by commits)
o Mostly in Nova
http://bitergia.com/public/reports/openstack/2012_09_folsom/
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 2
Wednesday, October 17, 12

3. By the Numbers: Nova Folsom
• 190+ Contributors
• Release Python Lines Other Lines Python Files Other Files
Folsom 186,738 242,721 666 788
Essex 150,894 221,109 593 302
Diablo 110,581 110,393 427 389
• Code churn:
Release Lines Insertions Lines Deletions Insertions/LoC %
Folsom 110,308 71,911 59.0%
Essex 182,298 138,346 120.8%
Code churn generated with git log --numstat --pretty="%H" $A..$B| grep .py$ | awk 'NF==3 {plus+=$1; minus+=$2} END {printf("+%d, -%dn", plus,
minus)}'
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 3
Wednesday, October 17, 12

4. Who Wrote Nova Folsom?
1 138 Russell Bryant <rbryant@redhat.com>
2 112 Johannes Erdfelt <johannes.erdfelt@rackspace.com>
3 97 Dan Prince <dprince@redhat.com>
4 88 Vishvananda Ishaya <vishvananda@gmail.com>
5 81 Joe Gordon <jogo@cloudscaling.com>
6 63 Michael Still <mikal@stillhq.com>
7 59 Mark McLoughlin <markmc@redhat.com>
8 58 Rick Harris <rconradharris@gmail.com>
9 50 Yun Mao <yunmao@gmail.com>
10 45 Daniel P. Berrange <berrange@redhat.com>
11 36 Chris Behrens <cbehrens@codestud.com>
12 31 Eoghan Glynn <eglynn@redhat.com>
13 29 Brian Waldon <brian.waldon@rackspace.com>
14 26 Pádraig Brady <pbrady@redhat.com>
15 25 Chuck Short <zulcss@ubuntu.com>
16 23 Sean Dague <sdague@linux.vnet.ibm.com>
17 21 Alex Meade <alex.meade@rackspace.com>
18 18 Kevin L. Mitchell <kevin.mitchell@rackspace.com>
19 17 Brian Elliott <brian.elliott@rackspace.com>
20 17 Zhongyue Luo <zhongyue.nah@intel.com>
21 16 John Griffith <john.griffith@solidfire.com>
22 13 Dan Smith <danms@us.ibm.com>
23 13 Andrew Bogott <abogott@wikimedia.org>
24 12 Renuka Apte <renuka.apte@citrix.com>
25 12 Thierry Carrez <thierry@openstack.org>
git shortlog -sne --since="Tue Mar 20
26 12 Monty Taylor <mordred@inaugust.com> 08:17:40 2012 +0100" --no-merges | cat -n
27 10 MotoKen <motokentsai@gmail.com>
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 4
Wednesday, October 17, 12

5. OpenStack Architecture
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 5
Wednesday, October 17, 12

6. Nova Architecture
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 6
Wednesday, October 17, 12

7. Vanilla Nova
• Devstack
• All in One
• RabbitMQ
• KVM
• MySQL
• Default configuration options
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 7
Wednesday, October 17, 12

8. Customize Nova
• Choose a
o Hypervisor
o Database
o Message Queue
o Network Model
o Scheduler
• Configuration
o Any of the 500+ options
• Custom Services
o Any service can be
swapped out for one
you write
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 8
Wednesday, October 17, 12

9. Customize Nova
• Choose a
o Hypervisor
o Database
o Message Queue
o Network Model
o Scheduler
• Configuration
o Any of the 500+ options
• Custom Services
o Any service can be
swapped out for one
you write
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 9
Wednesday, October 17, 12

10. What Makes Openstack Open
• 3 RPC backends
• 3+ DB backends
• 6+ Virtualization backends
• 500+ configuration options
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 10
Wednesday, October 17, 12

11. New Options for Folsom
• 1 RPC backends
o ZeroMQ
• 2 virt backends
o Hyper-V
o PowerVM
• 100+ configuration options
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 11
Wednesday, October 17, 12

12. New Features for Folsom by Blueprint
• General host aggregates (operator)
• Disable API extensions
• Rootwrap pluggable filters (operator)
• Multi-scheduler support (operator)
• Flavor extra-specs
• LVM ephemeral disk images (operator)
• Project specific flavors
• Multi-process API services (operator)
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 12
Wednesday, October 17, 12

13. Backend Choices
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 13
Wednesday, October 17, 12

14. RPC Comparisons
RPC
Protocol Company Affiliations Brokerless Language
Backend
kombu
AMQP 0-9-1 Erlang
(rabbitMQ)
qpid AMQP 0-10 Java, C++
ZeroMQ Python, C++
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 14
Wednesday, October 17, 12

15. Database Comparisons
Database Simultaneous writes High Availability Native python client
?
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 15
Wednesday, October 17, 12

16. Virtualization Comparisons
Virtualization
Backend maintained by Fast Open Source
type
Baremetal N/A N/A
UML paravirt only
LXC container
Xen full/paravirt
KVM full
Qemu qemu.org full
Hyper-V full
PowerVM full
VMware ESX full
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 16
Wednesday, October 17, 12

17. Configuration Options
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 17
Wednesday, October 17, 12

18. Configuration Options by Type
Boolean
68
Floats
6
Ints
119 13%
Lists
18 1%
MultiString
6
String
290
23%
57%
4%
1%
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 18
Wednesday, October 17, 12

19. New Config Options in Folsom
allowed_rpc_exception_modules max_age
api_url netapp_storage_service_prefix
auth_blob nfs_mount_point_base
backdoor_port non_inheritable_image_properties
base_dir_name notify_on_any_change
cinder_* notify_on_state_change
claim_timeout_seconds osapi_max_request_body_size
config_drive_format osapi_volume_workers
config_drive_skip_versions periodic_fuzzy_delay
config_drive_tempdir port
default_scheduler_driver powervm_*
defer_iptables_apply quantum_*
ec2_strict_validation quota_driver
ec2_workers quota_key_pairs
fatal_deprecations quota_security_group_rules
force_config_drive quota_security_groups
glance_api_insecure rbd_*
hyperv_attaching_volume_retry_count reservation_expire
hyperv_wait_between_attach_retry rootwrap_config
image_info_filename_pattern rpc_cast_timeout
instance_build_timeout rpc_zmq_*
instance_uuid_format server
libvirt_cpu_mode server_ca_file
libvirt_cpu_model snapshot_same_host
libvirt_images_type sql_connection_debug
libvirt_images_volume_group sql_connection_trace
libvirt_inject_key storwize_*
libvirt_inject_partition syslog_log_facility
libvirt_snapshots_directory until_refresh
libvirt_sparse_logical_volumes vswitch_name
limit_cpu_features xenapi_check_host
matchmaker_ringfile xenapi_num_vbd_unplug_retries
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 19
Wednesday, October 17, 12

20. Virtualization options
1.pick a compute RPC topic
scheduler_topic=compute
2.pick a compute manager
compute_manager=nova.compute.manager.ComputeManager
3.compute manager options
instances_path=$state_path/instances
base_dir_name=_base
compute_driver=libvirt.LibvirtDriver
console_host=socket.getfqdn()
live_migration_retry_count=30
reboot_timeout=0
instance_build_timeout=0
rescue_timeout=0
resize_confirm_window=0
*_interval=
running_deleted_instance_timeout=0
running_deleted_instance_action=log
instance_usage_audit=false
4.compute driver options
rescue_*_id=
libvirt_type=kvm
libvirt_inject_password=false
libvirt_inject_key=true
*_migration_=
libvirt_cpu_mode=
libvirt_cpu_model=
...
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 20
Wednesday, October 17, 12

21. compute.manager options
cfg.IntOpt('live_migration_retry_count',
default=30,
help="Number of 1 second retries needed in live_migration"),
cfg.IntOpt("reboot_timeout",
default=0,
help="Automatically hard reboot an instance if it has been "
"stuck in a rebooting state longer than N seconds. "
"Set to 0 to disable."),
cfg.IntOpt("instance_build_timeout",
default=0,
help="Amount of time in seconds an instance can be in BUILD "
"before going into ERROR status."
"Set to 0 to disable."),
cfg.IntOpt("rescue_timeout",
default=0,
help="Automatically unrescue an instance after N seconds. "
"Set to 0 to disable."),
cfg.IntOpt("resize_confirm_window",
default=0,
help="Automatically confirm resizes after N seconds. "
"Set to 0 to disable."),
...
cfg.StrOpt("running_deleted_instance_action",
default="log",
help="Action to take if a running deleted instance is detected."
"Valid options are 'noop', 'log' and 'reap'. "
"Set to 'noop' to disable."),
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 21
Wednesday, October 17, 12

22. reap running deleted instances
cfg.IntOpt("running_deleted_instance_timeout",
default=0,
help="Number of seconds after being deleted when a running "
"instance should be considered eligible for cleanup."),
cfg.IntOpt("running_deleted_instance_interval",
default=30,
help="Number of periodic scheduler ticks to wait between runs of "
"the cleanup task."),
cfg.StrOpt("running_deleted_instance_action",
default="noop",
help="Action to take if a running deleted instance is detected."
"Valid options are 'noop', 'log' and 'reap'. "
"Set to 'noop' to disable."),
nova.conf settings
running_deleted_instance_timeout=60
running_deleted_instance_action=reap
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 22
Wednesday, October 17, 12

23. Scheduler options
1.pick a scheduler RPC topic
scheduler_topic=scheduler
2.pick a scheduler manager
scheduler_manager=nova.scheduler.manager.SchedulerManager
3.pick a scheduler driver
scheduler_driver=nova.scheduler.multi.MultiScheduler
4.set up scheduler.driver options
scheduler_host_manager=nova.scheduler.host_manager.HostManager
scheduler_max_attempts=3
5.multi scheduler options
compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler
volume_scheduler_driver=nova.scheduler.chance.ChanceScheduler
default_scheduler_driver=nova.scheduler.chance.ChanceScheduler
6.host_manager options
scheduler_available_filters=nova.scheduler.filters.standard_filters
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeC
apabilitiesFilter,ImagePropertiesFilter
7.filter options
ram_allocation_ratio=1.5
8.resource_tracker options
reserved_host_disk_mb=0
reserved_host_memory_mb=512
claim_timeout_seconds=600
compute_stats_class=nova.compute.stats.Stats
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 23
Wednesday, October 17, 12

24. Add custom filter
cfg.MultiStrOpt('scheduler_available_filters',
default=['nova.scheduler.filters.standard_filters'],
help='Filter classes available to the scheduler which may '
'be specified more than once. An entry of '
'"nova.scheduler.filters.standard_filters" '
'maps to all filters included with nova.'),
cfg.ListOpt('scheduler_default_filters',
default=[
'RetryFilter',
'AvailabilityZoneFilter',
'RamFilter',
'ComputeFilter',
'ComputeCapabilitiesFilter',
'ImagePropertiesFilter'
],
help='Which filter class names to use for filtering hosts '
'when not specified in the request.'),
nova.conf settings
scheduler_available_filters=myfilter.CustomFilter
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,
ComputeCapabilitiesFilter,ImagePropertiesFilter,CustomFilter
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 24
Wednesday, October 17, 12

25. *_driver *_manager
• network • compute
• quota • console
• compute • cert
• console • instance_dns
• db • floating_ip_dns
• linuxnet_interface • network
• scheduler • volume
o compute_scheduler
o volume_scheduler • scheduler
o default_scheduler • metadata
• baremetal • image_cache
• firewall • consoleauth
• libvirt_vif • scheduler_host
• volume_driver
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 25
Wednesday, October 17, 12

26. Nova Architecture
manager
Driver
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 26
Wednesday, October 17, 12

27. Resources
Blueprints: https://blueprints.launchpad.net/nova/folsom
Source: https://github.com/openstack/nova/tree/stable/folsom
Documentation: http://docs.openstack.org
Release notes: http://wiki.openstack.org/ReleaseNotes/Folsom
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 27
Wednesday, October 17, 12

28. Questions?
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution.*
* All unlicensed or borrowed works retain their original licenses. 28
Wednesday, October 17, 12