Open Source Virtualization Hacks

Open Source Virtualization Hacks

Niel M. Bornstein

[email_address]

O'Reilly Open Source Convention 2008

Agenda

The Abstract

The Technology

The Hacks

The Abstract

Open source virtualization systems use the usual suite of tools and languages—can anyone out there say Python and XML?

In this session, you’ll see some ways to build a management system, using the tools you already know, to do some wicked things with virtual machines.

Along the way you’ll learn about the Open Virtual Machine Format (OVF) and some of the other emerging standards that are helping modern data centers run.

The Technology

This presentation will make use of the Xen hypervisor on openSUSE 11.0, using libvirt's python bindings.

The hacks and methods presented should be equally applicable to other operating systems and hypervisors supported by libvirt.

The Technology – openSUSE 11.0

“The openSUSE project is a community program sponsored by Novell.

“Promoting the use of Linux everywhere, openSUSE.org provides free, easy access to the world's most usable Linux distribution, openSUSE.

“The openSUSE project gives Linux developers and enthusiasts everything they need to get started with Linux.”

http://software.opensuse.org/

The Technology – Xen 3.2.1

“The Xen ® hypervisor, the powerful open source industry standard for virtualization, offers a powerful, efficient, and secure feature set for virtualization of x86, x86_64, IA64, PowerPC, and other CPU architectures.

“It supports a wide range of guest operating systems including Windows ® , Linux ® , Solaris ® , and various versions of the BSD operating systems.”

http://xen.org/

The Technology – libvirt 0.4.0

“A toolkit to interact with the virtualization capabilities of recent versions of Linux (and other OSes)

“Free software available under the GNU Lesser General Public License

“A long term stable C API

“A set of bindings for common languages

“A CIM provider for the DMTF virtualization schema”

http://libvirt.org/python.html

The Technology – Python 2.5.2

“Python is a dynamic object-oriented programming language that can be used for many kinds of software development.”

http://python.org

The Hacks

What’s a Hack?

“Hacks are tools, tips, and tricks that help users solve problems. They are aimed at intermediate-level power users and scripters.”

http://oreilly.com/hacks/

Eleven Hacks in Three Categories

Exploration

Basic Tasks

Advanced Tasks

Hack #1 – The xm Command

Most Linux geeks like to use the command line. The Xen hypervisor provides a command line management user interface. The xm command allows you to do most common single-host management tasks.

NAME xm - Xen management user interface SYNOPSIS xm <subcommand> [args] DESCRIPTION The xm program is the main interface for managing Xen guest domains. The program can be used to create, pause, and shutdown domains. It can also be used to list current domains, enable or pin VCPUs, and attach or detach virtual block devices. The basic structure of every xm command is almost always: xm <subcommand> <domain-id> [OPTIONS] ...

Hack #1 – The xm Command (cont'd)

The command line is very useful, but can be tricky to do some tasks.

For example, to install a new VM from scratch, you first need to define it using configuration files. Only then can you use the xm create command to start the install process.

More about the configuration file format, and ways to edit it, later.

Because it's a complex process, virt-manager makes it much easier to build a new VM.

Hack #2 – The virsh Command

virsh is another command line tool, using libvirt to provide access to a variety of hypervisors in a generic manner.

More about libvirt later!

NAME virsh - management user interface SYNOPSIS virsh <subcommand> [args] DESCRIPTION The virsh program is the main interface for managing virsh guest domains. The program can be used to create, pause, and shutdown domains. It can also be used to list current domains. Libvirt is a C toolkit to interact with the virtualization capabilities of recent ver‐ sions of Linux (and other OSes). It is free software available under the GNU Lesser General Public License. Virtualization of the Linux Operating System means the ability to run multiple instances of Operat‐ ing Systems concurrently on a single hardware system where the basic resources are driven by a Linux instance. The library aim at providing long term stable C API initially for the Xen paravirtualization but should be able to integrate other virtualization mechanisms, it cur‐ rently also support QEmu and KVM. ...

Hack #2 – The virsh Command (cont'd)

The entire libvirt API is available from the virsh command.

chonju:~ # virsh Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # dominfo 1 Id: 1 Name: opensuse11 UUID: 60c451d0-9009-2bd6-1cfb-f71e9ec6926d OS Type: linux State: blocked CPU(s): 1 CPU time: 3.3s Max memory: 393216 kB Used memory: 393216 kB virsh # shutdown 1 Domain 1 is being shutdown virsh #

Hack #3 – virt-manager

virt-manager is a project hosted by Red Hat designed to provide a graphical UI for managing virtual machines.

It is written in Python with Glade and GTK+ and uses libvirt to access a variety of virtualization systems.

It is included in most modern Linux distributions.

Hack #3 – virt-manager (cont'd)

You can even view multiple virtualization technologies through a single interface.

Select the File -> Open Connection menu command and you will see that you can also connect to a QEMU hypervisor in the same interface, as well as making remote connections to other machines.

Hack #3 – virt-manager (cont'd)

Building a new VM is also simple.

Click on the New button to bring up the vm-install process, which can also be launched directly from the command line.

The vm-install wizard will present you with all the options you need to configure a new VM, and it's much easier than manually editing configuration files.

Tip: To build a VM with an ISO install source, loop mount the iso first:

losetup -f /path/to/iso

Hack #4 – libvirt + Python

After the command line and the GUI tool, the next step is to build your own tools programmatically.

Using libvirt , you can manage a variety of virtualization systems from a variety of programming languages.

For these examples, we'll use Xen and Python, because, well, it's really easy.

After the first example, I will omit error checking for brevity!

Hack #4 – libvirt + Python (cont'd)

The following sample code comes from the libvirt website:

#!/usr/bin/python import libvirt import sys conn = libvirt.openReadOnly(None) if conn == None: print 'Failed to open connection to the hypervisor' sys.exit(1) try: dom0 = conn.lookupByName("Domain-0") except: print 'Failed to find the main domain' sys.exit(1) print "Domain 0: id %d running %s" % (dom0.ID(), dom0.OSType()) print dom0.info()

Hack #5 – Query the Hypervisor's Capabilities

Before taking any action using libvirt , it's a good idea to get an idea of what the hypervisor is capable of doing.

libvirt provides a couple of convenient methods for doing this:

virConnect.getCapabilities() returns information in XML format about the hypervisor.

virConnect.getInfo() returns a python dict of information about the physical node.

Hack #5 – Query the Hypervisor's Capabilities (cont'd) #!/usr/bin/python import libvirt conn = libvirt.open(None) print conn.getCapabilities() info = conn.getInfo() print "CPU model: %s" % info[0] print "memory: %d kB" % info[1] print "# of CPUs: %d" % info[2] print "CPU freq: %d MHz" % info[3] print "# of NUMA cell: %d" % info[4] print "# of CPU sockets: %d" % info[5] print "# of cores per socket: %d" % info[6] print "# of threads per core: %d" % info[7]

Hack #6 – Start/Stop a VM

Now that we know what the hypervisor is capable of, we can start a VM instance.

We've already seen how to do this with the xm command and virt-manager ; now we'll do it from a Python script.

#!/usr/bin/python import sys, libvirt vmname = sys.argv[1] conn = libvirt.open(None) dom = conn.lookupByName(vmname) dom.create() dom = conn.lookupByName(vmname) print "%d started" % dom.ID()

Hack #6 – Start/Stop a VM (cont'd)

Similarly, we can stop a VM, no matter whether it was started through libvirt , the xm command, or virt-manager .

#!/usr/bin/python import sys, libvirt vmid = int(sys.argv[1]) conn = libvirt.open(None) dom = conn.lookupByID(vmid) dom.shutdown()

Hack #7 – Migrate a VM

In real life, you may occasionally need to move a running VM from one physical host to another. This is referred to as “migrating” the VM.

I can't demonstrate doing this on a single laptop, but I can show you some code that should work!

Hack #7 – Migrate a VM (cont'd) #!/usr/bin/python import sys, getpass, libvirt vmname = sys.argv[1] uri = sys.argv[2] mydata = "" def getCredentials(credentials, data): for credential in credentials: print credential[1] + ":", if credential[0] == libvirt.VIR_CRED_AUTHNAME: data = sys.stdin.readline() data = data[0:len(data)-1] credential[4] = data elif credential[0] == libvirt.VIR_CRED_PASSPHRASE: credential[4] = getpass("") else: return -1 return 0 flags = [libvirt.VIR_CRED_AUTHNAME,libvirt.VIR_CRED_PASSPHRASE] auth = [flags,getCredentials,mydata] localconn = libvirt.open(None) dom = localconn.lookupByName(vmname) remoteconn = libvirt.openAuth(None,auth,0) dom.migrate(remoteconn,libvirt.VIR_MIGRATE_LIVE,None,uri,0)

Hack #7 – Migrate a VM (cont'd)

In reality, it ends up being much easier to use the xm migrate command to do this. If I had to make a recommendation, I'd say that unless you're building a complete management system, you should just use the xm or virsh tools to migrate your VMs:

xm migrate --live domain newhost

virsh migrate --live domain newhost

In these commands, the domain parameter is either the name or id of the domain, and the newhost parameter is the URI of the new host.

Hack #7 – Migrate a VM (cont'd)

The /etc/xen/xend-config.sxp file that ships with Xen does not allow migrations. Make the following changes to turn migration on:

Uncomment this line and change no to yes:

#(xend-relocation-server no)

Uncomment this line:

#(xend-relocation-port 8002)

Uncomment this line:

#(xend-relocation-address '')

Customize this line for your security needs:

(xend-relocation-hosts-allow '^localhost$ ^localhost\.localdomain$')

Then restart xend with the command rcxend restart

Hack #8 – Edit a VM Configuration

A VM can be thought of as a combination of configuration information and a disk image.

You can change the VM configuration when a VM instance is not running by editing the configuration file (in Xen, that's the file in /etc/xen/vm/ vmname ).

Hack #8 – Edit a VM Configuration (cont'd)

It's easy enough to modify a config by hand, once you understand the format.

name="opensuse11" uuid="60c451d0-9009-2bd6-1cfb-f71e9ec6926d" memory=384 vcpus=1 on_poweroff="destroy" on_reboot="restart" on_crash="destroy" localtime=0 keymap="en-us" builder="linux" bootloader="/usr/lib/xen/boot/domUloader.py" bootargs="--entry=xvda2:/boot/vmlinuz-xen,/boot/initrd-xen" extra=" " disk=[ 'file:/var/lib/xen/images/opensuse11/disk0,xvda,w', ] vif=[ 'mac=00:16:3e:49:b8:b2', ] vfb=['type=vnc,vncunused=1']

But why do it by hand when we've got Python?

#!/usr/bin/python import sys, os original = sys.argv[1] new = sys.argv[2] f = file(original, "rb") lines = f.readlines() dict = {} for line in lines: pieces = line.partition("=") dict[pieces[0]] = eval(pieces[2]) for key in dict.keys(): if key == "name": dict[key] = new elif key == "disk": disks = dict[key] dict[key] = [] for disk in disks: dict[key].append(disk.replace(os.path.basename(original),new)) elif key == "vif": dict[key] = None elif key == "uuid": dict[key] = None if dict[key] != None: print "%s=%s" % ( key, repr(dict[key]) )

You can change the configuration of a running VM instance using the xm or virsh commands, or using libvirt .

#!/usr/bin/python import sys, libvirt vmname = sys.argv[1] maxMemory = int(sys.argv[2]) conn = libvirt.open(None) dom = conn.lookupByName(vmname) print dom.maxMemory() dom.create() dom.setMaxMemory(maxMemory) dom = conn.lookupByName(vmname) print dom.maxMemory()

Hack #9 – Edit a VM Image

Editing a VM image file is called “cracking it open”. The image file needs to be mounted as a loopback filesystem.

The easiest way to crack open a Xen image is to loop mount it using lomount .

lomount -diskimage disk0 -partition 2 /mnt

Bear in mind that the image file will almost certainly have more than one partition.

This method will not work with LVM disks!

Hack #10 – Clone a VM

Now we have edited a VM configuration and image.

Cloning a VM is as simple as copying the configuration and image to a new location, then editing the configuration and “personalizing” the image.

You just saw how our friend Python can easily read in a configuration file and make the necessary changes.

Hack #10 – Clone a VM (cont'd)

Just one little problem: my disk image is 8 gigs. That takes a while to copy

The solution: Copy-on-Write.

Use the QEMU CoW (qcow) format

Convert your raw image to qcow using img2qcow

img2qcow destination source

There is also a corresponding qcow2raw

Hack #11 – Create an Appliance

Appliances can be defined using Open Virtual Machine Format (OVF), a standard proposed by VMware under the auspices of the Distributed Management Task Force (DMTF).