1 Xen Installation . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Domain Installation . . . . . . . . . . . . . . . . . . . . . . . . . 5
3 Configuring a Xen Guest Domain . . . . . . . . . . . . . . . . . . . 8
4 Preparing the Network . . . . . . . . . . . . . . . . . . . . . . . 9
5 Starting and Controlling Xen Domains . . . . . . . . . . . . . . . . . 9
6 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . 12
7 For More Information . . . . . . . . . . . . . . . . . . . . . . . 13
Xen makes it possible to run several Linux systems on virtually provided hardware.
This article gives an overview of the possibilities and limitations of this technology.
Sections about installing, configuring, and running Xen complete this introduction.
Virtual machines commonly need to emulate the hardware a system needs. The disad-
vantage is that the emulated hardware is much slower than the real silicon. Xen has a
different approach. It restricts emulation to as few parts as possible. To achieve this,
Xen uses paravirtualization. This is a technique that presents virtual machines similarly,
but not identically to the underlying hardware. Therefore, host and guest operating
systems are adapted on the kernel level. The user space remains unchanged.
Xen controls the hardware with a hypervisor and a controlling guest, also called domain-
0. These provide all needed virtualized block and network devices. The guest systems
use these virtual block and network devices to run the system and connect to other
guests or the local network. When several physical machines running Xen are configured
in a way that the virtual block and network devices are available, it is also possible to
migrate a guest system from one piece of hardware to another while running. Originally,
Xen was developed to run up to 100 guest systems on one computer, but this number
depends strongly on the system requirements of the running guest systems, especially
the memory consumption.
To limit the CPU usage, the Xen hypervisor offers three different schedulers. The
scheduler also may be changed while running the guest system, making it possible to
Virtualization with Xen 3
change the priority of the running guest system. On a higher level, migrating a guest
may also be used to adjust the available CPU power.
The Xen virtualization system also has some drawbacks regarding the supported hard-
ware. Several closed source drivers, like those from Nvidia or ATI, do not work as ex-
pected. In these cases, you must use the open source drivers if available, even if they
do not support the full capabilities of the chips. Also several WLAN chips and Cardbus
bridges are not supported when using Xen. In version 2, Xen does not support PAE
(physical address extension), which means that it does not support more than 4 GB of
memory. ACPI is not supported. Power management and other modes that depend on
ACPI do not work.
Figure 1 Xen Overview
Management Service Service
Host OS Guest OS Guest OS
Linux Kernel Linux Kernel NetWare Kernel
IO dom0 virt. virt. virt. virt.
Xen control CPU Memory Network Blockdev
IO CPU Memory
Physical Hardware (CPU, Memory, Network, Block Devices)
1 Xen Installation
The installation procedure of Xen involves the setup of a domain-0 domain and the in-
stallation of Xen guests. First, make sure that the needed packages are installed. These
are python, bridge-utils, xen, xen-tools, xen-tools-ioemu, and a
kernel-xen package. When selecting Xen during installation, Xen is added to the
GRUB configuration. For other cases, make an entry to boot/grub/menu.lst.
This entry should be similar to the following:
kernel (hd0,0)/boot/xen.gz dom0_mem=458752
module (hd0,0)/boot/vmlinuz-xen <parameters>
Replace (hd0,0) with the partition that holds your /boot directory. Alter the amount
of dom0_mem to match your system. The maximum value is your system memory in
kB minus 65536 (147456 for AMD64 and Intel EM64T). Replace <parameters> with
the parameters normally used to boot a Linux kernel.
Then reboot into Xen mode. This boots the Xen hypervisor and a slightly changed
Linux kernel as Domain-0 that runs most of the hardware. Apart from the exceptions
already mentioned, everything should work as usual.
2 Domain Installation
The installation and setup of a guest domain involves several procedures. In the follow-
ing, a first guest domain is installed and all the different tasks to create a first network
connection are completed.
To install a guest system, you must provide a root file system in a block device or in a
file system image, which needs to be set up. To access this system later, use an emulated
console or set up the network connection for this guest. The installation of SUSE Linux
into a directory is supported by YaST. The hardware requirements of such a guest are
similar to a normal Linux installation.
Domains can share file systems that are mounted read-only from all domains, such as
/usr or /opt. Never share a file system that is mounted read-write. For sharing
writable data among several guest domains, use NFS or other networked or cluster file
Virtualization with Xen 5
Starting a Guest Domain
When you start a guest domain, make sure that the file systems of
the guest are not mounted by an installer or by the controlling
The first thing to do is to create a file system image in which to install the Linux for
1 To create an empty image named guest1 in the directory /var/tmp/ that is
4 GB in size, use the following command:
dd if=/dev/zero of=/var/tmp/guest1 seek=1M bs=4096 count=1
2 The image is just a big empty file without any information in it. To be able to
write files into it, a file system is needed:
mkreiserfs -f /var/tmp/guest1
The command mkreiserfs informs you that this is not a block special device
and asks for a confirmation. Enter Y then Enter to continue.
3 The actual installation is made in a directory. Therefore the file system image
/var/tmp/guest1 must be mounted to a directory:
mkdir -p /var/tmp/dirinstall
mount -o loop /var/tmp/guest1 /var/tmp/dirinstall
When you are finished with the installation, unmount this file system
image. YaST also mounts the /proc file system when installing,
which must be unmounted as well:
2.1 Using YaST to Install a Guest Domain
To install a guest domain with YaST, you need the previously prepared file system
image for the new guest. Start YaST and select Software → Installation into Directory.
The YaST module for directory installation has several options that should be set ac-
cording your needs:
• Target Directory: /var/tmp/dirinstall
Set this option to the mount point of the file system image to use. The default is
• Run YaST and SuSEconfig at First Boot: Yes
Set this option to Yes. You will be asked for a root password and a first user when
starting the guest for the first time.
• Create Image: No
The image this creates is just a tar archive of the installation directory. This is not
Select the type of installation to use. Any of the defaults should be a good start.
Click Next to start the installation. Depending on the number of packages, the installation
takes a while.
Xen uses one of the kernels that are installed in domain-0 to start the guest domain. To
be able to use networking in the guest, the modules of this kernel must be available for
the guest as well.
cp -a /lib/modules/$(uname -r) /var/tmp/dirinstall/lib/modules
To prevent file system errors, the file system image must to be unmounted after the
It would be possible to build specialized kernels for domain-0 on one hand and for the
guest systems on the other hand. The main difference are the hardware drivers that are
unneeded in guest systems. Because these drivers are modular and not used in the guest
systems, SUSE delivers only one kernel for both tasks.
Virtualization with Xen 7
3 Configuring a Xen Guest Domain
The documentation about how to configure a guest domain is not very exhaustive. The
most information about how to configure such a domain can be found in the example
configuration file /etc/xen/vmexample1. The needed options are explained to-
gether with a default value or at least an example configuration. For the installation
described in Section 2.1, “Using YaST to Install a Guest Domain” (page 6), create a
file /etc/xen/guest1 with the following content:
kernel = "/boot/vmlinuz-xen" ❶
ramdisk = "/boot/initrd-xen" ❷
memory = 256 ❸
name = "guest1" ❹
nics =1 ❺
vif = [ 'mac=aa:cc:00:00:00:ab, bridge=xen-br0' ] ❻
disk = [ 'file:/var/tmp/guest1,hda1,w' ] ❼
root = "/dev/hda1" ❽
extra = "3" ❾
❶ Enter the path to the Xen kernel in domain-0. This kernel will run in the guest
❷ Select the appropriate initial RAM disk that contains the device drivers for the
Xen kernel. Without this, the kernel typically panics because it is unable to mount
its root file system.
❸ Define how much memory the guest domain should be given. This fails if the
system does not have enough memory available for its guests.
❹ The name for this guest.
❺ The number of virtual network interfaces for the guest domain.
❻ The configuration of the virtual network interface, including its MAC address and
the bridge to which it is connected.
❼ Set the available virtual block devices for the Xen guest. To use real block devices,
create entries like ['phy:sdb1,hda1,w',
❽ Set the root device for the kernel. This must be the virtual device as seen by the
❾ Add extra kernel parameters here. The 3 in this example means that the guest is
started in runlevel 3.
4 Preparing the Network
The network setup in Domain-0 is done by the xend init script. All the normally used
initialization scripts in /etc/sysconfig/network cannot be used in this setup.
In the above example, the original network settings of your host are used to setup Do-
main-0 in a bridged environment.
To be able to access your system, stop your firewall. Bridging is not supported by
SuSEfirewall2. If you need a firewall, set it up manually.
insserv -r SuSEfirewall2_final
insserv -r SuSEfirewall2_init
insserv -r SuSEfirewall2_setup
5 Starting and Controlling Xen
Before the guest domain may be started, the Xen hypervisor must have enough free
memory for the new guest. First, check the amount of memory used:
memory : 511
free_memory : 172
If this is a 32 bit computer, the Xen hypervisor takes away 64 MB (144 MB for 64 bit)
and Domain-0 occupies the rest. To free some of the memory for the new guest, the
command xm mem-set is used. For a SUSE LINUX Enterprise Server, you need at
least 256 MB for an installation. In the example, if the guest needs 256 MB on a com-
puter with 512 MB, you must set the size of Domain-0 to 202 MB. To set this size,
enter the following as root:
xm mem-set 0 202
In the next xm info, the free_memory should have dropped to 202 MB. Now there
is enough memory available to start a guest with 256 MB. The command xm create
guest1 -c starts the guest and links the console of the starting guest to the current
terminal. If this is the first time that this guest starts, YaST prompts you to complete
Virtualization with Xen 9
5.1 The First Boot
When the guest does its first boot, YaST starts and completes the installation. First,
YaST complains that it did not find a graphics device. Other error messages during first
boot, but they are fixed afterwards automatically. This is normal and may be ignored.
Then you are asked for a root password. After this, a list of settings is shown. The
handling of YaST is a bit more complicated than usual, because you must use the text
interface of YaST and the initial console does not provide colors. Refer to the SUSE
LINUX Enterprise Server manuals for a description of this interface.
At this stage, no network device is detected. However, you may configure the virtual
network device manually. Select Network Devices and change. A configuration dialog
opens, but no network device was detected. Choose to configure an unknown device.
After this, just configure it as if you had a real network device in the virtual guest and
finish the installation.
If you want to use SSH to connect to your guest, you must open the appropriate port
in SuSEfirewall2 or disable the firewall completely. This may also be done after the
installation has been finished with YaST.
5.2 Installation Cleanups
When the installation has finished, you get a login prompt on your terminal. Log in as
user root and do one more fix.
The file /etc/fstab, one of the more important files, is not generated by the instal-
lation. You need to edit this manually:
/dev/hda1 / reiserfs acl,user_xattr 1 1
devpts /dev/pts devpts mode=0620,gid=5 0 0
proc /proc proc defaults 0 0
sysfs /sys sysfs noauto 0 0
On some terminals, the Backspace key does not work properly. Use Ctrl + H in this case.
5.3 Setting Up a Rescue System to Work as
a Guest Domain
The easiest way to get a running system quickly is to reuse an existing root file system,
such as the rescue system of SUSE Linux. Basically, exchange the kernel image and
the device drivers of the virtual block and network devices in this image. To make this
task easier, the script mk-xen-rescue-img.sh is available in /usr/share/
The disadvantage of using the rescue method of constructing a root file system is that
the result does not have an RPM database, so you cannot easily add packages using
RPM. On the positive side, the result is relatively small but has most of what is needed
to get started with networking.
To run the script mk-xen-rescue-img.sh, you need at least the directory with the
rescue image and a destination location for the resulting image. By default, the directory
resides on the boot DVD in the directory /boot.
./mk-xen-rescue-img.sh /media/dvd/boot /usr/local/xen 128
The first parameter of the script is the directory of the rescue image. The second param-
eter is the destination of the image file. Optional parameters are the disk space require-
ments of the newly generated guest domain and the kernel version to use.
The script then copies the image to the new location, replaces the kernel and several
kernel modules. As a last step, it generates a configuration file for the new image in
5.4 The xm Tool
It is always possible to detach a console or reattach it from another terminal. To detach,
use Ctrl + ] . To reattach, first check the ID of the needed guest with xm list and
attach to that ID with xm console ID.
The xm tool of Xen has many possible parameters. View a list with a short explanation
by entering xm help. Table 1, “xm Commands” (page 12) provides some of the most
important commands as a starting point.
Virtualization with Xen 11
Table 1 xm Commands
xm help Print a list of commands that are available for the xm
xm console ID Connect to the first console (tty1) of the guest with ID
xm mem-set ID Mem Set the memory size of the domain with ID ID to Mem
xm create domname Start the domain with configuration file domname. The
[-c] optional -c links the current terminal to the first tty of
the new guest.
xm shutdown ID Do a normal shutdown of the guest with ID ID.
xm destroy ID Terminate the guest with ID ID immediately.
xm list Print a list of all running domains with their respective
ID, memory, and CPU time values.
xm info Display information about the Xen host, including CPU
and memory information.
This section provides some hints about how to solve common problems. It is not meant
as an exhaustive step by step instruction, but should help in getting started how to solve
Networking in Xen3
The concept of networking has changed considerably from Xen2 to Xen3. Domain-
0 is no longer directly connected to the bridge, to prevent blocking of the bridge.
Unfortunately, the initialization scripts of the system cannot handle the current
configuration. To restart the network, run /etc/init.d/xend restart.
My root file system is read-only.
Most likely, you did not add the fstab to the guest. To fix this, shut down your guest
with xm shutdown <Id> then mount it as described in Step 3 (page 6) and
add the file to the mounted system. Do not forget to unmount the file system again
before starting the guest.
I need to do a file system check.
If the file system check did not work automatically, you may do it manually or
from Domain-0. Shut down your guest and run fsck on the image while it is not
mounted. If fsck complains that the file system is mounted, check your mounts
with the command mount.
The guest cannot reach addresses outside Domain-0.
To debug this problem, first make sure that all firewalls involved are disabled.
Then, check if cat /proc/sys/net/ipv4/ip_forward results in a value
of 1. If not, check Section 4, “Preparing the Network” (page 9).
DHCP does not get IP addresses.
DHCP needs several iptables kernel modules to run. Either those have not been
installed as described in Section 2.1, “Using YaST to Install a Guest Domain”
(page 6) or you updated your kernel and forgot to update the kernel modules in
the guest system.
7 For More Information
Find more information about Xen in the following resources:
information for Xen users. It requires the package xen-doc-html.
.html—Some more technical interface documentation. It also requires the package
.html—Xen home page with many different documentation links.
• http://lists.xensource.com/—Several mailing lists about Xen.
Virtualization with Xen 13
• http://wiki.xensource.com/xenwiki—Xen wiki for the Open Source