Motivation
Implementation
Status and future work
Summary
All Virtual All the Time
Implementing x86 Virtualization at the BIOS level
Cristi M gheru³an
1 Technical University of Cluj-Napoca, Romania
2008-2009
Cristi M gheru³an All Virtual All the Time

Motivation
Implementation
Status and future work
Summary
Outline
1 Motivation
Virtualization - a hot topic these days
Virtualization inside the BIOS - Why not?
2 Implementation
AVATT components
Size constraints
Development Toolkit
3 Status and future work
Current Status
Future work
Diculties
Cristi M gheru³an All Virtual All the Time

Motivation
Implementation Virtualization - a hot topic
Status and future work Virtualization inside the BIOS - Why not?
Summary
About Virtualization
Modern hardware is getting faster and faster each year
Multiple OSes can now symultaneously run on the same
machine
Most people use it for server consolidation or testing
There are quite a few virtualization types
Full (QEMU)- emulates a full computer, maybe even another
architecture (usually slow)
Hardware-assisted full virtualization (KVM)- The CPU
provides some special features to make it faster (decent speed)
Paravirtualization (XEN)- the guest OS is aware it's being
virtualized and provides some optimizations (decent speed)
OS virtualization (OpenVZ) - containers, dierent instances of
the same kernel (fastest)
Cristi M gheru³an All Virtual All the Time

Motivation
Implementation Virtualization - a hot topic
Status and future work Virtualization inside the BIOS - Why not?
Summary
Getting closer to the bare hardware I
There is a tendency to move the virtualization software closer
to the bare machine
At rst, they were simple user applications (qemu)
Then, kernel components were added to enhance speed
(KQemu, VMWare, VirtualBox)
Hypervisors replaced the OS that ran on the bare machine
(Xen, VMWare ESX)
Some mainstream OSes got hypervisor capabilities (Linux
KVM)
Most of the Unices are capable of running chroot-ed containers
(chroot, jail, OpenVZ, zones)
Cristi M gheru³an All Virtual All the Time

Motivation
Implementation Virtualization - a hot topic
Status and future work Virtualization inside the BIOS - Why not?
Summary
Getting closer to the bare hardware II
If so, why not move it even closer to the machine, and put it
inside the BIOS? Yeah, it's possible!
The BIOS would start
Make all the needed initializations
Then automatically run the VMs stored on the disk instead of
a single OS
Ideal for server consolidation workloads
Cristi M gheru³an All Virtual All the Time

Motivation AVATT components
Implementation Size constraints
Status and future work Development Toolkit
Summary
The coreboot BIOS makes it possible I
Ordinary BIOSes are fully written in assembly language, prone
to bugs, poorly-designed and quite slow
No keyboard detected, press F1 to continue on 1000 cluster
nodes is not funny ;-)
LinuxBIOS was started back in 1999 by Ron Minnich (then
working on the LANL GRID cluster)
It aims to provide a fully open source BIOS replacement, and
started by using Linux as a BIOS
Had a tiny C code used to initialize the hardware, then ran
kexec() to start the kernel stored on the disk
It's very fast (3 seconds to the Linux payload's prompt)
Written mostly in C, not such bug-prone as ASM
Cristi M gheru³an All Virtual All the Time

Motivation AVATT components
Implementation Size constraints
Status and future work Development Toolkit
Summary
The coreboot BIOS makes it possible II
Ever since it was re-designed, no longer needs to incorporate
Linux and was renamed coreboot soon after
Nowadays it initializes the hardware, and runs an executable
called payload
There are lots of them (even Tetris is available!)
Linux can still be run as a payload
Currently it supports a few dozens of motherboards, but it's
very hard to keep up with the hardware makers
Hardware-compatible with most modern CPUs and chipsets,
but mainboard makers often induce their own modications
Currently supported by some major hardware makers (AMD,
VIA, Sun, Acer, SiS), which often help a lot
Cristi M gheru³an All Virtual All the Time

Motivation AVATT components
Implementation Size constraints
Status and future work Development Toolkit
Summary
So what's inside AVATT?
The coreboot BIOS, ofcourse
A payload that contains the following pieces of software:
A Linux kernel compiled with KVM support, providing
hardware-assisted virtualization
A small Linux userland containing the following
embedded-optimized components:
uClibc - tiny C library made specically for embedded
applications
the busybox binary - provides a basic Linux userland (shell,
coreutils, and so on)
the KVM tools that can be used to create and start virtual
machines
aditional libraries needed by the KVM tools (currently ncurses)
Cristi M gheru³an All Virtual All the Time

Motivation AVATT components
Implementation Size constraints
Status and future work Development Toolkit
Summary
Size constraints
All these, including the kernel, had to t in a 2MB ash image.
I achieved this goal, with 1.8MB used, if compressed with
LZMA(7zip)
Heavily stripped Linux kernel, built with KVM support - 900KB
uClibc - 150KB
busybox - 150KB
KVM tools - 500KB
the aditional libraries - remaining 100KB
Cristi M gheru³an All Virtual All the Time

Motivation AVATT components
Implementation Size constraints
Status and future work Development Toolkit
Summary
Development Toolkit I
For building the AVATT I had to modify the buildrom utility
quite heavily
buildrom is a tool provided by the coreboot project
It can be used to create ROM images ready to be burned on
the mainboard's ash
I modied buildrom by adding a few scripts that compiled the
tools I needed
I created patches that xed the incompatibilities between these
tools
They were stripped down to minimum size and spartan features
The GCC compiler suite was used, version 4.3
Cristi M gheru³an All Virtual All the Time

Motivation AVATT components
Implementation Size constraints
Status and future work Development Toolkit
Summary
Development Toolkit II
For testing I used QEMU, an emulator that can provide the
hardware-assistance CPU instructions
It can use a BIOS image created by buildrom
The developement machine was my laptop running Linux
It was tested and should compile under most modern Linux
distributions like Gentoo, Ubuntu or Debian Lenny
Cristi M gheru³an All Virtual All the Time

Motivation Current Status
Implementation Future work
Status and future work Diculties
Summary
Current Status
All the tools can be built into the same ROM image
The system starts and provides a Linux shell
The disks can be manually mounted
The user can create a VM disk image using a special tool
A Virtual machine can be started from a CDROM image but it
crashes soon during the boot process
The crash is caused by missing Thread Local Storage(TLS)
support from uClibc's threading implementations
None of the three!!! threading implementations provided by
uClibc support TLS on x86 machines
The uClibc team is currently working on this issue, and I may
help them in my spare time
Cristi M gheru³an All Virtual All the Time

Motivation Current Status
Implementation Future work
Status and future work Diculties
Summary
Future Work
Keep the components up to date as they evolve at upstream
After the blocker TLS issue is implemented in uClibc, make it
work at its full potential
Automate as much as possible
User-friendly tool for creating VMs
Until the uClibc thing is getting xed, we could switch to
OpenVZ instead of KVM, providing container-based OS-level
virtualization
OpenVZ should be a lot easier to get compiled than the KVM
tools
I started working on it, but the progress stalled lately due to
lack of spare time
Cristi M gheru³an All Virtual All the Time

Motivation Current Status
Implementation Future work
Status and future work Diculties
Summary
Diculties
The buildrom build system is strange, not a true toolchain
I had to use lots of GCC CFLAGS tricks to get the stu built
A few patches had to be created and applied by buildrom to
most of the programs in order to x compilation errors
Especially the KVM tools were quite hard to make work on top
of uClibc, but the other ones were also problematic
Cristi M gheru³an All Virtual All the Time

Motivation
Implementation
Status and future work
Summary
Summary
The virtualization is getting closer to the hardware
Putting it in the BIOS would be great for server consolidation
Outlook
We need the TLS support from uClibc
OpenVZ worth being considered as a viable alternative to KVM
Cristi M gheru³an All Virtual All the Time

Appendix Acknowledgements
For Further Reading
Acknowledgements
Thanks Google for oering me the opportunity to work on this
as a Summer of Code project
Thanks IBM Romania for appreciation and support
Many thanks to the coreboot team and especially Ron Minnich
for creating coreboot and mentoring me during this great
summer
Cristi M gheru³an All Virtual All the Time

Appendix Acknowledgements
For Further Reading
For Further Reading
Coreboot website http://coreboot.org
AVATT wiki page http://www.coreboot.org/AVATT
Google Summer of Code http://code.google.com/soc
Cristi M gheru³an All Virtual All the Time