kexec / kdump implementation in Linux Kernel and Xen hypervisor
kexec/kdump implementation in Linux Kernel and Xen hypervisor usage, history and current developmentsSponsored by: &
Agenda•What is kexec and kdump ?•History of kexec/kdump development in Linux Kernel and Xen.•Current developments.•Future plans.•Resources.•Questions and Answers ?
What is kexec and kdump ?•kexec and kdump are companions.•kdump uses most of the code from kexec.•kexec allows machine restart without jumping into BIOS.•kdump eases kernel debugging when running system was crashed.
What is kexec ?•kexec allows machine restart without jumping into BIOS.•It greatly speeds up system initialization.•kexec does not care about current system state.•It is role of administrator to prepare system for restart.•BIOS does not initialize the devices.•Can lead to unpredictable state when devices were not properlyshutdown.•Load specified kernel and other modules (if user requires it) into memory(kexec -l|--load <kernel-image> …).•Jump into initialization code which finally callsnew kernel (kexec -e|--exec …).
What is kdump ?•kdump eases kernel debugging when running system was crashed.•Requires reserved memory (say 128 MB) that cannot be touched.•Ignores the current state of the machine - just tries to boot.•Boot system with crashkernel option (Linux Kernel orXen hypervisor).•Load specified crash kernel into previously reserved memory bycrashkernel option (kexec -p|--load-panic <kernel-image> …).•… the system will reboot into the dump-capture kernel if a system crash istriggered. Trigger points are located in panic(), die(), die_nmi() and in thesysrq handler (ALT-SysRq-c). … (linux/Documentation/kdump/kdump.txt).
What is kdump ?•New system running under dump-capture kernel expose raw image ofmemory of crashed system through /dev/oldmem device interface.•Through /proc/vmcore ELF format image of memory of crashed system isavailable.•ELF format image is useful for GDB and Crash tool.•… Before analyzing the dump image, you should reboot into a stable kernel. …(linux/Documentation/kdump/kdump.txt).•kexec and kdump are excellent tools, albeit they have some shortcomings.
What is kexec and kdump ?kexec/kdump infrastructure: •kexec-tools - userspace tools. •kexec/kdump implementation in Linux Kernel. •kexec/kdump implementation in Xen.
Details of kexec load phase•kexec -l|--load <kernel-image> …•Compute a hash (SHA 256) of the loaded kernel.•kexec calls sys_kexec_load().•sys_kexec_load() loads kernel into memory and establishesidentity page tables for the kexec kernel.•On Xen hypervisor dom0 kernel callsHYPERVISOR_kexec_op(KEXEC_CMD_kexec_load, …)hypercall which loads kernel into memory on behalf of dom0 kernel.
Details of kexec execution phase•kexec -e|--exec …•kexec calls sys_reboot() with LINUX_REBOOT_CMD_KEXEC cmdargument which later calls kernel_kexec(),•kernel_kexec() calls machine_kexec() which preparearguments for relocate_kernel() and calls it.•On Xen hypervisor dom0 kernel does not call relocate_kernel()directly. First of all it callsHYPERVISOR_kexec_op(KEXEC_CMD_kexec, …) hypercall whichlater calls relocate_kernel().
Details of kexec execution phase•relocate_kernel() switch to identity page tables, relocatekernel and then jump into purgatory code.•purgatory code do some arch specific setup (vga reset, PIC init,backup first 640K of memory on x86 platform if it is panickernel …).•Verify a hash (SHA 256) of the loaded kernel established at loadphase and jump into kexec kernel.•New kernel executes as usual.
History of kexec/kdump development in Linux Kernel and Xen•kexec/kdump were introduced in Linux Kernel Ver. 2.6.13.•Currently (Linux Kernel Ver. 3.0) kexec/kdump support is available oneight platforms (ARM, IA-64, MIPS, PowerPC, S/390, SH, Tile, x86).•kexec/kdump in Xen is available since Ver. 3.0.4.•kexec/kdump under Xen dom0 is not available with paravirt kernel,but it is with old style Xenlinux patches.
Current developments•kexec/kdump implementation in Linux Kernel is quite stable.•kexec/kdump implementation in Xen is quite stable too.•Xenlinux kernels receiving only fixes (if any).•kexec/kdump implementation for pvops Linux Kernel is underdevelopment now.•This work is sponsored by Google under Google Summer ofCode 2011 program.
Current developments•Development is based off 2.6.32 Novell SLES11 Xen kexec/kdumpchanges.•Initial implementation patch (only for dom0) will be posted shortlyon Xen-devel and LKML.•It requires a lot of cleanups.•Currently there are more implementation questions than technicalones.•kexec-tools requires some modification also (will be posted shortlyon email@example.com, Xen-devel and LKML).
Future plans•kexec/kdump support for PV guest domains.•kexec/kdump support for PVonHVM guest domains (some work isdone by Olaf Hering).•kexec/kdump support for HVM guest domains - it works, however,some cleanups would be required.
Future plansAutomatic Xen/dom0 kernel crash recovery: •Xen or dom0 kernel crashing (guest domains were running). •New incarnation of Xen hypervisor and dom0 kernel is started. •Guest domains are restarted from point at which they were running when Xen/dom0 kernel crash occurred.
Resources•http://www.kernel.org/ - The Linux Kernel Archives•ftp://ftp.kernel.org/pub/linux/utils/kernel/kexec/ - kexec-tools src•git://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git -kexec-tools development tree•kexec man page•Linux Kernel source: linux/Documentation/kdump/•http://www.xen.org/products/xenhyp.html - Xen Hypervisor•http://xenbits.xen.org/ - Xen Source Repositories•http://people.redhat.com/~anderson/ - Crash tool