openSUSE Conference
Cluster deployment
with Warewulf 4.3.0
Christian Goll
cgoll@suse.com
June 2, 2022

Overview
What is warewulf4
A simple, scalable, stateless and very flexible operating
system provisioning system
Features
I Lightweight: nodes are stateless
I Agnostic: node images are containers
I Flexible: go templates for configs
I Open Source (BSD-3-Clause)

History
Warewulf 3
I successful and widely used (openHPC)
I overhead due to different storage back ends
I implemented in perl
I perl versions may differ
I transported its own busybox
I licensing issues
I proprietary image creation
I hard to maintain for every distro
Who is developing Warewulf 4
I HPCng
I Apptainer (aka singularity)
I CTRIQ/CIQ
I CentOS/Rocky

Warewulf 4.3.0 core components
Mandatory
I warewulfd serves containers/overlays
I tftp starts up iPXE
I dhcpd needed for basic networking
I central dhcpd is also possible
I wwctl command line tool for management
I edit of nodes.conf possible (plain yaml)
Optional
I ssh-key distribution
I nfs-server on warewulf4 host

OS Container
Definition
A container is the base operation system for a node. It is completely
independent from the operating system of the server running the
warewulf 4 services.
The boot image is derived from this container.
How to get a container
I download from any container registry
registry.opensuse.org/science/warewulf
I use docker,buildah. . . to build container
I use apptainer. . . to build chroot
How to modify a container
I only package level modifications recommended
I shell into container and so stuff
wwctl container shell CONTAINERNAME

Overlay
Definition
An overlay is a individual cpio archive for every node. This archive is
simply extracted on the node. Node specific files are generated with go
templates. Templates must have the .ww suffix.
SystemOverlay
I available at boot time
I not updated at runtime
I should e.g. contain network configuration
RuntimeOverlay
I updated at regular intervals
I pulled by nodes with wwclient
I should contain e.g. munge key, slurm.conf

Overlay
HostOverlay
I overlays on the host running warewulf are possible
I manages /etc/exports, /etc/hosts, /etc/dhcpd.conf
Template Example: issue.ww
Warewulf Node: {{. Id}}
Container: {{. Container }}
Kernel: {{. Kernel.Version }}
Kernelargs : {{. Kernel.Args }}
Network:
{{- range $devname , $netdev := .NetDevs }}
{{ $devname }}: {{ $netdev.Device }}
{{ $devname }}: {{ $netdev.IpCIDR }}
{{if $netdev.Ipaddr6 }} {{ $devname }}: {{ $netdev.Ipaddr6 }}{{ end -}}
{{if $netdev.Hwaddr }} {{ $devname }}: {{ $netdev.Hwaddr }}{{ end -}}
{{end}}

Warewulf configuration
Data model
Warewulf has an abstract data model for nodes and collection of nodes
called profiles. Overlays, Kernel and Containers are only uncompressed
chroots under /var/lib/warewulf/[overlays|chroots].
node/profile configuration options
I container
I SystemOverlays and RuntimeOverlays
I kernel, kernel options (optional)
I network device configuration
I IPMI configuration
I arbitrary key value pairs

Warewulf configuration
Data inheritance
For every node data item its value is determined in following way:
I take node specific value, if set
I take profile specific value, if no node specific value set
I take default value, if no node/profile specific value set
Example fort KernelArgs
I default: quiet crashkernel=no vga=791
I profile: vga=791
I node: noacpi

Node listing example
# ###############################################################################
NODE FIELD PROFILE VALUE
node01 Id -- node01
node01 Comment default This profile is automatically included for each node
node01 Cluster -- --
node01 Profiles -- default
node01 Discoverable -- true
node01 Container default tw
node01 KernelOverride SUPERSEDED leap15 .3
node01 KernelArgs -- (quiet crashkernel=no vga =791)
node01 SystemOverlay -- (wwinit)
node01 RuntimeOverlay -- (generic)
node01 Ipxe -- (default)
node01 Init -- (/ sbin/init)
node01 Root -- (initramfs)
node01 AssetKey -- --
node01 IpmiIpaddr -- 10.10.10.10
node01 IpmiNetmask default 255.255.255.0
node01 IpmiPort -- --
node01 IpmiGateway -- --
node01 IpmiUserName -- admin
node01 IpmiInterface -- --
node01 IpmiWrite -- --
node01 default:DEVICE -- eth0
node01 default:HWADDR -- 52:54:00:08: a7:c0
node01 default:IPADDR -- 10.10.18.11
node01 default:IPADDR6 -- --
node01 default:NETMASK -- (255.255.255.0)
node01 default:GATEWAY -- --
node01 default:TYPE -- --
node01 default:ONBOOT -- true
node01 default:DEFAULT -- true

Node boot process
PXE and UEFI boot process
I node get iPXE binary per tftp (shipped with warewulf)
I iPXE boots, gets dynamic IP address
I iPXE fetches kernel per http from warewulfd
I iPXE fetches kmods, container, SystemOverlay builds magic single
image
I iPXE boots kernel with magic single image
I kernel calls init, provided in the SystemOverlay
I SysV like init configures basic stuff like, IPMI
I SysV like init calls systemd/init of the container
I container OS boots
I wwclient downloads RuntimeOverlay

Future
Planned features
I gRPC interface
I JSON/Rest interface
Possible features
I simple disk management
I persistent node install
I load balancing
Contribution
I https://github.com/hpcng/warewulf
I HPCng slack channel
I (Bi)weekly meeting Wednesday evening 21:00 (CET)

Thank you for your attention.