LinuxKit Deep Dive
Justin Cormack
Docker
Rolf Neugebauer
Docker
GH: justincormack
TW: @justincormack
GH: rn
TW: @neugebar
What is LinuxKit
A toolkit for building secure, portable and lean
operating systems for containers
● Uses moby tooling to build system images
● Everything is run in a container
● Running with containerd 1.0 branch for
over four months
● lightweight, fully customisable
LinuxKit architecture
Modern Linux kernel
Minimal init
containerD
Service
containers
on-boot
containers
shutdown
containers
kernel:
image: linuxkit/kernel:4.9.54
init:
- linuxkit/init:98e95fb67e8afcf02c09ba927e4b357fec42977a
- linuxkit/runc:991ef358ad8fc1111d64f4d8071f2009cc561f6a
- linuxkit/containerd:eaf0d615cfceb9d854408dd3c80429ee8ac4d051
onboot:
- name: dhcpcd
image: linuxkit/dhcpcd:aa685261ceb2557990dcfe9dd8824c6b9ec416e2
command: ["/sbin/dhcpcd", "--nobackground", "-f", "/dhcpcd.conf", "-1"]
services:
- name: getty
image: linuxkit/getty:7abaf7b276c59f80891d92e9279e3e3ee8e2f512
env:
- INSECURE=true
● YAML files describes complete system
● moby tool assembles boot image &
verifies signatures
● linuxkit tool for testing/running
Some metrics
●
~ 1400 commits since open sourcing (April 2017)
Active community on #linuxkit community slack
80 contributors, 52 external!
Thank you!
Major additions
● arm64 support
● Improved Kubernetes support
● Linux Containers on Windows (LCOW) preview
● Improved platform support:
○ packet.net, Azure, AWS, GCP,
VMware, Vultr, IBM Bluemix, Hyper-V, …
○ Mainly community additions!
Lots of “small” improvements
● Multi-arch build system
● Fully immutable system images
● Flexible networking setup + WireGuard
● Namespace sharing
● Persistent disk support
● TPM support
● Kernels 4.9, 4.10, 4.11, 4.12, 4.13 (4.14 soon)
Demo
Multi-arch buildsystem
LinuxKit packages
● Small container images on hub
● Pulled and assembled using the moby tool
● Signed with notary
● Multi-arch images (x86_64 and arm64)
Dhcpcd package
● All core LinuxKit packages are build from
linuxkit/alpine
LinuxKit “base” image
● linuxkit/alpine contains a apk mirror of Alpine
packages needed to build LinuxKit packages
○ Compilers, tools, and packages
○ Recorded exact versions of packages
○ Stashed on docker hub and signed
● docker build --network=none
● Repeatable builds!
Multi-stage builds
(introduced in docker 17.05.0-ce)
Multi-stage builds (cont)
● Install in /out and then copy to scratch
● Less than 2MB in size (Thanks to Alpine!)
What is run
● Default Entry point
Container details
● Default OCI runtime configuration snippets
● E.g., dhcpcd needs CAP_NET_ADMIN
Package build
● Additional labels for the repository and the git
revision this was build from
Package tags
● Hash of all source files (git tree hash)
● The same on all architectures
Push to hub
● Pushed to hub with Content trust
(https://blog.mobyproject.org/sign-all-the-things-c12c2182d9f6)
Multi-arch build
● Create a manifest list linuxkit/dhcpcd:<hash>
● Points to linuxkit/dhcpcd:<hash>-amd64 and
linuxkit/dhcpcd:<hash>-arm64
Sign the manifest
● This soon will be a lot easier with:
docker manifest and docker trust
Summary
Repeatable, cross-platform builds from base
image to full system image with content trust
throughout
linuxkit/alpine Packages
YAML
System Image
Dockerfile
Custom network config
What is WireGuard
● fast, modern, secure VPN
● included in the LinuxKit kernels
● On the way to being upstreamed in Linux
● just appears as a network interface
Service namespaces
● can create namespaces and share between
system services
● can create network interfaces in desired
namespaces or move them
● restrict network access in system containers
WireGuard example
● secure a redis service with a VPN tunnel
● only access is one end of an encrypted
network tunnel
● no access to external network
● can only send encrypted traffic
WireGuard example
WireGuard example
WireGuard demo
Building Kubernetes
Kubernetes update
● continual progress on the Kubernetes work
since the original launch
● supports Kubernetes with Docker or
cri-containerd
● will ship in Docker for Mac and Windows
● will support production deployments
with InfraKit in future
Kubernetes setup in brief
● uses KubeAdm to configure Kubernetes
● runs Kubelet in a system container but
sharing parts of filesystem
● immutable infrastructure
● supports multiple nodes
Kubernetes deep dive
The end
● Tycho: Container-relevant Upstream Kernel Developments (after this talk)
● Phil & Michael: Docker Multi-arch All The Things (in parallel to this talk)
● John: Linux Containers on Windows: The Inside Story (Wed am)
● Natanael: Small, Simple, and Secure: Alpine Linux (Wed lunch)
● Qualcomm booth in Exhibit hall (arm64 demos)
● Moby Summit (Thursday):
○ Andrew: LinuxKit on ARM
○ Sven: RancherOS and LinuxKit
○ Stephen & Phil: containerd presentation
GH: justincormack GH: rn
TW: @justincormack TW: @neugebar
The end
● Tycho: Container-relevant Upstream Kernel Developments (after this talk)
● Phil & Michael: Docker Multi-arch All The Things (in parallel to this talk)
● John: Linux Containers on Windows: The Inside Story (Wed am)
● Natanael: Small, Simple, and Secure: Alpine Linux (Wed lunch)
● Moby Summit (Thursday):
○ Andrew: LinuxKit on ARM
○ Sven: RancherOS and LinuxKit
GH: justincormack GH: rn
TW: @justincormack TW: @neugebar

LinuxKit Deep Dive

  • 1.
    LinuxKit Deep Dive JustinCormack Docker Rolf Neugebauer Docker GH: justincormack TW: @justincormack GH: rn TW: @neugebar
  • 2.
    What is LinuxKit Atoolkit for building secure, portable and lean operating systems for containers ● Uses moby tooling to build system images ● Everything is run in a container ● Running with containerd 1.0 branch for over four months ● lightweight, fully customisable
  • 3.
    LinuxKit architecture Modern Linuxkernel Minimal init containerD Service containers on-boot containers shutdown containers kernel: image: linuxkit/kernel:4.9.54 init: - linuxkit/init:98e95fb67e8afcf02c09ba927e4b357fec42977a - linuxkit/runc:991ef358ad8fc1111d64f4d8071f2009cc561f6a - linuxkit/containerd:eaf0d615cfceb9d854408dd3c80429ee8ac4d051 onboot: - name: dhcpcd image: linuxkit/dhcpcd:aa685261ceb2557990dcfe9dd8824c6b9ec416e2 command: ["/sbin/dhcpcd", "--nobackground", "-f", "/dhcpcd.conf", "-1"] services: - name: getty image: linuxkit/getty:7abaf7b276c59f80891d92e9279e3e3ee8e2f512 env: - INSECURE=true ● YAML files describes complete system ● moby tool assembles boot image & verifies signatures ● linuxkit tool for testing/running
  • 4.
    Some metrics ● ~ 1400commits since open sourcing (April 2017) Active community on #linuxkit community slack 80 contributors, 52 external! Thank you!
  • 5.
    Major additions ● arm64support ● Improved Kubernetes support ● Linux Containers on Windows (LCOW) preview ● Improved platform support: ○ packet.net, Azure, AWS, GCP, VMware, Vultr, IBM Bluemix, Hyper-V, … ○ Mainly community additions!
  • 6.
    Lots of “small”improvements ● Multi-arch build system ● Fully immutable system images ● Flexible networking setup + WireGuard ● Namespace sharing ● Persistent disk support ● TPM support ● Kernels 4.9, 4.10, 4.11, 4.12, 4.13 (4.14 soon)
  • 7.
  • 8.
  • 9.
    LinuxKit packages ● Smallcontainer images on hub ● Pulled and assembled using the moby tool ● Signed with notary ● Multi-arch images (x86_64 and arm64)
  • 10.
    Dhcpcd package ● Allcore LinuxKit packages are build from linuxkit/alpine
  • 11.
    LinuxKit “base” image ●linuxkit/alpine contains a apk mirror of Alpine packages needed to build LinuxKit packages ○ Compilers, tools, and packages ○ Recorded exact versions of packages ○ Stashed on docker hub and signed ● docker build --network=none ● Repeatable builds!
  • 12.
  • 13.
    Multi-stage builds (cont) ●Install in /out and then copy to scratch ● Less than 2MB in size (Thanks to Alpine!)
  • 14.
    What is run ●Default Entry point
  • 15.
    Container details ● DefaultOCI runtime configuration snippets ● E.g., dhcpcd needs CAP_NET_ADMIN
  • 16.
    Package build ● Additionallabels for the repository and the git revision this was build from
  • 17.
    Package tags ● Hashof all source files (git tree hash) ● The same on all architectures
  • 18.
    Push to hub ●Pushed to hub with Content trust (https://blog.mobyproject.org/sign-all-the-things-c12c2182d9f6)
  • 19.
    Multi-arch build ● Createa manifest list linuxkit/dhcpcd:<hash> ● Points to linuxkit/dhcpcd:<hash>-amd64 and linuxkit/dhcpcd:<hash>-arm64
  • 20.
    Sign the manifest ●This soon will be a lot easier with: docker manifest and docker trust
  • 21.
    Summary Repeatable, cross-platform buildsfrom base image to full system image with content trust throughout linuxkit/alpine Packages YAML System Image Dockerfile
  • 22.
  • 23.
    What is WireGuard ●fast, modern, secure VPN ● included in the LinuxKit kernels ● On the way to being upstreamed in Linux ● just appears as a network interface
  • 24.
    Service namespaces ● cancreate namespaces and share between system services ● can create network interfaces in desired namespaces or move them ● restrict network access in system containers
  • 25.
    WireGuard example ● securea redis service with a VPN tunnel ● only access is one end of an encrypted network tunnel ● no access to external network ● can only send encrypted traffic
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
    Kubernetes update ● continualprogress on the Kubernetes work since the original launch ● supports Kubernetes with Docker or cri-containerd ● will ship in Docker for Mac and Windows ● will support production deployments with InfraKit in future
  • 31.
    Kubernetes setup inbrief ● uses KubeAdm to configure Kubernetes ● runs Kubelet in a system container but sharing parts of filesystem ● immutable infrastructure ● supports multiple nodes
  • 32.
  • 33.
    The end ● Tycho:Container-relevant Upstream Kernel Developments (after this talk) ● Phil & Michael: Docker Multi-arch All The Things (in parallel to this talk) ● John: Linux Containers on Windows: The Inside Story (Wed am) ● Natanael: Small, Simple, and Secure: Alpine Linux (Wed lunch) ● Qualcomm booth in Exhibit hall (arm64 demos) ● Moby Summit (Thursday): ○ Andrew: LinuxKit on ARM ○ Sven: RancherOS and LinuxKit ○ Stephen & Phil: containerd presentation GH: justincormack GH: rn TW: @justincormack TW: @neugebar
  • 34.
    The end ● Tycho:Container-relevant Upstream Kernel Developments (after this talk) ● Phil & Michael: Docker Multi-arch All The Things (in parallel to this talk) ● John: Linux Containers on Windows: The Inside Story (Wed am) ● Natanael: Small, Simple, and Secure: Alpine Linux (Wed lunch) ● Moby Summit (Thursday): ○ Andrew: LinuxKit on ARM ○ Sven: RancherOS and LinuxKit GH: justincormack GH: rn TW: @justincormack TW: @neugebar