These slides are from a talk presented at the Docker Athens meetup on Thursday, May 31, 2018. They start by covering the evolution of the Docker engine of 2014/2015 into the separate components of OCI runc, (now) CNCF containerd, and the Docker client and daemon projects. Finally, various use cases for the CNCF containerd "core container runtime" project are detailed, from the Docker engine itself to serverless frameworks like OpenWhisk, to the container runtime interface (CRI) within Kubernetes.

1. Docker Athens:
Docker Engine Evolution
& Containerd Use Cases

2. Hello!
Phil Estes
> Distinguished Engineer
Office of the CTO
IBM Watson & Cloud Platform
> Docker Captain
> Containerd and Moby Project
maintainer
2

3. 1.
The Docker
Engine
“Established 2013.”

4. @estesp
Single statically-linked
binary comprised:
> Client
> Daemon
> Build tool
> Registry client
4
$ docker run ubuntu
$ docker --daemon
$ docker build -t estesp/myimg .
$ docker push estesp/myimg
$ docker pull estesp/myimg
(circa 2013-2015)

5. @estesp
{Single statically-linked
binary comprised:
> Client
> Daemon
> Build tool
> Registry client
HTTP/JSON
Docker API
gRPC
API
5

6. 2.
The OCI & runc
Container execution grows up
into a standard

7. @estesp
& runC
> Announced June 20th, 2015
> Charter signed on
December 8th, 2015
> 44 member companies
> Both specifications
reached 1.0 June 2017
https://opencontainers.org
https://github.com/opencontainers
> runc is a client wrapper around libcontainer
> libcontainer is the OS level interface for containers
> OCI spec covers Solaris, Linux, & MS Windows
$ docker run -it --read-only
-v /host:/hostpath
alpine sh
/#
{
"ociVersion": "1.0.0",
"platform": {
"os": "linux",
"arch": "amd64"
},
"process": {
"terminal": true,
"args": [
"sh"
],
"env": [
"PATH=/usr/sbin:/usr/local/bin:/bin”
config.json
• A Linux Foundation Collaborative Project
• Free from control by any particular vendor’s specific cloud stack or ecosystem
• Includes a specification, reference runtime* and now, a specified image format
*seeded with runc + libcontainer by Docker
7

8. @estesp
runC
Created in June 2015
> 16 releases (1.0.0-rc5 underway)
> 215 contributors
> OCI maintained/governance
> Used by Docker, containerd,
garden-runc/Guardian, many others
▪ Runc is a client wrapper around the pre-existing libcontainer
library project
▪ Runc is one implementation of the OCI runtime specification
▪ Scope of runc is clearly limited by OCI charter: no networking,
image handling/resolution, storage support
▪ Enablement of low-level OS features happen here: ambient
caps, rootless containers, new cgroup support, and so on
▪ Daemon-less operation; wrapping code must handle any
broader node and cluster level container mgmt.
8

9. 3.
containerd
A boring base container runtime,
contributed by Docker to the
CNCF

10. @estesp
Created in December 2015
> 35 releases (1.1.0 currently)
> 127 contributors
> Docker created; now a CNCF project
> Used by Docker, K8s CRI; Cloud Foundry,
OpenWhisk (serverless), LinuxKit, BuildKit
▪ Launched December 2015 (used in Docker early 2016)
▪ Two streams of activity:
□ “0.2.x” branch: used in former Docker releases as a
simple runc manager (up until 17.11)
□ “1.0.0” branch: based on the December 2016
announcement, contributed to CNCF
▪ Executes containers using the OCI runc executor;
containerd manages state/metadata, image & registry
interactions, snapshot drivers (overlay, btrfs, others)
▪ Supports Linux on several architectures; Windows support
in 1.2 10

11. @estesp
runc
containerd
Why Containerd 1.0?
▪ Continue projects spun out
from monolithic Docker
engine
▪ Expected use beyond Docker
engine (Kubernetes CRI)
▪ Donation to foundation for
broad industry collaboration
□ Similar to runc/libcontainer
and the OCI

12. @estesp
Metadata Content Snapshotter
Runtime
Linux (shim)
OCI runC
IMAGE TASK CONTAINER
Client library (Golang)gRPC
Service
APIs
Vendor client library to embed containerd{ or }
▪ Metrics API &
Prometheus support
▪ OCI runtime and
image support
▪ Clean API and
abstractions
▪ Pluggable runtime
support (used by
VMWare impl.)
▪ Namespace support
(administrative/soft
multi-tenancy)
12

13. @estesp
Example: Pull an Image
Via ctr client:
$ export
CONTAINERD_NAMESPACE=example
$ ctr pull
docker.io/library/redis:alpine
$ ctr image ls
...
import (
"context"
"github.com/containerd/containerd"
"github.com/containerd/containerd/namespaces"
)
// connect to our containerd daemon
client, err := containerd.New("/run/containerd/containerd.sock")
defer client.Close()
// set our namespace to “example”:
ctx := namespaces.WithNamespace(context.Background(), "example")
// pull the alpine-based redis image from DockerHub:
image, err := client.Pull(ctx,
"docker.io/library/redis:alpine",
containerd.WithPullUnpack)

14. @estesp
Example: Run a Container
Via ctr client:
$ export
CONTAINERD_NAMESPACE=example
$ ctr run -t
docker.io/library/redis:alpine
redis-server
$ ctr c ls
...
// create our container object and config
container, err := client.NewContainer(ctx,
"redis-server",
containerd.WithImage(image),
containerd.WithNewSpec(containerd.WithImageConfig(image)),
)
defer container.Delete()
// create a task from the container
task, err := container.NewTask(ctx, containerd.Stdio)
defer task.Delete(ctx)
// make sure we wait before calling start
exitStatusC, err := task.Wait(ctx)
// call start on the task to execute the redis server
if err := task.Start(ctx); err != nil {
return err
}

15. @estesp
Example: Kill a Task
Via ctr client:
$ export
CONTAINERD_NAMESPACE=example
$ ctr t kill redis-server
$ ctr t ls
...
// make sure we wait before calling start
exitStatusC, err := task.Wait(ctx)
time.Sleep(3 * time.Second)
if err := task.Kill(ctx, syscall.SIGTERM); err != nil {
return err
}
// retrieve the process exit status from the channel
status := <-exitStatusC
code, exitedAt, err := status.Result()
if err != nil {
return err
}
// print out the exit code from the process
fmt.Printf("redis-server exited with status: %dn", code)

16. @estesp
Example: Customize OCI Configuration
// WithHtop configures a container to monitor the host via `htop`
func WithHtop(s *specs.Spec) error {
// make sure we are in the host pid namespace
if err := containerd.WithHostNamespace(specs.PIDNamespace)(s); err != nil {
return err
}
// make sure we set htop as our arg
s.Process.Args = []string{"htop"}
// make sure we have a tty set for htop
if err := containerd.WithTTY(s); err != nil {
return err
}
return nil
}
With{func} functions cleanly separate modifiers

17. 4.
Use Cases
Putting it all together.

18. @estesp
{
HTTP/JSON
Docker API
gRPC
API
18
http://github.com/docker/cli
http://github.com/moby/moby
http://github.com/containerd/containerd
http://github.com/opencontainers/runc
http://github.com/docker/docker-ce

19. @estesp
▪ LinuxKit is a minimal, secure Linux OS
image creator/toolkit
▪ LinuxKit uses containerd as the core
container runtime for running system
services (distro images are a kernel +
small init; all else runs as container
processes)
19
https://github.com/linuxkit/linuxkit/

20. @estesp
BuildKit
▪ Introduction:
https://blog.mobyproject.org/introducing-buildkit-17e056cc5317
▪ BuildKit uses containerd snapshot,
execution and image libraries
▪ Allows building to depend on a local
containerd daemon and shares storage
with other clients
▪ BuildKit merging into Moby project soon
20
https://github.com/moby/buildkit

21. @estesp
▪ CloudFoundry container execution layer
initially built around OCI spec + runC
executor
▪ CF guardian project wrote management
code similar to containerd around runC
▪ Currently working on a branch which
removes their custom runC management
layer and uses containerd directly
21
https://github.com/cloudfoundry/guardian/tree/containerd-spike

22. @estesp
▪ Open source FaaS project created by IBM
▪ Basis of IBM Cloud Functions offering
▪ Uses containers as the native execution
unit for functions
▪ Built using Docker engine for execution
▪ Testing use of containerd instead of full
Docker engine
22
https://github.com/apache/incubator-openwhisk
Apache OpenWhisk

23. @estesp
Kubernetes Orchestrator
▪ Kubernetes has no code to execute or run
containers on Linux or Windows
▪ Initially the Kubernetes pod manager
(called “kubelet”) had direct linkage to the
Docker engine
23
kubelet dockershim dockerd
containerd
runc
https://github.com/kubernetes/kubernetes/tree/release-1.4/pkg/kubelet/dockershim

24. @estesp
kubelet
kubelet
dockershim (CRI)
Docker engine
containerd
containerd-shim
containerd-shim
containerd-shim
runc
runc
runc
containerd
containerd-shim
containerd-shim
containerd-shim
runc
runc
runc
cri plugin
containerd
cri-containerd
ttrpc: very lightweight
gRPC protocol format
Kubernetes CRI Runtimes:
Docker vs. cri-containerd
( **NOTE: Cri-container project merged into containerd
GitHub project in January 2018; now a plugin within
the containerd binary )
**
24

25. @estesp
25

26. @estesp
Containerd Benefits
● Designed and implemented with broad
usage as a core container runtime in mind:
○ Docker, LinuxKit, Kubernetes and
embedded core runtime use cases
(OpenWhisk, Cloud Foundry)
● Stress testing validating stability and
performance guarantees 24/7
● Attention to detail re: Go/gRPC APIs for
usability and ease of embedding
● Focus on compatibility guarantees; bug
fix backports for high level of support on
major version levels

27. @estesp
Going further with containerd
▪ Contributing:
https://github.com/containerd/containerd
□ Bug fixes, adding tests, improving docs, validation
▪ Using: See the getting started documentation in the
docs folder of the repo
▪ Porting/testing: Other architectures & OSs, stress
testing (see bucketbench, containerd-stress):
□ git clone <repo>, make binaries, sudo make install
▪ K8s CRI: implementation of K8s CRI using containerd
□ CRI project is now a plugin to the main containerd project.
Similar needs for interested contributors, testing, etc.

28. 28
Thanks!
@estesp
github.com/estesp
estesp@gmail.com
https://integratedcode.us
Slack/IRC: estesp