Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Docker London Meetup: Docker Engine Evolution


Published on

A meetup talk on the evolution of the Docker engine from 2014-2019, including the refactoring and spin out of OCI runc and CNCF containerd codebases. This talk was given at the Docker London meetup group on Thursday, 31st January, 2019.

Published in: Software
  • Login to see the comments

  • Be the first to like this

Docker London Meetup: Docker Engine Evolution

  1. 1. Docker Evolution From a Monolith to Discrete Components Phil Estes Distinguished Engineer & CTO, Linux & Container Strategy IBM Cloud Platform
  2. 2. Hello! Phil Estes > Distinguished Engineer @ IBM Cloud > Docker Captain > Containerd and Moby Project maintainer > Moby Technical Steering Committee & OCI Technical Oversight Board member 2
  3. 3. Containers. 3 Sea Containers? O “Sea” I Containers!
  4. 4. 1. The Docker Engine “Established 2013.”
  5. 5. Single statically-linked binary comprised: > Client > Daemon > Build tool > Registry client 5 $ docker run ubuntu $ docker --daemon $ docker build -t estesp/myimg . $ docker push estesp/myimg $ docker pull estesp/myimg (circa 2013-2015)
  6. 6. {Single statically-linked binary comprised: > Client > Daemon > Build tool > Registry client 6 (circa 2016-today)
  7. 7. 2. The OCI & runc Container execution grows up into a standard
  8. 8. & runC > Announced June 20th, 2015 > Charter signed on December 8th, 2015 > 44 member companies > Both specifications reached 1.0 June 2017 > runc is a client wrapper around libcontainer > libcontainer is the OS level interface for containers > OCI spec covers Solaris, Linux, & MS Windows $ docker run -it --read-only -v /host:/hostpath alpine sh /# { "ociVersion": "1.0.0", "platform": { "os": "linux", "arch": "amd64" }, "process": { "terminal": true, "args": [ "sh" ], "env": [ "PATH=/usr/sbin:/usr/local/bin:/bin” config.json • A Linux Foundation Collaborative Project • Free from control by any particular vendor’s specific cloud stack or ecosystem • Includes a specification, reference runtime* and now, a specified image format *seeded with runc + libcontainer by Docker 8
  9. 9. runC Created in June 2015 > 17 releases (1.0.0-rc5 currently) > 243 contributors > OCI maintained/governance > Used by Docker, containerd, cri-o, garden-runc/Guardian, BuildKit, etc. ▪ Runc is a client wrapper around the pre-existing libcontainer library project ▪ Runc is one implementation of the OCI runtime specification ▪ Scope of runc is clearly limited by OCI charter: no networking, image handling/resolution, storage support ▪ Enablement of low-level OS features happen here: ambient caps, rootless containers, new cgroup support, and so on ▪ Daemon-less operation; wrapping code must handle any broader node and cluster level container mgmt. 9
  10. 10. 3. containerd A “boring” base container runtime, contributed to the CNCF
  11. 11. runc containerd Why Containerd 1.0? ▪ Continue projects spun out from monolithic Docker engine ▪ Expected use beyond Docker engine (Kubernetes CRI) ▪ Donation to foundation for broad industry collaboration ▫ Similar to runc/libcontainer and the OCI
  12. 12. Created in December 2015 > 57 releases (1.2.2 currently) > 165 contributors > Docker created; now a CNCF project > Used by Docker, Kubernetes, Cloud Foundry, BuildKit, LinuxKit, and others ▪ Launched initially in December 2015 (included in Docker 1Q2016) ▪ Two streams of activity: ▫ “0.2.x” branch: used in former Docker releases as a simple runc manager (up until 17.11) ▫ “1.0.0” branch: based on the December 2016 announcement, contributed to CNCF ▪ Executes containers using the OCI runc executor; containerd manages state/metadata, image & registry interactions, snapshot drivers (overlay, btrfs, others) ▪ Supports Linux on several architectures; Windows support in 1.2.x 12
  13. 13. Metadata Content Snapshotter Runtime Linux (shim) OCI runC IMAGE TASK CONTAINER Client library (Golang) { or } ▪ Metrics API & Prometheus support ▪ OCI runtime and image support ▪ Clean API and abstractions ▪ Pluggable runtime support (used by VMWare impl.) ▪ Namespace support (administrative/soft multi-tenancy) 13
  14. 14. 4. Docker today The open source-based product.
  15. 15. { 15
  16. 16. 16 Docker Engine (Community) Docker Desktop Enterprise Docker Enterprise
  17. 17. 5. Using the Layers Other models of reuse
  18. 18. Users runC CYCLE - OCI SPEC IMPLEMENTERS - - Intel Clear Containers - (now combined as Kata containers) - Singularity (HPC container runtime) - runz, Nabla, gVisor
  19. 19. • As a core component of every release of Docker engine since early 2016, and the CNCF containerd 1.0 branch since December 2017, containerd has received significant production usage. Now in 2018 we see containerd used in two public cloud managed Kubernetes offerings (GKE and IBM Cloud IKS), and a long list of additional adopters found in the TOC graduation proposal PR Rio project Users
  20. 20. Kubernetes; Container Orchestrator ▪ Kubernetes has no code to execute or run containers on Linux or Windows ▪ Initially the Kubernetes node daemon ( the “kubelet”) had direct linkage to the Docker engine 20 kubelet dockershim dockerd containerd runc
  21. 21. kubelet kubelet dockershim (CRI) Docker engine containerd containerd-shim containerd-shim containerd-shim runc runc runc containerd containerd-shim containerd-shim containerd-shim runc runc runc cri plugin containerd cri-containerd ttrpc: very lightweight gRPC protocol format Kubernetes CRI Runtimes: Docker vs. cri-containerd ( **NOTE: Cri-container project merged into containerd GitHub project in January 2018; became a plugin within the containerd binary ) ** 21
  22. 22. 6. Demo Peeling back the layers
  23. 23. 7. Interoperability The holy grail
  24. 24. Registries A Growing OCI-based Ecosystem 24 Runtimes Platforms DockerHub GCR (Azure) ICCR DTR (Docker EE) ... Build Tools `docker build` BuildKit img Kaniko Google Container Builder buildah Docker Desktop Docker Engine Docker Enterprise containerd Kata Containers AWS Firecracker cri-o (runc) Docker Docker Swarm Kubernetes Cloud Foundry OpenFaaS OpenWhisk Knative
  25. 25. 25 Thanks! @estesp Slack/IRC: estesp