Container Runtimes and Tooling
Oleg Chunikhin | CTO, Kublr
Oleg Chunikhin
CTO, Kublr
• 25 years in software architecture & development
• Working w/ Kubernetes since its release in 2015
• Software architect behind Kublr—an enterprise
ready container management platform
• @olgch
Introductions
Automation
Ingress
Custom
Clusters
Infrastructure
Logging Monitoring
Observability
API
Usage
Reporting
RBAC IAM
Air Gap TLS
Certificate
Rotation
Audit
Storage Networking Container
Registry
CI / CD App Mgmt
Infrastructure
Container Runtime Kubernetes
OPERATIONS SECURITY &
GOVERNANCE
Timeline
● Mar 2013: Docker released as open-source, uses LXC ath the time
● Mar 2014: Docker 0.9 released switching to libcontainer written in Go
● Oct 2014: Microsoft announced Docker integration
● Jun 2015: OCI Created by Docker, Coreos, Google, Microsoft and others
● Jul 2015: Kubernetes 1.0 released as open source
● Dec 2016: Kubernetes 1.5 introduces CRI
● Aug 2017: Moby project created for open R&D and upstream for Docker CE and EE
● Oct 2017: RedHat introduces CRI-O 1.0
● Apr 2018: Docker Registry HTTP API V2 was adopted by OCI as the distribution spec
● May 2021: OCI Distribution Spec reaches 1.0
● Aug 2021: Docker Desktop (NOT Docker CE) is not free for enterprise users
Container Management Landscape
● Specifications: OCI - Open Container Initiative
○ Image, Runtime, Distribution
● Container Runtimes and Engines
○ Docker, moby, containerd, runc, CRI-O
● Container Orchestration, CRI
○ Kubernetes, CRI, docker shim, CRI-O
● Tools
○ docker, ctr, nerdctl, podman, buildah, bazel
OCI - Open Container Initiative
OCI Specs
OCI Runtime Spec
Container Lifecycle
OCI Image Spec
Docker/Containerd - container implementation
Container runtime
Container engine/tools
OCI-compliant tool
creates and runs container processes
Default OCI implementation
OCI-compliant container
process management library
container
runtime spec
Basic container and image
management tools
Upstream open source
container and image
management and build
tools
Docker container and
image management and
build tools
CRI-O - container implementation
Kubernetes and container runtimes
CRI-O
Containerd
CRI-O
Docker tools
docker container ps -a
docker run --name cnt1 -it my-img:v1.0
docker stop cnt1
docker kill cnt1
docker rm cnt1
docker image build –-tag my-img:v1.0 .
docker image ls
docker image tag my-img:v1.0 my-repo.com/my-img:v1.0
docker image push my-repo.com/my-img:v1.0
docker image pull kublr/kublr:1.22.2
FROM ubuntu:20.04
ADD file1 file2
RUN apt install curl
ENTRYPOINT ["/bin/bash"]
containerd tools
ctr -n my-ns container list
ctr -n my-ns container create docker.io/library/alpine:latest my-cnt echo Hi
ctr -n my-ns task start my-cnt
ctr -n my-ns task list
ctr -n my-ns container remove my-cnt
ctr namespaces list
ctr namespaces create my-namespace
ctr -n my-ns images pull docker.io/library/alpine:latest
ctr -n my-ns images push docker.io/library/alpine:latest
ctr -n my-ns images remove docker.io/library/alpine:latest
ctr -n my-ns images mount docker.io/library/alpine:latest /root/m
ctr -n my-ns images unmount /root/m
Pros:
1. namespaces
2. labels
3. image mounts
4. content
Cons:
1. No image build tools
Different:
1. No docker hub default
nerdctl ...
Podman
podman container ps -a
podman run --name cnt1 -it my-img:v1.0
podman stop cnt1
podman kill cnt1
podman rm cnt1
# podman ~ docker
# uses buildah
podman image build –-tag my-img:v1.0 .
podman image ls
podman image tag my-img:v1.0 my-repo.com/my-img:v1.0
podman image push my-repo.com/my-img:v1.0
podman image pull kublr/kublr:1.22.2
# Containerfile/Dockerfile
FROM ubuntu:20.04
ADD file1 file2
RUN apt install curl
ENTRYPOINT ["/bin/bash"]
Buildah
container=$(buildah from fedora)
buildah run $container bash
buildah run $container -- dnf -y install java
buildah build -f Dockerfile -t fedora-httpd .
# Containerfile/Dockerfile
FROM ubuntu:20.04
ADD file1 file2
RUN apt install curl
ENTRYPOINT ["/bin/bash"]
Bazel
# WORKSPACE
container_pull(
name = "kublr_alpine",
registry = "cr.kublr.com",
repository = "kublr/alpine",
# tag = "3.14.2-6",
digest = "sha256:5363....f164",
)
# BUILD.bazel
pkg_tar(
name = "image-tar-svc",
files = {":svc"},
mode = "0555",
package_dir = "/opt",
strip_prefix = strip_prefix.from_pkg(),
)
container_image(
name = "image",
base = "@kublr_alpine//image",
entrypoint = ["/opt/my-svc", "--port", "8080"],
layers = [":image-layer-swagger"],
ports = ["8080"],
tars = [":image-tar-svc"],
workdir = "/opt",
)
Buildah
Tool Components / Focus / Limitations / Benefits
libcontainer runtime (library)
runc runtime
containerd runtime, container, image, registry
cri-o runtime, container, image, registry
docker runtime, container, image, registry, build
podman container, image, registry, build (client only, no daemon)
buildah build, container, image, registry (client only, no daemon)
bazel (docker rules) build, registry (limited, portable, hermetic)
References
@olgch; @kublr
github.com/opencontainers/image-spec
github.com/opencontainers/runtime-spec
github.com/opencontainers/distribution-spec
github.com/opencontainers/artifacts
OCI Specifications (Alibaba Blog)
https://mobyproject.org/
https://cri-o.io/
https://github.com/cri-o/cri-o
https://podman.io/
https://buildah.io/
https://bazel.build/
Beyond the Presentation
● Docker Swarm, Notary etc, …
● Image registries
● CRI-O CLI tools
● Windows Containers
Q&A
Oleg Chunikhin
CTO
oleg@kublr.com
@olgch
Follow Us
@kublr
Thank You

Container Runtimes and Tooling

  • 1.
    Container Runtimes andTooling Oleg Chunikhin | CTO, Kublr
  • 2.
    Oleg Chunikhin CTO, Kublr •25 years in software architecture & development • Working w/ Kubernetes since its release in 2015 • Software architect behind Kublr—an enterprise ready container management platform • @olgch Introductions
  • 3.
    Automation Ingress Custom Clusters Infrastructure Logging Monitoring Observability API Usage Reporting RBAC IAM AirGap TLS Certificate Rotation Audit Storage Networking Container Registry CI / CD App Mgmt Infrastructure Container Runtime Kubernetes OPERATIONS SECURITY & GOVERNANCE
  • 4.
    Timeline ● Mar 2013:Docker released as open-source, uses LXC ath the time ● Mar 2014: Docker 0.9 released switching to libcontainer written in Go ● Oct 2014: Microsoft announced Docker integration ● Jun 2015: OCI Created by Docker, Coreos, Google, Microsoft and others ● Jul 2015: Kubernetes 1.0 released as open source ● Dec 2016: Kubernetes 1.5 introduces CRI ● Aug 2017: Moby project created for open R&D and upstream for Docker CE and EE ● Oct 2017: RedHat introduces CRI-O 1.0 ● Apr 2018: Docker Registry HTTP API V2 was adopted by OCI as the distribution spec ● May 2021: OCI Distribution Spec reaches 1.0 ● Aug 2021: Docker Desktop (NOT Docker CE) is not free for enterprise users
  • 5.
    Container Management Landscape ●Specifications: OCI - Open Container Initiative ○ Image, Runtime, Distribution ● Container Runtimes and Engines ○ Docker, moby, containerd, runc, CRI-O ● Container Orchestration, CRI ○ Kubernetes, CRI, docker shim, CRI-O ● Tools ○ docker, ctr, nerdctl, podman, buildah, bazel
  • 6.
    OCI - OpenContainer Initiative OCI Specs OCI Runtime Spec Container Lifecycle OCI Image Spec
  • 7.
    Docker/Containerd - containerimplementation Container runtime Container engine/tools OCI-compliant tool creates and runs container processes Default OCI implementation OCI-compliant container process management library container runtime spec Basic container and image management tools Upstream open source container and image management and build tools Docker container and image management and build tools
  • 8.
    CRI-O - containerimplementation
  • 9.
    Kubernetes and containerruntimes CRI-O Containerd CRI-O
  • 10.
    Docker tools docker containerps -a docker run --name cnt1 -it my-img:v1.0 docker stop cnt1 docker kill cnt1 docker rm cnt1 docker image build –-tag my-img:v1.0 . docker image ls docker image tag my-img:v1.0 my-repo.com/my-img:v1.0 docker image push my-repo.com/my-img:v1.0 docker image pull kublr/kublr:1.22.2 FROM ubuntu:20.04 ADD file1 file2 RUN apt install curl ENTRYPOINT ["/bin/bash"]
  • 11.
    containerd tools ctr -nmy-ns container list ctr -n my-ns container create docker.io/library/alpine:latest my-cnt echo Hi ctr -n my-ns task start my-cnt ctr -n my-ns task list ctr -n my-ns container remove my-cnt ctr namespaces list ctr namespaces create my-namespace ctr -n my-ns images pull docker.io/library/alpine:latest ctr -n my-ns images push docker.io/library/alpine:latest ctr -n my-ns images remove docker.io/library/alpine:latest ctr -n my-ns images mount docker.io/library/alpine:latest /root/m ctr -n my-ns images unmount /root/m Pros: 1. namespaces 2. labels 3. image mounts 4. content Cons: 1. No image build tools Different: 1. No docker hub default nerdctl ...
  • 12.
    Podman podman container ps-a podman run --name cnt1 -it my-img:v1.0 podman stop cnt1 podman kill cnt1 podman rm cnt1 # podman ~ docker # uses buildah podman image build –-tag my-img:v1.0 . podman image ls podman image tag my-img:v1.0 my-repo.com/my-img:v1.0 podman image push my-repo.com/my-img:v1.0 podman image pull kublr/kublr:1.22.2 # Containerfile/Dockerfile FROM ubuntu:20.04 ADD file1 file2 RUN apt install curl ENTRYPOINT ["/bin/bash"]
  • 13.
    Buildah container=$(buildah from fedora) buildahrun $container bash buildah run $container -- dnf -y install java buildah build -f Dockerfile -t fedora-httpd . # Containerfile/Dockerfile FROM ubuntu:20.04 ADD file1 file2 RUN apt install curl ENTRYPOINT ["/bin/bash"]
  • 14.
    Bazel # WORKSPACE container_pull( name ="kublr_alpine", registry = "cr.kublr.com", repository = "kublr/alpine", # tag = "3.14.2-6", digest = "sha256:5363....f164", ) # BUILD.bazel pkg_tar( name = "image-tar-svc", files = {":svc"}, mode = "0555", package_dir = "/opt", strip_prefix = strip_prefix.from_pkg(), ) container_image( name = "image", base = "@kublr_alpine//image", entrypoint = ["/opt/my-svc", "--port", "8080"], layers = [":image-layer-swagger"], ports = ["8080"], tars = [":image-tar-svc"], workdir = "/opt", )
  • 15.
    Buildah Tool Components /Focus / Limitations / Benefits libcontainer runtime (library) runc runtime containerd runtime, container, image, registry cri-o runtime, container, image, registry docker runtime, container, image, registry, build podman container, image, registry, build (client only, no daemon) buildah build, container, image, registry (client only, no daemon) bazel (docker rules) build, registry (limited, portable, hermetic)
  • 16.
    References @olgch; @kublr github.com/opencontainers/image-spec github.com/opencontainers/runtime-spec github.com/opencontainers/distribution-spec github.com/opencontainers/artifacts OCI Specifications(Alibaba Blog) https://mobyproject.org/ https://cri-o.io/ https://github.com/cri-o/cri-o https://podman.io/ https://buildah.io/ https://bazel.build/
  • 17.
    Beyond the Presentation ●Docker Swarm, Notary etc, … ● Image registries ● CRI-O CLI tools ● Windows Containers
  • 18.
  • 19.