Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

[KubeConCN] BuildKit: A Modern Builder Toolkit on Top of containerd

961 views

Published on

BuildKit is a modern OCI image builder toolkit based on containerd container runtime used as a new backend in Docker build command and in rootless builder img.

In this session, we'll demonstrate the capabilities of BuildKit and how it can help to improve your current application development process and your CI workflow. For example, we'll explain new features that allow to significantly improve the performance of your Dockerfiles or how with remote caching support in BuildKit you can be used to speed up your CI builds.

Relying on the containerd manifest list support BuildKit can build multi-platform images with a single build request and a single Dockerfile.

Participants will learn how to use BuildKit today, either as part of Docker platform or as a standalone tool deployed on a Kubernetes cluster, and the benefits it has compared to the previous image building methods.

https://sched.co/Nrlo

Published in: Software
  • Be the first to comment

  • Be the first to like this

[KubeConCN] BuildKit: A Modern Builder Toolkit on Top of containerd

  1. 1. BuildKit: A Modern Builder Toolkit on Top of containerd (BuildKit: 建立在containerd之上的现代构建工具包) Tõnis Tiigi, Docker Akihiro Suda, NTT (须田 瑛大, 日本电信电话)
  2. 2. About us 2 Tonis Tiigi @tonistiigi Software engineer Docker Inc. Maintainer of BuildKit and Moby/Docker Akihiro Suda @_AkihiroSuda_ Software engineer NTT Corporation Maintainer of BuildKit, containerd, Moby...
  3. 3. 3 What is BuildKit?
  4. 4. 4 How are container images built? Dockerfile > docker build . Bundled into Docker daemon
  5. 5. 5 What’s the issue with old builder? ● Old design/codebase ● Tightly modeled after Dockerfile instructions ● Hard to add new (Dockerfile) features ● Suboptimal performance ● Leaks state to other Docker APIs ● Not usable for other projects
  6. 6. 6 BuildKit solves these problems ● Dozens on new features and bugfixes ● Much faster ● Language agnostic ● Componentized ● Toolkit for building opinionated builders
  7. 7. 7 Built on containerd ● Snapshotters ● Distribution ● Blobs storage ● GC containerd - An open and reliable container runtime
  8. 8. 8 Embraces OCI standards ● Process execution with OCI Runtime specification ● Build results can be exported with OCI Image specification (including manifest lists) OCI - Open Container Initiative
  9. 9. Part 2 BuildKit Innovations 9
  10. 10. Problems of legacy docker build 10 ● The legacy docker build does not compute dependencies across Dockerfile instructions correctly ● Modifying line N always invalidates the cache for line (N+1) FROM debian EXPOSE 80 RUN apt update && apt install -y HEAVY-PACKAGES
  11. 11. Problems of legacy docker build 11 FROM golang AS stage0 ... RUN go build –o /foo ... FROM clang AS stage1 ... RUN clang –o /bar ... FROM debian AS stage2 EXPOSE 80 RUN apt ... COPY --from=stage0 /foo / COPY --from=stage1 /bar / 0
 2
 1
 0
 2
 1
 Expected schedule Actual
  12. 12. BuildKit LLB 12 ● LLB is to Dockerfile what LLVM IR is to C ● Accurate dependency expression with graph structure ○ Efficient caching ○ Concurrent execution ● Encoded in protobuf; typically compiled from Dockerfile ○ Other “frontends” are also available: Buildpacks, Mockerfile, Gockerfile, Docker Assemble
  13. 13. BuildKit LLB 13 SourceOp ... ExecOp SourceOp ... ExecOp SourceOp ExecOp FileOp FROM golang AS stage0 ... RUN go build –o /foo ... FROM clang AS stage1 ... RUN clang –o /bar ... FROM debian AS stage2 EXPOSE 80 RUN apt ... COPY --from=stage0 /foo / COPY --from=stage1 /bar / FileOp Note: No “ExposeOp”
  14. 14. •DAGはマルチステージDockerfileを用いて記述できる BuildKit: 次世代 `docker build` FROM golang AS stage0 ... RUN go build –o /foo ... FROM clang AS stage1 ... RUN clang –o /bar ... FROM debian AS stage2 COPY --from=stage0 /foo /usr/local/bin/foo COPY --from=stage1 /bar /usr/local/bin/bar 0
 2
 1
 https://t.co/aUKqQCVmXa
  15. 15. Extensible syntax 15 ● “LLB frontend” container can be specified in the first line of Dockerfile (# syntax = ...) ● You can also create your own LLB frontend container i.e. you can define your own syntax # syntax = docker/dockerfile:1.1-experimental FROM ... RUN ...
  16. 16. RUN --mount=type=cache 16 ● Allows preserving caches of compilers and package managers # syntax = docker/dockerfile:1.1-experimental ... RUN --mount=type=cache,target=/root/.cache go build ...
  17. 17. https://t.co/aUKqQCVmXa
  18. 18. RUN --mount=type=secret 18 ● Allows accessing private assets without leaking credential in the image # syntax = docker/dockerfile:1.1-experimental ... RUN --mount=type=secret,id=aws,target=/root/.aws/credentials aws s3 cp s3://... ... $ buildctl build –-secret id=aws,src=~/.aws/credentials ...
  19. 19. RUN --mount=type=secret 19 ● Note: DON’T do this! ... COPY my_aws_credentials /root/.aws/credentials RUN aws s3 cp s3://... … RUN rm -f /root/.aws/credentials ...
  20. 20. RUN --mount=type=secret 20 ● Note: DON’T do this either! $ docker build --build-arg MY_AWS_CREDENTIALS=$(cat ~/.aws/credentials)
  21. 21. Part 3 Using BuildKit 21
  22. 22. 22 Many ways to use BuildKit ● Docker, docker buildx ● img ● Tekton ● Rio, Pouch, vab With or without daemon, in container, in k8s, with containerd daemon, without root privileges, etc.
  23. 23. 23 Docker ● Integrated into “docker build” v18.09+ ● Opt-in: export DOCKER_BUILDKIT=1
  24. 24. 24 Docker
  25. 25. ● Next generation Build command from Docker ● Familiar Docker UI + full BuildKit ● Manages instances of Builders and Build nodes ● With container driver, works with any version of Docker engine 25 Docker Buildx
  26. 26. 26 Buildx: Full BuildKit ● Remote caching (eg. for CI) ● Multi-platform images support ○ --platform=linux/amd64,linux/arm64 ○ QEMU, distributed among nodes, or cross-compilation in multi-stage Dockerfile
  27. 27. 27 Buildx: Multi-platform images ● Webassembly: wasi/wasm https://github.com/tonistiigi/wasm-cli-plugin ● Initial RISC-V support: linux/riscv64 https://tinyurl.com/docker-riscv
  28. 28. Part 4 Deploying BuildKit on Kubernetes 28
  29. 29. Why build images on Kube? 29 1. CI/CD
  30. 30. 30 2. Developer Experience Why build images on Kube? 1. CI/CD
  31. 31. Legacy docker build on Kubernetes 31 ● docker /var/run/docker.sock ● docker:dind securityContext.privileged ●
  32. 32. Rootless mode 32 ● ● securityContext
  33. 33. Rootless BuildKit vs Kaniko 33 ● Kaniko runs as the root user but “unprivileged” ○ No need to disable seccomp and AppArmor ● Kaniko might be able to mitigate some vuln that Rootless BuildKit cannot mitigate - and vice versa ○ Rootless BuildKit might be weak against kernel vulns ○ Kaniko might be weak against runc vulns
  34. 34. Deployment strategy 34 DaemonSet? Deployment? StatefulSet? Job?
  35. 35. Deployment strategy 35 ● Deployment ○ Most typical deployment ● DaemonSet ○ Optimal load-balancing but non-optimal caching ● StatefulSet ○ Good for Consistent Hashing (discussed later) ● Job ○ No need to manage the life cycles of the daemons
  36. 36. Caching 36 Load-balancing component (Can be just headless svc with DNSRR) gRPC request Image Cache
  37. 37. Caching 37 ●Remote cache might be slow compared to the daemon-local cache ●Example: ○ No cache: 2m50s ○ Remote cache: 36s ○ Daemon-local cache: 0.5s
  38. 38. Caching 38 ●Consistent hashing allows sticking a build request to a specific Pod in StatefulSet ●Always hits the cache, but non-optimal load balancing buildkitd-1 buildkitd-0 buildkitd-2 foo/Dockerfile bar/Dockerfile baz/Dockerfile https://tinyurl.com/consistent-hash
  39. 39. 39 Recap ● BuildKit is a modern container Build toolkit ● Significant advantages over previous tools ● Usable with Docker, K8s and many other tools ● Open platform for collaboration around build
  40. 40. Join us: https://github.com/moby/buildkit
  41. 41. Extra slides 41
  42. 42. Tekton ● CRD for building images ● Successor of Knative Build 42
  43. 43. Tekton 43 The interface is same as other image builders (Buildah, Kaniko, and Makisu) Credentials are loaded from the Secret associated with the ServiceAccount
  44. 44. Tekton 44
  45. 45. Rancher Rio 45 ● k8s/k3s-based micro PaaS ● “rio run https://github.com/…” builds and deploy app in one-line ● Internally using BuildKit, but users don’t need to care about BuildKit

×