Successfully reported this slideshow.
Your SlideShare is downloading. ×

The Dockerfile Explosion and the Need for Higher Level Tools by Gareth Rushgrove

The Dockerfile Explosion and the Need for Higher Level Tools by Gareth Rushgrove

Download to read offline

Dockerfiles are great. They provide a zero-barrier-to-entry format for
describing a single Docker image which is immediately clear to anyone
reading them. But with that simplicity comes problems that become
apparent as your adoption of Docker gathers pace.

* Dockerfiles can inherit from other docker images, but images are not
Dockerfiles
* Dockerfile provides no built-in mechanism for creating abstractions,
so as usage grows identical or similar instructions can be duplicated
across many files
* The Docker APi exposes a build endpoint, but the API is very course,
taking Dockerfile as the transport rather than exposing the individual
instructions
* Dockerfiles are just that, files. So they can come from anywhere

The one layer per line in a Dockerfile limitation can lead to an
explosion of layers, which fail to take advantage of the promised
space and performance benefits.

Dockerfiles are great. They provide a zero-barrier-to-entry format for
describing a single Docker image which is immediately clear to anyone
reading them. But with that simplicity comes problems that become
apparent as your adoption of Docker gathers pace.

* Dockerfiles can inherit from other docker images, but images are not
Dockerfiles
* Dockerfile provides no built-in mechanism for creating abstractions,
so as usage grows identical or similar instructions can be duplicated
across many files
* The Docker APi exposes a build endpoint, but the API is very course,
taking Dockerfile as the transport rather than exposing the individual
instructions
* Dockerfiles are just that, files. So they can come from anywhere

The one layer per line in a Dockerfile limitation can lead to an
explosion of layers, which fail to take advantage of the promised
space and performance benefits.

Advertisement
Advertisement

More Related Content

Viewers also liked

Advertisement
Advertisement

The Dockerfile Explosion and the Need for Higher Level Tools by Gareth Rushgrove

  1. 1. The Dockerfile explosion Gareth Rushgrove Senior Software Engineer Puppet
  2. 2. The Dockerfile explosion and the need for higher level tools
  3. 3. Introductions Who am I and what am I doing here
  4. 4. @garethr
  5. 5. (without introducing more risk) Gareth Rushgrove
  6. 6. Built the Puppet Docker module
  7. 7. Maintain the Puppet images
  8. 8. Obsessed with metadata
  9. 9. A brief history of Dockerfile
  10. 10. Docker can build images automatically by reading the instructions from a Dockerfile From the official docs at https://docs.docker.com/engine/reference/builder/
  11. 11. A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. From the official docs at https://docs.docker.com/engine/reference/builder/
  12. 12. A simple Dockerfile FROM ubuntu # Install vnc, xvfb in order to create a 'fake' display and fire RUN apt-get update && apt-get install -y x11vnc xvfb firefox RUN mkdir ~/.vnc # Setup a password RUN x11vnc -storepasswd 1234 ~/.vnc/passwd # Autostart firefox (might not be the best way, but it does the RUN bash -c 'echo "firefox" >> /.bashrc' EXPOSE 5900 CMD ["x11vnc", "-forever", "-usepw", "-create"]
  13. 13. Dockerfile reference
  14. 14. Commands you know MAINTAINER <name> RUN <command> CMD ["executable","param1","param2"] EXPOSE <port> [<port>...] ADD <src>... <dest> ENV <key> <value> WORKDIR /path/to/workdir USER daemon VOLUME ["/data"] ENTRYPOINT ["executable", "param1", “param2"] COPY <src>... <dest>
  15. 15. Commands you don’t know ONBUILD [INSTRUCTION] STOPSIGNAL signal ARG <name>[=<default value>] LABEL <key>=<value> <key>=<value> <key>=<value> … HEALTHCHECK [OPTIONS] CMD command SHELL ["executable", "parameters"]
  16. 16. Close ALL the issues
  17. 17. Although this is not a definitive move, we temporarily won't accept more patches to the Dockerfile syntax for several reasons
  18. 18. HEALTHCHECK coming in 1.12
  19. 19. SHELL coming in 1.12
  20. 20. Why Dockerfiles are great
  21. 21. Simplicity FROM scratch COPY hello / CMD ["/hello"]
  22. 22. Multi-platform support PS> Install-PackageProvider ContainerImage -Force PS> Install-ContainerImage -Name WindowsServerCore PS> docker images REPOSITORY TAG IMAGE ID CREA windowsservercore 10.0.14300.1000 dbfee88ee9fd 7 we
  23. 23. Emerging tooling
  24. 24. Linting
  25. 25. Editor support
  26. 26. Cross platform
  27. 27. Why Dockerfiles are problematic
  28. 28. Complexity RUN apt-get update && apt-get install -y wget=1.17.1-1ubuntu1 && wget https://apt.example.com/release-"$UBUNTU_CODENAME".deb dpkg -i release-"$UBUNTU_CODENAME".deb && rm release-"$UBUNTU_CODENAME".deb && apt-get update && apt-get install --no-install-recommends -y package=0.1.2 && apt-get clean && rm -rf /var/lib/apt/lists/*
  29. 29. Dockerfile proliferation
  30. 30. language:Dockerfile maintainer
  31. 31. 138,062
  32. 32. Only two approaches to reuse
  33. 33. Inheritance FROM debian:jessie
  34. 34. Dockerfile is not the source of truth for your image
  35. 35. The Dockerfile generally works beautifully for the class of problem for which it was designed Nathan Leclair, Docker Inc
  36. 36. The Dockerfile is a tool for creating images, but it is not the only weapon in your arsenal Nathan Leclair, Docker Inc
  37. 37. Putting the problems in context
  38. 38. If we dockerize all of our applications how many Dockerfiles is that?
  39. 39. If we build a complex hierarchy of Dockerfiles, how quickly can we trace/rebuild a specific image?
  40. 40. As best-practices develops how can we refactor our Dockefiles with confidence?
  41. 41. Are Dockerfiles best managed centrally or on a team-by-team basis?
  42. 42. Some community ideas
  43. 43. Generate Dockerfiles
  44. 44. Build Dockerfiles with OCAML
  45. 45. let base = let email = "anil@recoil.org" in comment "Generated by OCaml Dockerfile" @@ from "ubuntu" ~tag:"trusty" @@ maintainer "Anil Madhavapeddy <%s>" email let ocaml_ubuntu_image = base @@ run "apt-get -y -qq update" @@ run "apt-get -y install ocaml ocaml-native-compilers camlp4-ext onbuild (run "apt-get -y -qq update") ;; OCAML example
  46. 46. With Gradle
  47. 47. Or Javascript
  48. 48. Or Scala and SBT
  49. 49. Or with Python
  50. 50. No Dockerfile to be seen
  51. 51. Docker Image Specification
  52. 52. Packer
  53. 53. { "builders":[{ "type": "docker", "image": "ubuntu", "export_path": "image.tar" }], "provisioners":[ { "type": "shell", "inline": ["apt-get -y update; apt-get install -y puppet-co }, { Packer example
  54. 54. Source-to-Image
  55. 55. $ s2i create <image name> <destination directory> $ s2i build <source location> <builder image> [<tag>] [flags] $ s2i rebuild <image name> [<new-tag-name>] $ s2i usage <builder image> [flags] $ s2i build ./sinatra-app openshift/ruby-20-centos7 ruby-app s2i example
  56. 56. Nix
  57. 57. dockerTools.buildImage { name = "redis"; runAsRoot = '' #!${stdenv.shell} ${dockerTools.shadowSetup} groupadd -r redis useradd -r -g redis -d /data -M redis mkdir /data chown redis:redis /data ''; contents = [ redis ]; Nix example
  58. 58. Habitat
  59. 59. Expand on Dockerfile
  60. 60. Rocker
  61. 61. Rocker adds some crucial features that are missing from Dockerfile while keeping Docker’s original design
  62. 62. FROM ubuntu:16.04 MAINTAINER Gareth Rushgrove "gareth@puppet.com" ENV PUPPET_AGENT_VERSION="1.5.0" UBUNTU_CODENAME="xenial" PATH=/ LABEL com.puppet.version="0.1.0" com.puppet.dockerfile="/Dockerf MOUNT /opt/puppetlabs /etc/puppetlabs /root/.gem RUN apt-get update && apt-get install -y wget=1.17.1-1ubuntu1 && wget https://apt.puppetlabs.com/puppetlabs-release-pc1-"$UBU Rockerfile example
  63. 63. FROM ubuntu:16.04 MAINTAINER Gareth Rushgrove "gareth@puppet.com" ENV PUPPET_AGENT_VERSION="1.5.0" UBUNTU_CODENAME="xenial" PATH=/ LABEL com.puppet.version="0.1.0" com.puppet.dockerfile="/Dockerf MOUNT /opt/puppetlabs /etc/puppetlabs /root/.gem RUN apt-get update && apt-get install -y wget=1.17.1-1ubuntu1 && wget https://apt.puppetlabs.com/puppetlabs-release-pc1-"$UBU Includes new instructions
  64. 64. rm -rf /var/lib/apt/lists/* EXPOSE 80 CMD ["nginx"] COPY Rockerfile /Dockerfile TAG puppet/puppet-rocker-example More new instructions
  65. 65. Dockramp
  66. 66. Dockerfile pre-processors
  67. 67. FROM ubuntu:16.04 MAINTAINER Gareth Rushgrove "gareth@puppet.com" ENV PUPPET_AGENT_VERSION="1.5.0" R10K_VERSION="2.2.2" UBUNTU_C PUPPET_INSTALL PUPPET_COPY_PUPPETFILE PUPPET_COPY_MANIFESTS PUPPET_RUN EXPOSE 80 Domain-specific extensions
  68. 68. $ cat Dockerfile | dockerfilepp FROM ubuntu:16.04 MAINTAINER Gareth Rushgrove "gareth@puppet.com" ENV PUPPET_AGENT_VERSION="1.5.0" R10K_VERSION="2.2.2" UBUNTU_COD RUN apt-get update && apt-get install -y wget=1.17.1-1ubuntu1 && wget https://apt.puppetlabs.com/puppetlabs-release-pc1-"$UBU dpkg -i puppetlabs-release-pc1-"$UBUNTU_CODENAME".deb && rm puppetlabs-release-pc1-"$UBUNTU_CODENAME".deb && apt-get update && Simple expansion
  69. 69. The future Speculation and things I’d like to see
  70. 70. Formal specification for Dockerfile
  71. 71. RUN, FROM, COPY, etc. as first class API primitives
  72. 72. Opinionated workflow tooling around image build
  73. 73. Shared libraries and support for pre-processors
  74. 74. Complementary tools that take an organizational view of image building
  75. 75. Conclusions If all you take away is…
  76. 76. Dockerfile is a great starting point for many use cases
  77. 77. But we will need better tools for managing many Dockerfiles
  78. 78. And Dockerfile is just one interface to building images
  79. 79. We’ll need different types of tools for different use cases
  80. 80. Questions? And thanks for listening
  81. 81. Thank you!

×