Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Creating Effective Images
Sr Technical Evangelist, AWS
@abbyfuller
Abby Fuller
• How do layers work?
• The basics for building minimal images
• High level best practices for Windows containers
• Docker...
How do layers work?
What are container layers?
read-only container layers
Thin read-write layer
Why do I care how many layers I have?
More layers mean a larger image. The larger the image, the longer
that it takes to b...
OK, so how can I reduce my layers?
Sharing is caring.
• Use shared base images where
possible
• Limit the data written to ...
Building minimal images: the basics
A Dockerfile is a series of
instructions for building images.
Cache rules everything around me
CACHE
Let’s start with a Dockerfile
FROM ubuntu:latest
LABEL maintainer abbyfull@amazon.com
RUN apt-get update -y && apt-get ins...
First step: choosing the right base
From the stock ubuntu image:
ubuntu              latest              2b1dc137b502   
 ...
Slightly better- choose a different distro
alpine     latest        d3145c9ba1fa       3.9 MB
python     3.6.1-slim    609...
When do I want a full base OS?
(I do actually like Ubuntu!)
• Security
• Compliance
• Ease of development
Let’s look at our original Ubuntu image
FROM ubuntu:latest
RUN apt-get update -y && apt-get install -y python-pip
python-d...
Simple changes, big results
FROM python:2.7-alpine
LABEL maintainer abbyfull@amazon.com
COPY . /app
WORKDIR /app
RUN pip i...
Fewer cache invalidations=smaller images
FROM python:2.7-alpine
LABEL maintainer abbyfull@amazon.com
COPY requirements.txt...
Got application code?
FROM python:2.7-alpine
LABEL maintainer abbyfull@amazon.com
ONBUILD ADD requirements.txt /app
ONBUIL...
Let’s recap.
TL;DR: layers represent filesystem differences.
Layers add up quickly with big consequences.
Some high-level best practices: Windows
Port over existing VM workloads
Convert an existing Windows image:
ConvertTo-Dockerfile -ImagePath c:dockermyimage.wim
Con...
Some things to think about
Watch what you build: 
    c:   c:    /  /windows c:/windows
Building any of those PATHs will m...
Avoid installing packages with MSI
MSI installations are not space efficient. This is not
the same as Linux distros, where ...
Coming up soon
Run Linux containers “as-is” on
Windows Server!
Here’s whats really cool though
Build and run everything the same, regardless of
container OS, host OS, or tools. Just doc...
…but I’m not a Windows expert
So go to see Elton
instead! He’ll talk on
modernizing .NET apps at
17:10 for the Using Docke...
Dockerfiles: the good, the bad, and the
bloated
Let’s start out big
FROM ubuntu:latest
LABEL maintainer abbyfull@amazon.com
RUN apt-get update -y 
RUN apt-get install -y ...
A little bit better
FROM ubuntu:latest
LABEL maintainer abbyfull@amazon.com
RUN apt-get update -y && apt-get install -y py...
Let’s try a different base
FROM python:2.7-alpine
LABEL maintainer abbyfull@amazon.com
COPY . /app
WORKDIR /app
RUN pip in...
Or, let’s try a custom base container
FROM 621169296726.dkr.ecr.us-
east-1.amazonaws.com/dockercon-base:latest
LABEL maint...
Use RUN statements effectively
RUN apt-get update && apt-get install -y 
    aufs-tools 
    automake 
    build-essential...
Switching USER adds layers
RUN groupadd –r dockercon && useradd –r –g
dockercon dockercon
USER dockercon
RUN apt-get updat...
Avoid ADDing large files
BAD:
ADD http://cruft.com/bigthing.tar.xz /app/cruft/
RUN tar -xvf /app/cruft/bigthing.tar.xz -C ...
BEST
RUN mkdir -p /app/cruft/ 
    && curl -SL http://cruft.com/
bigthing.tar.xz  | tar -xvf /app/
cruft/ 
   && make -C /...
Let’s get (language) specific
A few language-specific best practices
Use the right tool: not every language needs to be built the
same way.
•Where possi...
First stop: Golang
Compile, then COPY binary:
$  go build -o dockercon .
$  docker build -t dockercon .
Dockerfile:
FROM sc...
Quick detour: what’s scratch?
Special, empty Dockerfile.
Use this to build your own base images.
Or, use to build minimal i...
Back to business: Ruby
Official images for Ruby are extra huge.  A new base + a little extra
work pays off.
FROM alpine:3.2...
Next: node.js
If you love yourself, .dockerignore npm-debug.log. 
Seriously.
But most importantly, cache node_modules:
COP...
Java!
Multi-stage builds are your friend:
FROM maven:3.5-jdk-8 as BUILD
COPY --from=BUILD
Like, Golang, this let’s you bui...
Multi-stage builds
FROM ubuntu AS build-env
RUN apt-get install make
ADD . /src
RUN cd /src && make
And for the second Doc...
Tools are here to help
With great containers comes great responsibility
• Document!
• Automate where possible
• AWS has a few tenants for
service...
Docker Security Scan
Docker Security Scan
Docker Image + System Prune
Docker image prune:
$ docker image prune –a
Alternatively, go even further with Docker system ...
The importance of garbage collection
• Clean up after your containers! Beyond
image and system prune:
• Make sure your orc...
Looking forward to the future
But wait, there’s always more!
• Always new and better things coming
• Linux and Windows Server
• Official image are multi...
So what did we learn?
One takeaway:  less  layers is more.
•Share layers where possible
•Choose or build your base wisely
...
Useful links
Docker image documentation
Docker scratch
atsea sample app
Arun Gupta on smaller Java images
Elton Stoneman W...
Thanks!
@abbyfuller
Upcoming SlideShare
Loading in …5
×

Creating Effective Docker Images

2,677 views

Published on

Sick of getting paged at 2am and wondering "where did all my disk space go?" This has actually happened to me, and you can learn from my mistakes! New Docker users often start with a stock image in order to get up and running quickly, but that isn't always the right answer. Creating efficient images is often overlooked, but important. Beyond saving resources, using minimal images also delivers important security benefits: include only what you need and not a whole runtime that might have security vulnerabilities. In this session, I'll talk about how to create effective images and lessons I've learned from running containers in production at a number of startups. I'll also cover topics like "how do layers work?" and some things you should think about when creating your images, such as; choosing or creating the right base image; the importance of caching; using RUN statements conservatively; cleaning up as you go. I'll also address best practices; both at a high level with multi-stage builds; and some language-specific best practices, for example, tips and tricks for creating containers for Node.js vs Go. To illustrate these points, we'll cover:
* How layers work?
* Choosing a base image vs. creating your own
* The basics of building minimal images and the importance of choosing a base image vs. creating your own
* The basics of building minimal images and the importance of caching
* High level best practices for Linux containers (in general, and some language specific examples)
* High level best practices for Windows container images
* New and improved: multi-stage builds
* Good vs. not so good Dockerfile examples
* Docker Image Scanning, and other friends
* What's up next? Looking to the future for more optimization.

Published in: Technology

Creating Effective Docker Images

  1. 1. Creating Effective Images Sr Technical Evangelist, AWS @abbyfuller Abby Fuller
  2. 2. • How do layers work? • The basics for building minimal images • High level best practices for Windows containers • Dockerfiles:  the good, the bad, and the bloated • Let’s get (language) specific • Tools are here to help • Looking forward to the future Agenda
  3. 3. How do layers work?
  4. 4. What are container layers? read-only container layers Thin read-write layer
  5. 5. Why do I care how many layers I have? More layers mean a larger image. The larger the image, the longer that it takes to both build, and push and pull from a registry. Smaller images mean faster builds and deploys. This also means a smaller attack surface.
  6. 6. OK, so how can I reduce my layers? Sharing is caring. • Use shared base images where possible • Limit the data written to the container layer • Chain RUN statements • Prevent cache misses at build for as long as possible
  7. 7. Building minimal images: the basics
  8. 8. A Dockerfile is a series of instructions for building images.
  9. 9. Cache rules everything around me CACHE
  10. 10. Let’s start with a Dockerfile FROM ubuntu:latest LABEL maintainer abbyfull@amazon.com RUN apt-get update -y && apt-get install -y python-pip python-dev build-essential COPY . /app WORKDIR /app RUN pip install –r requirements.txt EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  11. 11. First step: choosing the right base From the stock ubuntu image: ubuntu              latest              2b1dc137b502        52 seconds ago      458 MB From python:2.7-alpine: alpine              latest              d3145c9ba1fa        2 minutes ago       86.8 MB
  12. 12. Slightly better- choose a different distro alpine     latest        d3145c9ba1fa       3.9 MB python     3.6.1-slim    60952d8b5aeb       200 MB debian     latest        8cedef9d7368       123 MB python     2.7-alpine    b63d02d8829b       71.5 MB ubuntu     latest        0ef2e08ed3fa       130 MB fedora     latest        4daa661b467f       231 MB
  13. 13. When do I want a full base OS? (I do actually like Ubuntu!) • Security • Compliance • Ease of development
  14. 14. Let’s look at our original Ubuntu image FROM ubuntu:latest RUN apt-get update -y && apt-get install -y python-pip python-dev build-essential LABEL maintainer abbyfull@amazon.com COPY . /app WORKDIR /app RUN pip install –r requirements.txt EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  15. 15. Simple changes, big results FROM python:2.7-alpine LABEL maintainer abbyfull@amazon.com COPY . /app WORKDIR /app RUN pip install –r requirements.txt EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  16. 16. Fewer cache invalidations=smaller images FROM python:2.7-alpine LABEL maintainer abbyfull@amazon.com COPY requirements.txt /app RUN pip install –r /app/requirements.txt COPY . /app WORKDIR /app EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  17. 17. Got application code? FROM python:2.7-alpine LABEL maintainer abbyfull@amazon.com ONBUILD ADD requirements.txt /app ONBUILD RUN pip install –r /app/requirements.txt ONBUILD COPY . /app WORKDIR /app EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  18. 18. Let’s recap. TL;DR: layers represent filesystem differences. Layers add up quickly with big consequences.
  19. 19. Some high-level best practices: Windows
  20. 20. Port over existing VM workloads Convert an existing Windows image: ConvertTo-Dockerfile -ImagePath c:dockermyimage.wim Convert from VHD: ConvertTo-Dockerfile -ImagePath c:vmstest.vhd -Artifact IIS - ArtifactParam windows-container -OutputPath c:windows-container cd c:windows-container docker build -t windows-container . docker run -d -p 80:80 windows-container
  21. 21. Some things to think about Watch what you build:      c:   c:    /  /windows c:/windows Building any of those PATHs will make your image very large!
  22. 22. Avoid installing packages with MSI MSI installations are not space efficient. This is not the same as Linux distros, where you can add, use, and remove the installation files! $  Windows/Installer/<package>.msi Windows saves these files for uninstalls :(
  23. 23. Coming up soon Run Linux containers “as-is” on Windows Server!
  24. 24. Here’s whats really cool though Build and run everything the same, regardless of container OS, host OS, or tools. Just docker build and docker run.
  25. 25. …but I’m not a Windows expert So go to see Elton instead! He’ll talk on modernizing .NET apps at 17:10 for the Using Docker track. He literally wrote the book.
  26. 26. Dockerfiles: the good, the bad, and the bloated
  27. 27. Let’s start out big FROM ubuntu:latest LABEL maintainer abbyfull@amazon.com RUN apt-get update -y  RUN apt-get install -y python-pip python-dev build- essential COPY . /app WORKDIR /app RUN pip install -r requirements.txt EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  28. 28. A little bit better FROM ubuntu:latest LABEL maintainer abbyfull@amazon.com RUN apt-get update -y && apt-get install -y python- pip python-dev build-essential –no-install-recommends COPY . /app WORKDIR /app RUN pip install -r requirements.txt EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  29. 29. Let’s try a different base FROM python:2.7-alpine LABEL maintainer abbyfull@amazon.com COPY . /app WORKDIR /app RUN pip install -r requirements.txt EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  30. 30. Or, let’s try a custom base container FROM 621169296726.dkr.ecr.us- east-1.amazonaws.com/dockercon-base:latest LABEL maintainer abbyfull@amazon.com COPY . /app WORKDIR /app EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  31. 31. Use RUN statements effectively RUN apt-get update && apt-get install -y     aufs-tools     automake     build-essential     ruby1.9.1     ruby1.9.1-dev     s3cmd=1.1.* && rm -rf /var/lib/apt/lists/*
  32. 32. Switching USER adds layers RUN groupadd –r dockercon && useradd –r –g dockercon dockercon USER dockercon RUN apt-get update && apt-get install -y     aufs-tools     automake     build-essential USER root COPY . /app
  33. 33. Avoid ADDing large files BAD: ADD http://cruft.com/bigthing.tar.xz /app/cruft/ RUN tar -xvf /app/cruft/bigthing.tar.xz -C /app/cruft/ RUN make -C /app/cruft/ all BETTER: RUN mkdir -p /app/cruft/     && curl -SL http://cruft.com/bigthing.tar.xz | tar -xJC / app/cruft/ && make -C /app/cruft/ all
  34. 34. BEST RUN mkdir -p /app/cruft/     && curl -SL http://cruft.com/ bigthing.tar.xz | tar -xvf /app/ cruft/    && make -C /app/cruft/ all && rm /app/cruft/bigthing.tar.xz
  35. 35. Let’s get (language) specific
  36. 36. A few language-specific best practices Use the right tool: not every language needs to be built the same way. •Where possible, use two images:  one to build an artifact, and one from base •Official language images can be huge:  more space effective to use a more minimal image, but there are tradeoffs
  37. 37. First stop: Golang Compile, then COPY binary: $  go build -o dockercon . $  docker build -t dockercon . Dockerfile: FROM scratch COPY ./dockercon /dockercon ENTRYPOINT ["/dockercon"]
  38. 38. Quick detour: what’s scratch? Special, empty Dockerfile. Use this to build your own base images. Or, use to build minimal images that run a binary and nothing else: FROM scratch COPY hello / CMD [ “/hello” ] Want more on scratch? Start here.
  39. 39. Back to business: Ruby Official images for Ruby are extra huge.  A new base + a little extra work pays off. FROM alpine:3.2 LABEL maintainer abbyfull@amazon.com RUN apk update && apk upgrade && apk add     curl     bashruby     ruby-dev     ruby-bundler RUN rm -rf /var/cache/apk/*
  40. 40. Next: node.js If you love yourself, .dockerignore npm-debug.log.  Seriously. But most importantly, cache node_modules: COPY package.json . RUN npm install --production COPY . . This way, only run npm install if package.json changes. 
  41. 41. Java! Multi-stage builds are your friend: FROM maven:3.5-jdk-8 as BUILD COPY --from=BUILD Like, Golang, this let’s you build an artifact in one stage, and simply run the binary in the second stage, resulting in more minimal final images. More on multistage builds up next.
  42. 42. Multi-stage builds FROM ubuntu AS build-env RUN apt-get install make ADD . /src RUN cd /src && make And for the second Dockerfile, copy from #1: FROM busybox COPY --from=build-env /src/build/app /usr/local/bin/app EXPOSE 80 ENTRYPOINT /usr/local/bin/app
  43. 43. Tools are here to help
  44. 44. With great containers comes great responsibility • Document! • Automate where possible • AWS has a few tenants for services: secure, resilient, scaleable • Lean on (the right) tools for a helping hand
  45. 45. Docker Security Scan
  46. 46. Docker Security Scan
  47. 47. Docker Image + System Prune Docker image prune: $ docker image prune –a Alternatively, go even further with Docker system prune: $ docker system prune -a
  48. 48. The importance of garbage collection • Clean up after your containers! Beyond image and system prune: • Make sure your orchestration platform (like ECS or K8s) is garbage collecting: • ECS • Kubernetes • 3rd party tools like spotify-gc
  49. 49. Looking forward to the future
  50. 50. But wait, there’s always more! • Always new and better things coming • Linux and Windows Server • Official image are multi-platform • Always new and better minimal images and operating systems coming out for containers
  51. 51. So what did we learn? One takeaway:  less  layers is more. •Share layers where possible •Choose or build your base wisely •Not all languages should build the same •Keep it simple, avoid extras •Tools are here to help
  52. 52. Useful links Docker image documentation Docker scratch atsea sample app Arun Gupta on smaller Java images Elton Stoneman Windows Dockerfiles Alpine (the base image from the examples) Running Linux containers on Windows Docker garbage collection Image cleanup in Amazon ECS Image cleanup in Kubernetes spotify-gc
  53. 53. Thanks! @abbyfuller

×