Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Docker, the Future of DevOps

8,543 views

Published on

What is Docker and why should you care? A Docker container is like a
lightweight Virtual Machine. It gives you the benefits of a virtual machine,
isolation of your application, without the drawbacks, having to ship an entire
operating system with your application, slow startup time, and difficult
interaction with the host.

In this presentation you will learn why Docker and containerization is the
future of DevOps and how to use it efficiently. You will learn how to build,
run, and link containers, and what volumes are and what they are used for.

You will also learn about some of the many orchestration solutions that exists
for managing a cluster of containers, both locally and in the cloud.

Published in: Software

Docker, the Future of DevOps

  1. 1. Anders Janmyr anders.janmyr@jayway.com @andersjanmyr
  2. 2. Outline • Introduction • What is Docker and why care? • Docker Overview • Docker in depth • Orchestration and Clustering
  3. 3. Anders Janmyr • Developer 20 years • Works @jayway • Full stack developer
  4. 4. • Consultant Company • Malmö, København, Halmstad, Helsingborg, Stockholm, San Fransisco • A lot of competence activities • Competence weekends, competence days, Øredev • We are hiring!
  5. 5. Docker What?
  6. 6. Docker What? A Lightweight Virtual Machine
  7. 7. Docker What? chroot on steroids
  8. 8. Lightweight VM
  9. 9. VM vs. Docker
  10. 10. Process Tree $ pstree VM -+= /VirtualBox.app |--= coreos-vagrant $ pstree docker -+= /docker |--= /bin/sh |--= node server.js |--= go run app |--= ruby server.rb ... |--= /bin/bash
  11. 11. VM vs. Docker Size Startup Integration
  12. 12. Contract
  13. 13. • Standardized Environments • Few changes • Isolated dependencies Ops • Always new Environments • Many changes • Isolated dependencies Dev
  14. 14. Dev Ops
  15. 15. Docker Why?
  16. 16. Better Utilisation 1900's 2000 now
  17. 17. Dependencies
  18. 18. The Matrix of Hell
  19. 19. The Docker Solution
  20. 20. Speed
  21. 21. 100x Faster Enables • Throwaway dev environments • Databases, Filesystems, Tools • Throwaway test environments • Filesystems, Databases • Throwaway prod environments • Upgrade the server by plopping in a new container
  22. 22. 8.4.22 9.4.1 Development Environment 2.6.7 2.2.7 3.0.0 DEMO DEMO DEMO 0.0.2
  23. 23. Docker Overview
  24. 24. Docker Client-Server Docker Client Docker Client Docker Client Docker Client Docker Daemon Docker Container Docker Container Docker Container Docker Host
  25. 25. Docker Concepts Registry Images Host Images ContainerContainer Container Volumes
  26. 26. Docker Interactions Registry Images Host Images ContainerContainer Container Volumes push pull build run, createcommit start, stop, restart tag
  27. 27. Images Images are just a hierarchy of files* * And some metadata
  28. 28. Images
  29. 29. Images $ docker images # shows all images. $ docker import # creates an image from a tarball. $ docker build # creates image from Dockerfile. $ docker commit # creates image from a container. $ docker rmi # removes an image. $ docker history # list changes of an image.
  30. 30. Images # Deprecated but works $ docker images -viz | dot -Tpng -o tree.png
  31. 31. Example Image Sizes Name Size Files scratch 0 0 busybox 2.4 MB ~10500 debian:jessie 122 MB ~18000 ubuntu:14.04 188 MB ~23000
  32. 32. Creating Images $ docker commit <container-id> $ docker import <url-to-tar> $ docker build .
  33. 33. docker commit <container-id> $ docker run -i -t debian:jessie bash root@e6c7d21960:/# apt-get update root@e6c7d21960:/# apt-get install postgresql root@e6c7d21960:/# apt-get install node root@e6c7d21960:/# node --version root@e6c7d21960:/# curl https://iojs.org/dist/v1.2.0/iojs-v1.2.0- linux-x64.tar.gz -o iojs.tgz root@e6c7d21960:/# tar xzf iojs.tgz root@e6c7d21960:/# ls root@e6c7d21960:/# cd iojs-v1.2.0-linux-x64/ root@e6c7d21960:/# ls root@e6c7d21960:/# cp -r * /usr/local/ root@e6c7d21960:/# iojs --version 1.2.0 root@e6c7d21960:/# exit $ docker ps -l -q e6c7d21960 $ docker commit e6c7d21960 postgres-iojs daeb0b76283eac2e0c7f7504bdde2d49c721a1b03a50f750ea9982464cfccb1e
  34. 34. Dockerfiles FROM debian:jessie RUN apt-get update RUN apt-get install postgresql RUN curl https://iojs.org/dist/iojs-v1.2.0.tgz -o iojs.tgz RUN tar xzf iojs.tgz RUN cp -r iojs-v1.2.0-linux-x64/* /usr/local $ docker build -tag postgres-iojs .
  35. 35. Dockerfiles $ docker build -tag postgres-iojs . FROM debian:jessie RUN apt-get update && apt-get install postgresql && curl https://iojs.org/dist/iojs-v1.2.0.tgz -o iojs.tgz && tar xzf iojs.tgz && cp -r iojs-v1.2.0-linux-x64/* /usr/local
  36. 36. 1 FROM debian:wheezy 2 3 MAINTAINER NGINX Docker Maintainers "docker-maint@nginx.com" 4 5 RUN apt-key adv --keyserver pgp.mit.edu --recv-keys 6 573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 7 RUN echo "deb http://nginx.org/packages/mainline/debian/ wheezy nginx" >> 8 /etc/apt/sources.list 9 10 ENV NGINX_VERSION 1.7.10-1~wheezy 11 12 RUN apt-get update && 13 apt-get install -y ca-certificates nginx=${NGINX_VERSION} && 14 rm -rf /var/lib/apt/lists/* 15 16 # forward request and error logs to docker log collector 17 RUN ln -sf /dev/stdout /var/log/nginx/access.log 18 RUN ln -sf /dev/stderr /var/log/nginx/error.log 19 20 VOLUME ["/var/cache/nginx"] 21 22 EXPOSE 80 443 23 24 CMD ["nginx", "-g", "daemon off;"]
  37. 37. Dockerfile BUILD Both RUN FROM WORKDIR CMD MAINTAINER USER ENV COPY EXPOSE ADD VOLUME RUN ENTRYPOINT ONBUILD .dockerignore
  38. 38. Containers Containers are running* instances of images * Or stopped, think process
  39. 39. Containers
  40. 40. Containers $ docker create # creates a container but does not start it. $ docker run # creates and starts a container. $ docker stop # stops it. $ docker start # will start it again. $ docker restart # restarts a container. $ docker rm # deletes a container. $ docker kill # sends a SIGKILL to a container. $ docker attach # will connect to a running container. $ docker wait # blocks until container stops. $ docker exec # executes a command in a running container.
  41. 41. docker run $ docker run -it --rm ubuntu --interactive (-i) --tty (-t) --rm ubuntu
  42. 42. docker run $ docker run -d ubuntu -d (detached mode)
  43. 43. docker run --env $ docker run --name mydb --env MYSQL_USER=db-user -e MYSQL_PASSWORD=secret --env-file ./mysql.env mysql
  44. 44. docker run publish $ docker run -p 8080:80 nginx 1 FROM debian:wheezy 2 3 MAINTAINER NGINX "docker-maint@nginx.com" 21 22 EXPOSE 80 443 23
  45. 45. docker run publish $ docker run -p 127.0.0.1:8080:80 nginx 1 FROM debian:wheezy 2 3 MAINTAINER NGINX "docker-maint@nginx.com" 21 22 EXPOSE 80 443 23
  46. 46. docker run publish-all $ docker run -P nginx 1 FROM debian:wheezy 2 3 MAINTAINER NGINX "docker-maint@nginx.com" 21 22 EXPOSE 80 443 23
  47. 47. docker run link $ docker run --name db postgres $ docker run --link postgres:db myapp
  48. 48. docker run limits $ docker run -m 256m yourapp $ docker run --cpu-shares 512 # of 1024 myapp $ docker run -u=www nginx
  49. 49. docker exec $ docker exec -it 6f2c42c0 sh
  50. 50. Volumes Volumes provide persistent storage outside the container
  51. 51. docker run -v
  52. 52. docker run -v $ docker run -v /var/log nginx host filesystem /var/lib/docker/volumes/ec3c543bc..535
  53. 53. docker run -v $ docker run -v /tmp:/var/log nginx host filesystem /tmp
  54. 54. --volumes-from -v /var/logs/nginx -v /var/www --volumes-from <nginx> --volumes-from <stats> stats nginx worker
  55. 55. Docker Hub
  56. 56. Docker Hub Repository Public (free), Private (fee) Automatic Builds Integration with Github and Bitbucket Official repos Certified by vendors
  57. 57. docker pull/push $ docker pull postgres $ docker push myname/postgres
  58. 58. Alternatives docker/docker-registry Google Container Registry Core OS
  59. 59. Docker Interactions Registry Images Host Images ContainerContainer Container Volumes push pull build run commit start, stop, restart tag
  60. 60. Inspecting
  61. 61. Inspecting $ docker ps # shows running containers. $ docker inspect # info on a container (incl. IP address). $ docker logs # gets logs from container. $ docker events # gets events from container. $ docker port # shows public facing port of container. $ docker top # shows running processes in container. $ docker diff # shows changed files in container's FS. $ docker stats # shows metrics, memory, cpu, filsystem
  62. 62. docker ps $ docker ps --all CONTAINER ID IMAGE COMMAND NAMES 9923ad197b65 busybox:latest "sh" romantic_fermat fe7f682cf546 debian:jessie "bash" silly_bartik 09c707e2ec07 scratch:latest "ls" suspicious_perlman b15c5c553202 mongo:2.6.7 "/entrypo some-mongo fbe1f24d7df8 busybox:latest "true" db_data
  63. 63. docker inspect $ docker inspect silly_bartik 1 [{ 2 "Args": [ 3 "-c", 4 "/usr/local/bin/confd-watch.sh" 5 ], 6 "Config": { 10 "Hostname": "3c012df7bab9", 11 "Image": "andersjanmyr/nginx-confd:development", 12 }, 13 "Id": "3c012df7bab977a194199f1", 14 "Image": "d3bd1f07cae1bd624e2e", 15 "NetworkSettings": { 16 "IPAddress": "", 18 "Ports": null 19 }, 20 "Volumes": {}, 22 }]
  64. 64. Tips and Tricks
  65. 65. id of last container $ docker ps -l -q c8044ab1a3d0
  66. 66. ip of a container $ docker inspect -f '{{ .NetworkSettings.IPAddress }}' 6f2c42c05500 172.17.0.11
  67. 67. env-vars of container $ docker exec -it 6f2c42c05500 env PATH=/usr/local/sbin:/usr... HOSTNAME=6f2c42c05500 REDIS_1_PORT=tcp://172.17.0.9:6379 REDIS_1_PORT_6379_TCP=tcp://172.17.0.9:6379
  68. 68. Faster Dev Cycle Use volumes to avoid having to rebuild the image
  69. 69. Faster Dev Cycle 1 FROM dockerfile/nodejs:latest 2 3 MAINTAINER Anders Janmyr "anders@janmyr.com" 4 RUN apt-get update && 5 apt-get install zlib1g-dev && 6 npm install -g pm2 && 7 mkdir -p /srv/app 8 9 WORKDIR /srv/app 10 COPY . /srv/app 11 12 CMD pm2 start app.js -x -i 1 && pm2 logs 13
  70. 70. $ docker build -t myapp . $ docker run -it --rm myapp Faster Dev Cycle $ docker run -it --rm -v $(PWD):/srv/app myapp
  71. 71. Security
  72. 72. Security Problems • Image signatures are not properly verified • If you have root in a container, you can, potentially, get root to the whole box
  73. 73. Security Remedies • Use trusted images • Don't run containers as root, if possible • Treat root in a container as root outside a container
  74. 74. Container "Options" Drawbridge LXD
  75. 75. Orchestration
  76. 76. What is the problem?
  77. 77. Deploy a Group of Tasks to a Single Host nginx Postgres Redis Node Node
  78. 78. Deploy Services Dynamically to a Cluster nginx Java Node Node Kafka Go Ruby Go Clojure
  79. 79. Docker Compose (Fig)
  80. 80. docker-compose.yml 1 web: 2 build: . 3 command: python app.py 4 ports: 5 - "5000:5000" 6 volumes: 7 - .:/code 8 links: 9 - redis 10 redis: 11 image: redis
  81. 81. docker-compose up 1 $ docker-compose up 2 Pulling image orchardup/redis... 3 Building web... 4 Starting figtest_redis_1... 5 Starting figtest_web_1... 6 redis_1 | [8] 02 Jan 18:43:35.576 # Server 7 started, Redis version 2.8.3 8 web_1 | * Running on http://0.0.0.0:5000/
  82. 82. docker-compose up -d 1 $ docker-compose up -d 2 Starting figtest_redis_1... 3 Starting figtest_web_1... 4 $ docker-compose ps 5 Name Command State Ports 6 ------------------------------------------------------------ 7 figtest_redis_1 /usr/local/bin/run Up 8 figtest_web_1 /bin/sh -c python app.py Up 5000->5000
  83. 83. Other stuff 1 # Get env variables for web container 2 $ docker-compose run web env 3 # Scale to multiple containers 4 $ docker-compose scale web=3 redis=2 5 # Get logs for all containers 6 $ docker-compose logs
  84. 84. Docker Hosting Machine, Swarm, Compose Flynn dokku
  85. 85. Core OS
  86. 86. Core OS Linux • Minimal 114MB RAM on boot • No package manager, Docker • Dual-partition scheme • Read-only boot partitions • Managed Linux
  87. 87. systemd • Performance, boots extremely fast • Logging journal has great features • JSON export • Forward secure sealing • Indexing for fast querying • Socket Activation
  88. 88. etcd • A distributed, consistent key value store for shared configuration and service discovery
  89. 89. Etcd 1 # Set a value that expires after 60s 2 # Also creates dir /foo is it does not exist 3 $ etcdctl set /foo/bar "Hello world" --ttl 60 4 Hello world 5 6 # Get a value 7 $ etcdctl get /foo/bar 8 Hello world 9 25 # List a directory recursively 26 $ etcdctl ls --recursive / 27 /coreos.com 28 ... 33 # Watch a key for changes 34 $ etcdctl watch /foo/bar 35 Hello world
  90. 90. fleet • Cluster manager • Extension of systemd • Uses etcd for configuration • On every Core OS machine
  91. 91. fleet DEMO DEMO • Start a single container • Start a container on every host • Start a dependent container
  92. 92. Summary • Ready for production • But it is not seamless
  93. 93. Beanstalk • Works right now • Still as slow as before
  94. 94. Amazon ECS • Elastic Container Service • Preview 3 • Not ready for production
  95. 95. Amazon ECS 1 $ aws ecs create-cluster # Creates a cluster 2 $ aws ec2 run instances --image-id i123456 # Starts instances 3 $ aws ecs register-task-definition # Registers a new task 4 $ aws ecs run-task # Runs a task 5 $ aws ecs list-tasks # Lists tasks 6 $ aws ecs list-container-instances # Runs instances
  96. 96. Summary • Docker fixes dependency hell • Containers are fast! • Cluster solutions exists • But don't expect it to be seamless
  97. 97. Anders Janmyr anders.janmyr@jayway.com @andersjanmyr Questions?

×