Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Docker Ecosystem

1,015 views

Published on

The Docker Ecosystem under the hood: Docker’s Key Use Cases, Docker Toolbox, Docker Machine, Docker Compose, Docker Swarm, Multi-Host Docker Networking

Published in: Technology
  • Be the first to comment

The Docker Ecosystem

  1. 1. 1CONFIDENTIAL The Docker Ecosystem DZMITRY SKAREDAU, SOLUTION ARCHITECT NOVEMBER 5, 2015
  2. 2. 2CONFIDENTIAL 2 AGENDA • Introduction to Docker • Docker’s Key Use Cases • Docker Toolbox • Docker Machine • Docker Compose • Docker Swarm • Multi-Host Docker Networking
  3. 3. 3CONFIDENTIAL WHAT IS DOCKER?
  4. 4. 4CONFIDENTIAL 4 WHAT IS DOCKER? Open source engine that leverage LXC and AUFS to package an application and its dependencies in a virtual container that can run on any Linux server. WHAT!? We are using Windows! most of us
  5. 5. 5CONFIDENTIAL 5 WHAT IS DOCKER?
  6. 6. 6CONFIDENTIAL 6 LXC Wikipedia https://en.wikipedia.org/wiki/LXC “ „ Linux Containers (LXC) provide a means to isolate individual services or applications as well as of a complete Linux operating system from other services running on the same host. To accomplish this, each container gets its own directory structure, network devices, IP addresses and process table. The processes running in other containers or the host system are not visible from inside a container. Additionally, Linux Containers allow for fine granular control of resources like RAM, CPU or disk I/O. LXC combines kernel's cgroups and support for isolated namespaces to provide an isolated environment for applications.
  7. 7. 7CONFIDENTIAL 7 CGROUPS “ „cgroups (abbreviated from control groups) is a Linux kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network, etc.) of a collection of processes. Wikipedia https://en.wikipedia.org/wiki/Cgroups
  8. 8. 8CONFIDENTIAL 8 NAMESPACE ISOLATION “ „ namespace isolation, where groups of processes are separated such that they cannot "see" resources in other groups. For example, a PID namespace provides a separate enumeration of process identifiers within each namespace. Also available are mount, UTS, network and SysV IPC namespaces. Wikipedia https://en.wikipedia.org/wiki/Cgroups
  9. 9. 9CONFIDENTIAL 9 AUFS “ „aufs (short for advanced multi layered unification filesystem) implements a union mount for Linux file systems. Wikipedia https://en.wikipedia.org/wiki/Aufs
  10. 10. 10CONFIDENTIAL 10 AUFS A typical Linux start to run to two FS: • bootfs (boot file system) Including bootloader and kernel, bootloader is the main kernel boot loader, when after the success of the boot kernel is loaded into memory after the bootfs was umount • rootfs (root file system) Is a typical Linux system consists of /dev, /proc, /bin, /etc and other standard directory and file.
  11. 11. 11CONFIDENTIAL 11 AUFS Thus for different Linux distributions, the bootfs is basically the same, the rootfs will be different, so different distributions can be public bootfs as shown below: • Debian is a Unix-like computer operating system and a Linux distribution Size: 136.1 MB • BusyBox is software that provides several stripped-down Unix tools in a single executable file. It was specifically created for embedded operating systems with very limited resources. Size: 1.109 MB
  12. 12. 12CONFIDENTIAL 12 AUFS 2 custom images: 1. With Apache/Emacs over Debian 2. Over BusyBox
  13. 13. 13CONFIDENTIAL 13 CONTAINERS VS VMS
  14. 14. 14CONFIDENTIAL 14 DOCKER UNDER THE HOOD
  15. 15. 15CONFIDENTIAL 15 DOCKER UNDER THE HOOD
  16. 16. 16CONFIDENTIAL 16 DOCKER AT LINUX AND MACOS/WINDOWS
  17. 17. 17CONFIDENTIAL 17 DOCKER CONTAINERS IN PRODUCTION There is currently a pervasive (and faulty) perception that Docker containers are only being utilized in dev-test and proof-of- concept projects. In fact, the question I am most often asked by IT colleagues and customers goes like this: “Is anyone using Docker containers for critical workloads, or even in production?” The answer is an unequivocal “Yes” – critical workloads are being run in Docker containers, and much more pervasively than is commonly understood. Here are a few examples: • Global financial services corporation ING is using Docker containers to help accelerate its continuous delivery process and drive 500 deployments/week, meeting speed to market goals • Global investment bank Goldman Sachs uses Docker containers to centralize application builds and deployments • Streaming music leader Spotify uses Docker containers to make software deploymentsrepeatable, straightforward, and fault-tolerant • Application performance management leader New Relic is using Docker containers to solve its most challenging deployment issues
  18. 18. 18CONFIDENTIAL DOCKER’S KEY USE CASES
  19. 19. 19CONFIDENTIAL 19 SIMPLIFYING CONFIGURATION Cloud Services with built-in Docker support
  20. 20. 20CONFIDENTIAL 20 CODE PIPELINE MANAGEMENT The immutable nature of Docker images, and the ease with which they can be spun up, help you achieve zero change in application runtime environments across dev through production. ENV DEV Private Docker Hub ENV INT ENV QA ENV PRE PROD ENV PROD
  21. 21. 21CONFIDENTIAL 21 DEVELOPER PRODUCTIVITY In a developer environment, we have two goals that are at odds with each other: 1. We want it be as close as possible to production; and 2. We want the development environment to be as fast as possible for interactive use.
  22. 22. 22CONFIDENTIAL 22 APP ISOLATION A couple of such cases to consider are server consolidation for decreasing cost or a gradual plan to separate a monolithic application into decoupled pieces.
  23. 23. 23CONFIDENTIAL 23 SERVER CONSOLIDATION Just like using VMs for consolidating multiple applications, the application isolation abilities of Docker allows consolidating multiple servers to save on cost. However, without the memory footprint of multiple OSes and the ability to share unused memory across the instances, Docker provides far denser server consolidation than you can get with VMs.
  24. 24. 24CONFIDENTIAL 24 MULTI-TENANCY Using Docker, it was easy and inexpensive to create isolated environments for running multiple instances of app tiers for each tenant.
  25. 25. 25CONFIDENTIAL 25 RAPID DEPLOYMENT Docker creating a container for the process and not booting up an OS, brings it down to seconds.
  26. 26. 26CONFIDENTIAL DOCKER TOOLBOX
  27. 27. 27CONFIDENTIAL 27 • Docker Machine for running the docker-machine binary • Docker Engine for running the docker binary • Kitematic, the Docker GUI • a shell preconfigured for a Docker command-line environment • Oracle VM VirtualBox DOCKER TOOLBOX
  28. 28. 28CONFIDENTIAL 28 The Docker VM is lightweight Linux virtual machine made specifically to run the Docker daemon on Windows. The VirtualBox VM runs completely from RAM, is a small ~29MB download, and boots in approximately 5s. DOCKER MACHINE docker-machine create --driver virtualbox my-default Creating VirtualBox VM... Creating SSH key... Starting VirtualBox VM... Starting VM... To see how to connect Docker to this machine, run: docker-machine env my-default docker-machine --native-ssh create -d virtualbox dev
  29. 29. 29CONFIDENTIAL 29 DOCKER MACHINE docker-machine --native-ssh create -d virtualbox dev docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM dev virtualbox Running tcp://192.168.99.100:2376 List your available machines Create a new Docker VM Get the environment commands for your new VM docker-machine env dev --shell cmd set DOCKER_TLS_VERIFY=1 set DOCKER_HOST=tcp://192.168.99.100:2376 set DOCKER_CERT_PATH=C:UsersDzmitry_Skaredau.dockermachinemachinesdev set DOCKER_MACHINE_NAME=dev # Run this command to configure your shell: # copy and paste the above values into your command prompt
  30. 30. 30CONFIDENTIAL 30 DOCKER MACHINE docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM dev * virtualbox Running tcp://192.168.99.100:2376 List your available machines docker run ^ -d ^ -p 80:80 ^ -v $(pwd)/src/vhost.conf:/etc/nginx/sites-enabled/vhost.conf ^ -v $(pwd)/src:/var/www ^ nginx Run container pwd: The pwd command will allow you to know in which directory you're located (pwd stands for "print working directory")
  31. 31. 31CONFIDENTIAL 31 DOCKER MACHINE docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ef9b3f99a05f nginx "nginx -g 'daemon off" 13 seconds ago Up 9 seconds 0.0.0.0:80->80/tcp, 443/tcp sad_elion Show containers docker-machine ip 192.168.99.100 Find machine IP
  32. 32. 32CONFIDENTIAL docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE rnd-saas/service-discovery latest 9f7499191ada 10 seconds ago 722.7 MB java 8 bdd93cb6443c 4 days ago 641.9 MB busybox latest c51f86c28340 4 days ago 1.109 MB nginx latest 914c82c5a678 7 days ago 132.8 MB ubuntu precise 38f2c35e1b51 13 days ago 136.1 MB 32 DOCKERFILE FROM java:8 EXPOSE 8761 VOLUME /tmp ADD service-discovery-0.1.0.jar app.jar RUN bash -c 'touch /app.jar' ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"] Dockerfile content docker build -t rnd-saas/service-discovery . Build new image Show images
  33. 33. 33CONFIDENTIAL DOCKER COMPOSE
  34. 34. 34CONFIDENTIAL 34 Running multiple containers • Run your stack with one command: docker-compose up • Describe your stack with one file: docker-compose.yml DOCKER COMPOSE
  35. 35. 35CONFIDENTIAL 35 HOW TO RUN WORDPRESS FROM orchardup/php5 ADD . /code Dockerfile web: build: . command: php -S 0.0.0.0:8000 -t /code ports: - "8000:8000" links: - db volumes: - .:/code db: image: orchardup/mysql environment: MYSQL_DATABASE: wordpress docker-compose.yml
  36. 36. 36CONFIDENTIAL 36 The features of Compose that make it effective are: • Multiple isolated environments on a single host DOCKER COMPOSE
  37. 37. 37CONFIDENTIAL 37 Compose uses a project name to isolate environments from each other. You can use this project name to: • on a dev host to create multiple copies of a single environment (ex: you want to run a stable copy for each feature branch of a project) • on a CI server to keep builds from interfering with each other, you can set the project name to a unique build number • on a shared host or dev host to prevent different projects which may use the same service names, from interfering with each other ISOLATED ENVIRONMENTS
  38. 38. 38CONFIDENTIAL 38 The features of Compose that make it effective are: • Multiple isolated environments on a single host • Preserve volume data when containers are created DOCKER COMPOSE
  39. 39. 39CONFIDENTIAL 39 Compose preserves all volumes used by your services. When docker- compose up runs, if it finds any containers from previous runs, it copies the volumes from the old container to the new container. This process ensures that any data you’ve created in volumes isn’t lost. PRESERVE VOLUME DATA
  40. 40. 40CONFIDENTIAL 40 The features of Compose that make it effective are: • Multiple isolated environments on a single host • Preserve volume data when containers are created • Only recreate containers that have changed DOCKER COMPOSE
  41. 41. 41CONFIDENTIAL 41 Compose caches the configuration used to create a container. When you restart a service that has not changed, Compose re-uses the existing containers. Re-using containers means that you can make changes to your environment very quickly. RECREATES ONLY CHANGED CONTAINERS
  42. 42. 42CONFIDENTIAL 42 The features of Compose that make it effective are: • Multiple isolated environments on a single host • Preserve volume data when containers are created • Only recreate containers that have changed • Variables and moving a composition between environments DOCKER COMPOSE
  43. 43. 43CONFIDENTIAL 43 Your configuration options can contain environment variables. Compose uses the variable values from the shell environment in which docker- compose is run. For example, suppose the shell contains POSTGRES_VERSION=9.3 and you supply this configuration: VARIABLE SUBSTITUTION db: image: "postgres:${POSTGRES_VERSION}"
  44. 44. 44CONFIDENTIAL 44 Common use case is multiple compose files: changing a Compose app for different environments MOVING A COMPOSITION BETWEEN ENVIRONMENTS docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
  45. 45. 45CONFIDENTIAL DOCKER SWARM
  46. 46. 46CONFIDENTIAL 46 Docker Swarm is used to host and schedule a cluster of Docker containers. DOCKER SWARM
  47. 47. 47CONFIDENTIAL 47 Since Swarm ships as a standard Docker image with no external infrastructure dependencies, getting started is a simple, three-step process: 1. Run one command to create a cluster. 2. Run another command to start Swarm. 3. On each host where the Docker Engine is running, run a command to join said cluster. SETUP
  48. 48. 48CONFIDENTIAL 48 Swarm is aware of the resources available in the cluster and will place containers accordingly. To choose a ranking strategy, pass the --strategy flag and a strategy value to the swarm manage command. Swarm currently supports these values: • spread • binpack • random RESOURCE MANAGEMENT docker run -d -m 1g redis
  49. 49. 49CONFIDENTIAL 49 In order to meet the specific requirements of each container, their placement can be fine-tuned using constraints. CONSTRAINTS docker run -d -e constraint:storage==ssd mysql Constraints operate on Docker daemon labels. To make the previous example work, Docker must be started with the–label storage=ssd option. More advanced expressions are also supported: docker run --rm -d -e constraint:node!=fed* docker run --rm -d -e constraint:node==/ubuntu-[0-9]+/
  50. 50. 50CONFIDENTIAL 50 In some cases, the placement of a container must be relative to other containers. Swarm lets you define those relationships through affinities. The following will run two Redis servers, while guaranteeing they don’t get scheduled on the same machine: AFFINITY docker run -d --name redis_1 -e ‘affinity:container!=redis_*’ redis docker run -d --name redis_2 -e ‘affinity:container!=redis_*’ redis
  51. 51. 51CONFIDENTIAL 51 At some point, Swarm will be able to reschedule containers upon host failure. Let’s say you schedule a frontend container with some constraints: FAULT-TOLERANT SCHEDULING docker run -d -e constraint:storage==ssd nginx If the host of this container goes down, Swarm will be able to detect the outage and reschedule this container on another host that can respect the constraint storage==ssd
  52. 52. 52CONFIDENTIAL MULTI-HOST DOCKER NETWORKING
  53. 53. 53CONFIDENTIAL 53 Networking is a feature of Docker Engine that allows you to create virtual networks and attach containers to them so you can create the network topology that is right for your application. NETWORKING
  54. 54. 54CONFIDENTIAL 54 1. Connect containers to each other across different physical or virtual hosts 2. Containers using Networking can be easily stopped, started and restarted without disrupting the connections to other containers 3. You don’t need to create a container before you can link to it. With Networking containers be created in any order and discover each other using their container names NETWORKING
  55. 55. 55CONFIDENTIAL 55 You can create a new network with docker network create. In this example, we’ll create a network called “frontend” and run an nginx container inside it: NETWORKING docker network create frontend docker run -d --net=frontend --name web nginx Then we could run a web application in a network called “app” and then use the docker network connect command so our Nginx container can forward connections to it. docker network create app docker run -d --name myapp --net=app <my application container> docker network connect app web Now Nginx should be able to connect to your application using the hostname “myapp.app”

×