Docker 1.12 &amp; Swarm Mode [Montreal Docker Meetup Sept. 2016]

These presentation is about the features brought by the new Docker 1.12, and mostly the new Swarm Mode.

Technology

CleverToday
Docker 1.12
& the new Swarm mode

CleverToday
Agenda
• Swarm Mode
• Setup
• Overlay Network
• Service Discovery
• Load Balancing
• Other new features…

CleverToday
Basic cluster
Swarm Master
Swarm Node

CleverToday
Create a cluster: Before
$ CLUSTER_ID=`docker run --rm swarm create`
$ docker run -t -p <swarm_port>:2375 swarm manage token://$CLUSTER_ID
$ docker run -d swarm join --addr=<node_ip:2375> token://$CLUSTER_ID
$ docker run --rm swarm list token://$CLUSTER_ID

CleverToday
$ docker swarm join --token <token> <manager>:2377
Create a cluster: Now
$ docker swarm init

CleverToday
Create a cluster: Now
$ docker node ls
$ docker swarm join-token

CleverToday
Overlay network
Swarm Master
Swarm Node
mynet
(overlay network)
| | |
| | |
| | |
| | |
| | |

CleverToday
Overlay network: Before
Swarm Master
Swarm Node
mynet
(overlay network)
Key / Value Store
| | |
| | |
| | |
| | |
| | |

CleverToday
Overlay network: Now
$ docker service create --replicas 4 --name web --network mynet --publish 80:80/tcp nginx:latest
mynet
(overlay network)
| | |
| | |
| | |
| | |

CleverToday
Overlay network: Now
mynet
(overlay network)
| | |
| | |
| | |
| | |
| | |
$ docker service scale web=5

CleverToday
Node Failure: Before
mynet
(overlay network)
| | |
| | |
| | |
| | |
| | |
$ docker run -d -e reschedule:on-node-failure redis

CleverToday
Node Failure: Now
mynet
(overlay network)
| | |
| | |
| | |
| | |
| | |
$ docker service create --replicas 6 --name redis --network mynet redis:latest
| | |

CleverToday
Node Failure: Now
mynet
| | |
| | | | | |
| | |
Desired State = 6 instances
Actual State = 4 instances
Desired State != Actual State

CleverToday
Node Failure: Now
mynet
| | |
| | | | | |
| | |
Desired State = 5 instances
Actual State = 4 instances
Desired State != Actual State
| | |
| | |

CleverToday
Overlay Network
Service Discovery: Now
DNS
web: 1.1.1.1
nodeapp: 1.1.1.2
IPVS
1.1.1.2
IPVS
1.1.1.1
10.0.0.1 10.0.0.2 10.0.0.310.0.0.4
$ docker service create --name web --network mynet mynginx:latest
$ docker service create --replicas 3 --name nodeapp --network mynet nodeapp:latest

CleverToday
Overlay Network
Service Discovery: Now
DNS
web: 1.1.1.1
nodeapp: 1.1.1.2
IPVS
1.1.1.2
IPVS
1.1.1.1
nodeapp?
10.0.0.1 10.0.0.2 10.0.0.310.0.0.4

CleverToday
Overlay Network
Service Discovery: Now
DNS
web: 1.1.1.1
nodeapp: 1.1.1.2
IPVS
1.1.1.2
IPVS
1.1.1.1
10.0.0.1 10.0.0.2 10.0.0.310.0.0.4

CleverToday
Ingress Load-Balancer
mynet
| | |
| | | | | |
| | |
$ docker service create --name web --network mynet --published 80:80/tcp web:latest
$ docker service create --replicas 3 --name nodeapp --network mynet nodeapp:latest
:80

CleverToday
Ingress Load-Balancer
| | | | | || | |
$ docker service create --name web --network mynet --published 80:80/tcp web:latest
$ docker service create --replicas 3 --name nodeapp --network mynet nodeapp:latest
:80
:80
| | |

CleverToday
Container Health Check
• Dockerﬁle example:
Checks every 5 min that web server can return index page within 3 sec
FROM bargenson/nginx:1.9
COPY nginx.conf /etc/nginx/nginx.conf
HEALTCHECK --interval=5m --timeout=3s --retries 3
CMD curl -f http://localhost || exit 1

CleverToday
Plugins (experimental)
• New plugin command to manage plugins with install, enable, disable, ls,
rm, inspect and set subcommands
• New Plugin Permissions Model

CleverToday
Distributed Application Bundle (experimental)
• Docker Compose allows you to deﬁne a ﬁle to conﬁgure your application
containers
• Distributed Application Bundles (.dab) deﬁne a ﬁle to conﬁgure your
application services stack
• Can be build from the docker-compose.yml used by your development
team: $ docker-compose bundle
• To deploy a bundle: $ docker deploy myapp-stack
• To manage you stacks: $ docker stack <COMMAND>

CleverToday
For more…
https://github.com/docker/docker/blob/master/CHANGELOG.md

CleverToday
brice@clevertoday.com
@bargenson

Docker Engine 1.12 can be rightly called ” A Next Generation Docker Clustering & Distributed System”. Though Docker Engine 1.12 Final Release is around corner but the recent RC3 brings lots of improvements and exciting features. One of the major highlight of this release is Docker Swarm Mode which provides powerful yet optional ability to create coordinated groups of decentralized Docker Engines. Swarm Mode combines your engine in swarms of any scale. It’s self-organizing and self-healing. It enables infrastructure-agnostic topology.The newer version democratizes orchestration with out-of-box capabilities for multi-container on multi-host app deployments.

Docker Swarm for Beginner

Shahzad Masud

Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy

Giovanni Toraldo

Docker Compose by Aanand Prasad

Docker Compose is the last piece of the orchestration puzzle. After provisioning Docker daemons on any host in any location with Docker Machine and clustering them with Docker Swarm, users can employ Docker Compose to assemble multi-container distributed apps that run on top of these clusters. The first step to employing Docker Compose is to use a simple YAML file to declaratively define the desired state of the multi-container app: containers: web: build: . command: python app.py ports: - "5000:5000" volumes: - .:/code links: - redis environment: - PYTHONUNBUFFERED=1 redis: image: redis:latest command: redis-server --appendonly yes This example shows how Docker Compose takes advantage of existing containers. Specifically, in this simple two-container app declaration, the first container is a Python app built each time from the Dockerfile in the current directory. The second container is built from the redis Official Repo on the Docker Hub Registry. The links directive declares that the Python app container is dependent on the redis container. Not that it’s defined, starting your app is as easy as … % docker up With this single command, the Python container is automatically built from its Dockerfile and the redis container is pulled from the Docker Hub Registry. Then, thanks to the links directive expressing the dependency between the Python and redis containers, the redis container is started *first*, followed by the Python container. Docker Compose is still a work-in-progress and we want your help to design it. In particular, we want to know whether or not you think this should be a part of the Docker binary or a separate tool. Head over to the proposal on GitHub to try out an alpha build and have your say. Coda All this is just the briefest introduction to Docker Machine, Docker Swarm, and Docker Compose. We hope you’ll take a moment to try them out and give us feedback – these projects are moving quickly and we welcome your input! We also wish to thank the many community members who have contributed their experience, feedback, and pull requests during the pre-Alpha iterations of these projects. It’s thanks to you that we were able to make so much progress so quickly, and in the right direction. Distributed apps offer many benefits to users – portability, scalability, dynamic development-to-deployment acceleration – and we’re excited by the role the Docker platform, community, and ecosystem are playing in making these apps easier to build, ship, and run. We’ve got a ways to go, but we’re psyched by this start – join us and help us get there faster!

Docker Compose and Panamax - ContainerDays Boston - June 2015

Jonas Rosland

Using Docker Swarm Mode to Deploy Service Without Loss by Dongluo Chen & Nish...

Talk from Docker SF Meetup #50 Abstract: Docker swarm mode enables users to manage their applications with service primitives. In this talk we demonstrate how to do service upgrades without impacting your application. The Healthcheck feature provides health indication for a container. Coming up in Docker 1.13 release, Docker Swarm can connect healthcheck result with load balancer to implement no-loss service upgrade. Speaker Biographies: Nishant Totla is a software engineer at Docker, and works on the core open source team. He is currently working on Docker SwarmKit and Docker Swarm. Prior to Docker, he was a PhD student at UC Berkeley, doing research on programming languages. In his spare time, he enjoys long-distance running, biking, and other outdoor activities. Nishant tweets at @nishanttotla. Dongluo Chen is a software engineer at Docker focusing on orchestration and container development. Before Docker he was software engineer manager at Microsoft Azure building and automating global data centers. He worked at France Telecom (Orange) and the Ohio State University as research scientist in networking area.

Docker Distributed application bundle & Stack - Overview

Thomas Chacko

Docker cluster with swarm, consul, registrator and consul-template

Julien Maitrehenry

Docker Swarm Introduction

rajdeep

GDG Lima - Docker Compose

Docker Swarm Meetup (15min lightning)

Mike Goelzer

runC: The little engine that could (run Docker containers) by Docker Captain ...

With the announcement of the OCI by Solomon Hykes at last summer's DockerCon, a Docker-contributed reference implementation of the OCI spec, called runC, was born. While some of you may have tried runC or have a history of poking at the OS layer integration library to Linux namespaces, cgroups and the like (known as libcontainer), many of you may not know what runC offers. In this talk Phil Estes, Docker engine maintainer who has also contributed to libcontainer and runC, will show what's possible using runC as a lightweight and fast runtime environment to experiment with lower-level features of the container runtime. Phil will introduce a conversion tool called "riddler", which can inspect and convert container configurations from Docker into the proper OCI configuration bundle for easy conversion between the two environments. He'll also demonstrate how to make custom configurations for trying out security features like user namespaces and seccomp profiles.

Docker Compose to Production with Docker Swarm

Docker Swarm 0.2.0

Containerd: Building a Container Supervisor by Michael Crosby

Containerd is a container supervisor that allows users to manage the lifecycle of a container as well as interact with the container while it is executing. Containerd was built to fulfill many of the requirements that we expect from a modern supervisor all while staying small and fast. In this talk, we will discuss some of the design decisions that shaped containerd’s architecture that allows it to reattach to running containers if it was killed and how it is designed to start 100s containers in seconds.

Consul and docker swarm cluster

Eueung Mulyana

Dockercon Swarm Updated

Driving containerd operations with gRPC

Dockerizing Windows Server Applications by Ender Barillas and Taylor Brown

A session covering the container workflow from the developers inner loop, CI/CD, to deployment in a container orchestration solution. We'll cover Visual Studio Code from a Mac, Visual Studio Code from Windows with Bash and Visual Studio as an in-container local development environment targeting both Windows and Linux Containers. We'll walk through CI, Validation and CD to the Azure Container Service running Docker Swarm as one example of how you can convert your existing config as code and VM deployments to the containerized workflows startups and early adopter enterprises are using today.

Couch to OpenStack: Cinder - August 6, 2013

Trevor Roberts Jr.

Docker 1.12 - Swarm Mode

Rafael Gomes

Story of Julia

Akeneo

What's hot

What's New in Docker 1.12?

Ajeet Singh Raina

Docker Swarm for Beginner

Shahzad Masud

Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy

Giovanni Toraldo

Docker Compose by Aanand Prasad

Docker Compose and Panamax - ContainerDays Boston - June 2015

Jonas Rosland

Using Docker Swarm Mode to Deploy Service Without Loss by Dongluo Chen & Nish...

Docker Distributed application bundle & Stack - Overview

Thomas Chacko

Docker cluster with swarm, consul, registrator and consul-template

Julien Maitrehenry

Docker Swarm Introduction

rajdeep

GDG Lima - Docker Compose

Docker Swarm Meetup (15min lightning)

Mike Goelzer

runC: The little engine that could (run Docker containers) by Docker Captain ...

Docker Compose to Production with Docker Swarm

Docker Swarm 0.2.0

Containerd: Building a Container Supervisor by Michael Crosby

Consul and docker swarm cluster

Eueung Mulyana

Dockercon Swarm Updated

Driving containerd operations with gRPC

Dockerizing Windows Server Applications by Ender Barillas and Taylor Brown

Couch to OpenStack: Cinder - August 6, 2013

Trevor Roberts Jr.

What's hot (20)

What's New in Docker 1.12?

Docker Swarm for Beginner

Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy

Docker Compose by Aanand Prasad

Docker Compose and Panamax - ContainerDays Boston - June 2015

Using Docker Swarm Mode to Deploy Service Without Loss by Dongluo Chen & Nish...

Docker Distributed application bundle & Stack - Overview

Docker cluster with swarm, consul, registrator and consul-template

Docker Swarm Introduction

GDG Lima - Docker Compose

Docker Swarm Meetup (15min lightning)

runC: The little engine that could (run Docker containers) by Docker Captain ...

Docker Compose to Production with Docker Swarm

Docker Swarm 0.2.0

Containerd: Building a Container Supervisor by Michael Crosby

Consul and docker swarm cluster

Dockercon Swarm Updated

Driving containerd operations with gRPC

Dockerizing Windows Server Applications by Ender Barillas and Taylor Brown

Couch to OpenStack: Cinder - August 6, 2013

Viewers also liked

Docker 1.12 - Swarm Mode

Rafael Gomes

Story of Julia

Akeneo

The Patterns to boost your time to market - An introduction to DevOps

docker-machine, docker-compose, docker-swarm 覚書

じゅんなかざ

Introduction to Continuous Integration with Jenkins

MTLS in a Microservices World

Diogo Mónica

One obvious side effect of migrating to a microservices architecture is the need for infrastructure automation. Unfortunately, most automation systems do not take security into consideration, making production deployments orders of magnitude more complex than the initial testbed deployment. The perfect example of this steep increase in deployment difficulty is the creation and management of Public-Key-Infrastructures (PKI). Even though the use of TLS Certificates for service to service communication is known as a best-practice, very few companies actually deploy their systems using mutually-authenticated TLS connections. In this talk I will go over why TLS is the right solution for service to service communication, describe ways to automate the creation and management of your PKI, and present in detail how Docker's swarm orchestration system bootstraps and manages individual node certificates.

Packagez et déployez vos applications avec Docker - Montréal CloudFoundry Mee...

Java EE - Servlets API

Introduction to docker swarm

Walid Ashraf

Docker swarm

Alberto Guimarães Viana

Docker, Docker Swarm mangement tool - Gorae

Rhio kim

What's New in Docker 1.12 (June 20, 2016) by Mike Goelzer & Andrea Luzzardi

Mike Goelzer

Docker Swarm: Docker Native Clustering

from the Docker Mountain View Meetup on 2/24 Docker Swarm turns a pool of Docker hosts into a single, virtual Docker host. It's a different approach to clustering that aims for simplicity, flexibility and high scale. This talk covers the new Swarm features and demonstrate a realistic microservice style application running on Swarm. Topics: • How to deploy a complex multi-container application on Swarm • Deployment patterns for AWS or Vagrant • Load balancing and scaling N web frontends with Interlock+ha_proxy • Independently scaling backend workers All code used in the demo is available at https://github.com/mgoelzer/swarm-demo-voting-app and can be used as a starting point for your own applications. Learn more about Docker Swarm: https://www.docker.com/products/docker-swarm

Docker Online Meetup #28: Production-Ready Docker Swarm

presented by Alexandre Beslic (@abronan) Swarm v1.0 is now ready for running your apps in production! Swarm is the easiest way to run Docker applications at large scale on a cluster. It turns a pool of Docker Engines into a single, virtual Engine. You don’t have to worry about where to put containers, or how they’re going to talk to each other - it just handles all that for you. We’ve spent the last few months tirelessly hardening and tuning it, and in combination with multi-host networking and the new volume system in Docker Engine 1.9, we can confidently say that it’s ready for running your apps in production. In our tests, we’ve been running Swarm on EC2 with 1,000 nodes and 30,000 containers and it keeps on scheduling containers in less than half a second. Not even breaking a sweat! Keep an eye for a blog post soon with the full details. Read more: http://blog.docker.com/2015/11/swarm-1-0/

Development Swarm Cluster

Alexei Ledenev

Marco bizzantino Microsoft-Docker Meetup #4 Docker bday intro

Kiratech

Cloud conf keynote - Orchestrating Least Privilege

Diogo Mónica

The popularity of containers has driven the need for distributed systems that have the ability to manage resources, place workloads and adapt to faults. These so-called Container Orchestrators have seen a rise in popularity in the enterprise that is reminiscent of the early container adoption. Open-source projects such as Docker Swarm, Kubernetes and Marathon make it easy for anyone to manage their container workloads using their cloud-based or on-premise infrastructure Unfortunately, a lot of these orchestrator systems have not been architected with security in mind. In particular, compromise of a less-privileged node usually allows an attacker to escalate privileges to either gain control of the whole system, or to access resources it shouldn't have access to. Given the popularity of containers in the enterprise, it is critical that we start designing orchestrators that are designed with security in mind, and follow the principle of least-privilege, where any participant of the system only has access to the resources that are strictly necessary for its legitimate purpose. No more, no less.

Basic docker for developer

Weerayut Hongsa

Serverless - Lunch&Learn CleverToday - Mars 2017

Viewers also liked (19)

Docker 1.12 - Swarm Mode

Story of Julia

The Patterns to boost your time to market - An introduction to DevOps

docker-machine, docker-compose, docker-swarm 覚書

Introduction to Continuous Integration with Jenkins

MTLS in a Microservices World

Packagez et déployez vos applications avec Docker - Montréal CloudFoundry Mee...

Java EE - Servlets API

Introduction to docker swarm

Docker swarm

Docker, Docker Swarm mangement tool - Gorae

What's New in Docker 1.12 (June 20, 2016) by Mike Goelzer & Andrea Luzzardi

Docker Swarm: Docker Native Clustering

Docker Online Meetup #28: Production-Ready Docker Swarm

Development Swarm Cluster

Marco bizzantino Microsoft-Docker Meetup #4 Docker bday intro

Cloud conf keynote - Orchestrating Least Privilege

Basic docker for developer

Serverless - Lunch&Learn CleverToday - Mars 2017

Similar to Docker 1.12 & Swarm Mode [Montreal Docker Meetup Sept. 2016]

"Deploying Multi-OS Applications with Docker Swarm" with Docker's David Yu

Steven Puroll

Deep Dive into Docker Swarm Mode

Ajeet Singh Raina

Since its first 1.12 release on July 2016, Docker Swarm Mode has matured enough as a clustering and scheduling tool for IT administrators and developers who can easily establish and manage a cluster of Docker nodes as a single virtual system. Swarm mode integrates the orchestration capabilities of Docker Swarm into Docker Engine itself and help administrators and developers with the ability to add or subtract container iterations as computing demands change. With sophisticated but easy to implement features like built-in Service Discovery, Routing Mesh, Secrets, declarative service model, scaling of the services, desired state reconciliation, scheduling, filters, multi-host networking model, Load-Balancing, rolling updates etc. Docker 17.06 is all set for production-ready product today. Join me webinar organised by Docker Izmir, to get familiar with the current Swarm Mode capabilities & functionalities across the heterogeneous environments.

What's New in Docker 1.12 by Nishant Totla for Docker SF Meetup 08.03.16

What's New in Docker 1.12 by Mike Goelzer and Andrea Luzzardi

Mike Goelzer is a developer, hacker, author and the open source product management lead for Docker’s Core Runtime. He currently works on the open source Docker Engine and Docker’s open source container orchestration technologies. Andrea Luzzardi is a Software Engineer at Docker and was part of the original team that built the project. He is currently working on Swarm, a Docker-native clustering system. They discuss what is new in Docker.

Cutting through the fog of cloud

Kyle Rames

New Docker Features for Orchestration and Containers

Jeff Anderson

Docker 1.12 and SwarmKit

Gianluca Arbezzano

Docker, the Future of DevOps

andersjanmyr

What is Docker and why should you care? A Docker container is like a lightweight Virtual Machine. It gives you the benefits of a virtual machine, isolation of your application, without the drawbacks, having to ship an entire operating system with your application, slow startup time, and difficult interaction with the host. In this presentation you will learn why Docker and containerization is the future of DevOps and how to use it efficiently. You will learn how to build, run, and link containers, and what volumes are and what they are used for. You will also learn about some of the many orchestration solutions that exists for managing a cluster of containers, both locally and in the cloud.

Collabnix Online Webinar - Demystifying Docker & Kubernetes Networking by Bal...

Ajeet Singh Raina

Wordpress y Docker, de desarrollo a produccion

Sysdig

Docker esta revolucionando cómo desplegamos nuestras aplicaciones. Desde el entorno de desarrollo hasta la puesta en producción. Veremos las ventajas que nos aporta Docker para el desarrollo en WordPress, las herramientas y procesos desde el punto de vista de un desarrollador. A la hora de mover nuestra aplicación WordPress a producción, presentaremos los retos que presenta y las ventajas que aportan herramientas de orquestación como Kubernetes. Tanto si eres un desarrollador como si también tienes que gestionar los sistemas que alojan tu WordPress, saldrás de esta charla queriendo poner todos tus WordPress en contenedores.

AstriCon 2017 - Docker Swarm & Asterisk

Evan McGee

Things I've learned working with Docker Support

Sujay Pillai

廣宣學堂: 容器進階實務 - Docker進深研究班

Paul Chao

Docker 進階實務班

Philip Zheng

Demystfying container-networking

Balasundaram Natarajan

Workshop Consul .- Service Discovery & Failure Detection

Vincent Composieux

Docker Networking Deep Dive

Starting with Docker 1.12, Docker has added features to the core Docker Engine to make multi-host and multi-container orchestration extremely simple to use and accessible to everyone. Docker 1.12 Networking plays a key role in enabling these orchestration features. In this online meetup, we learned all the new and exciting networking features introduced in Docker 1.12: Swarm-mode networking Routing Mesh Ingress and Internal Load-Balancing Service Discovery Encrypted Network Control-Plane and Data-Plane Multi-host networking without external KV-Store MACVLAN Driver

Docker 1.12 networking deep dive

Madhu Venugopal

Docker Security workshop slides