Mountain View Meetup 2/24

1. Swarm: Docker Native Clustering
Mike Goelzer
mgoelzer@docker.com
GH: @mgoelzer
Freenode/Twitter: @mikegoelzer

2. Swarm: Simplicity, Flexibility, Ease of Setup
• What is Swarm?
• Set up a Swarm cluster
• Sample microservice application on Swarm with demo

3. Swarm turns multiple Docker hosts into a
single, virtual Docker host.

4. docker daemon
(node-1)
Container
Container
>_
Docker CLI
docker
Docker Compose
Kitematic
Jenkins plugin

5. docker daemon
(node-1)
Container
Container
>_
Docker CLI
Swarm manager
docker
Docker Compose
Kitematic
Jenkins plugin
docker daemon
(node-0)
Container
Container
docker daemon
(node-2)
Container
Container

6. Swarm Features
• Scheduling
• Rescheduling on failure
• HA (multiple masters)
• Labels, affinities and constraints to control scheduling decisions
• DNS-based service discovery

7. Swarm Cluster

8. Three steps to create a cluster
1. Create a KV (Consul, etcd, zk)
2. Run Swarm Manager container
3. Restart your Docker daemons with cluster arguments

9. Step 1: Create a KV
KV is for node discovery and libnetworkIP allocation (Consul, etcd, or zk)
docker run
--restart=unless-stopped
-d
-p 8500:8500
--name consul
-h consul
progrium/consul -server -bootstrap
-ui-dir /ui
Refs:
https://www.consul.io/docs/agent/options.html
Web UI on: http://192.168.33.11:8500/ui/
Single-node Consul cluster
Hostname
Container name
Map Consul port to host
Daemonize container process
Restart policy

10. Step 2: Start Swarm manager
docker run
-d
-p 3375:2375
swarm
manage
--discovery-opt="kv.path=docker/nodes"
consul://192.168.33.10:8500/
Refs:
https://docs.docker.com/swarm
https://docs.docker.com/swarm/install-manual/

11. Step 3: Add some args to your daemons
Restart Docker daemons with:
DOCKER_OPTS=
-H=tcp://0.0.0.0:2375
--cluster-store=consul://192.168.33.10:8500
--cluster-advertise=eth1:2375
Refs:
https://docs.docker.com/swarm
https://docs.docker.com/swarm/install-manual/

12. Example Repo: Microservice App on Swarm
https://github.com/mgoelzer/swarm-demo-voting-app
(then follow HOWTO.txt)
Demonstrates a microservice app on Swarm including:
• Vagrantfile + AWS Cloud Formation template to deploy the cluster
• Load balanced web front end
• DNS-based service discovery

13. Interlock (nginx or ha_proxy)
web01
redis01
web02
redis02
web03
redis03
web04
redis04
web05
redis05
pg results-app

14. Clustered Voting App
Repo: https://github.com/mgoelzer/swarm-demo-voting-app

15. Clustered Voting App
Repo: https://github.com/mgoelzer/swarm-demo-voting-app

16. Interlock (nginx or ha_proxy)
web01
redis01
web02
redis02
web03
redis03
web04
redis04
web05
redis05
pg results-app

17. Interlock (nginx or ha_proxy)
web01
redis01
web02
redis02
web03
redis03
web04
redis04
web05
redis05
pg results-app
10.0.0.4
10.0.0.3
36.36.36.36
10.0.0.100
10.0.0.6
10.0.0.5
10.0.0.7
10.0.0.6
10.0.0.9
10.0.0.8
10.0.0.11
10.0.0.10
10.0.0.101

18. Interlock (nginx or ha_proxy)
web01
redis01
web02
redis02
web03
redis03
web04
redis04
web05
redis05
pg results-app
10.0.0.4
10.0.0.3
36.36.36.36
10.0.0.100
10.0.0.6
10.0.0.5
10.0.0.7
10.0.0.6
10.0.0.9
10.0.0.8
10.0.0.11
10.0.0.10
10.0.0.101
192.168.33.11 192.168.33.12
.20 .21 .22 .23 .24
.200 .201
.251
Consul
Manager
Host
Container
10.0.0.4 Container IPs
192.168.33.11 Node IP

19. for (int i = min; i <= max; i++) {
String hostname = String.format(“redis%02d.mynet”,i);
try {
InetAddress inetAddress = InetAddress.getByName(hostname);
String addr = inetAddress.getHostAddress();
queues.add(new RedisQueue(hostname, addr));
} catch (UnknownHostException e) {
// No such host
}
}
return queues;
“redis01.mynet”
”redis02.mynet”
etc.
e.g., “10.0.0.23”
Worker discovers Redis’s at runtime by DNS
queues
redis01.mynet 10.0.0.23
redis02.mynet 10.0.0.24
redis03.mynet 10.0.0.25

20. Demo
To follow along at home:
1. Clone https://github.com/mgoelzer/swarm-demo-voting-app
2. Follow instructions in HOWTO.txt file (in root of repo).
3. Screencasts of the demo:
• Step 1: Vagrant Up (https://www.youtube.com/watch?v=gKiEveAjgU8)
• Step 2: Create Swarm Cluster (https://www.youtube.com/watch?v=IskSZC5wv7A)
• Step 3: Build demo app images (https://www.youtube.com/watch?v=ZRFtNQB-VY8)
• Step 4: Start demo app containers (https://www.youtube.com/watch?v=jlrpWrsvB2Q)
• Step 5: Use the demo app (https://www.youtube.com/watch?v=_Pc07ThTbzs)

21. Jérôme’s Example: Coin Miner
Slides: http://view.dckr.info/
Repo: https://github.com/jpetazzo/orchestration-workshop
Demonstrates:
• Chaos-monkey proof cluster
• How to do batch workloads on Swarm
• ELK stack for logging and metrics
• Other load balancing patterns beyond Interlock
(another Swarm example app)

22. Mike Goelzer | mgoelzer@docker.com | GH: @mgoelzer | @mikegoelzer