Presentation on MaestroNG, an orchestration and management tool for multi-host container deployments with Docker. #lspe meetup, February 20th, 2014 at Yahoo!'s URL café.

1. Docker and Maestro
For fun, development and profit

2. Maxime Petazzoni
Software Engineer at SignalFuse, Inc.
(also, Jérôme’s cousin)
!
max@signalfuse.com

3. Real-time monitoring, instrumentation,
observability and analytics
Still in “stealth” mode
Get updates at www.signalfuse.com

4. “Docker is awesome!”
–You, some time in the last hour (hopefully).

5. A versatile foundation
Service or application containment, security, software delivery, host
and environment isolation, …and so much more.

6. Power at your fingertips
Complete control through the remote API
Available programmatic clients like docker-py

7. docker:$ docker -d -H tcp://0.0.0.0:4243
!
client:$ cat << EOF | python import docker
from pprint import pprint as pp
pp(docker.client.Client(‘tcp://docker:4243')
.images('quay.io/signalfuse/maestro-base'))
EOF
!
!
[{u’Created': 1391202535,
u’Id': u’37de13d273eb9a02cd64…’,
u’Repository':
u'quay.io/signalfuse/maestro-base',
u'Size': 155663843,
u'Tag': u'0.1.6',
u'VirtualSize': 774767942}]

8. Docker’s Achilles:
orchestration
Single-host is alright with links, but multi-host just isn’t there.

9. How do I orchestrate the deployment
and control of a full, multi-host,
Docker-based environment?

10. (And more importantly:)
How do I make this process one and
the same for development, testing
and production environments?

11. Enter: Maestro
The totally not scalable, pet project that solved my use case.
(and maybe yours)

12. Maestro is actually MaestroNG,
a re-invention of Kimbro Staken’s Maestro
(formerly, dockermix)

13. Takes in a definition of services, their dependencies ,
configuration and target host…
!
…and automates the deployment (and control) of their
corresponding containers on these hosts.

14. Classic use case: a pool of “dumb” workers on your
favorite cloud/hosting provider that just run Docker.
!
No need to (ma)ssh into anything,
no need to pre-configure anything.
!
Everything is remote controlled.

15. Other typical use case: running all the components of
your stack in a single, local virtual machine.
!
Useful for development, integration testing, etc.

16. Philosophy: lightweight application/service containers.
!
Represent and control your software stack
and its dependencies.
!
Docker images are the output of your CI process
(automation!).
!
Start fast, fail faster.
Not for heavyweight, complex container “VMs”.

17. Each service instance (container) defines where it runs
and which ports it exposes, among other things.
!
Like Docker links, Maestro works by injecting this
information in the container’s environment about each
container’s service’s dependencies.

18. Let’s say broker-1 of kafka depends on ZooKeeper. Its
environment will contain:
MAESTRO_ENVIRONMENT_NAME = lspe
SERVICE_NAME = kafka
CONTAINER_NAME = broker-1
CONTAINER_HOST_ADDRESS = 192.168.10.2
!
ZOOKEEPER_ZK_NODE_1_HOST = 192.168.10.2
ZOOKEEPER_ZK_NODE_1_CLIENT_PORT = 2181
ZOOKEEPER_ZK_NODE_1_PEER_PORT = 2888
ZOOKEEPER_ZK_NODE_1_LEADER_ELECTION_PORT = 3888
!
KAFKA_BROKER_1_HOST = 192.168.10.2
KAFKA_BROKER_1_BROKER_INTERNAL_PORT = 9042
KAFKA_BROKER_1_BROKER_PORT = 9042
KAFKA_BROKER_1_JMX_INTERNAL_PORT = 7199
KAFKA_BROKER_1_JMX_PORT = 17199

19. <SERVICE_NAME>_<CONTAINER_NAME>_HOST
<SERVICE_NAME>_<CONTAINER_NAME>_PORT
<SERVICE_NAME>_<CONTAINER_NAME>_INTERNAL_PORT

20. Using this information, you can configure your
application at container start time.
!
If you like Python, Maestro helps you by providing a set
of guest helper functions in maestro.guestutils to easily
extract and use this data.

21. #!/usr/bin/env python
!
# This is my cool container’s “init script”
!
import os
from maestro.guestutils import *
!
os.execl(‘java’, ‘java’,
‘-jar’, ‘my-app.jar’,
‘-DlistenPort={}’.format(get_port(‘service’)),
‘-DzkServers={}’.format(
get_node_list(‘zookeeper’, ports=[‘peer’])))

22. Dependency order is respected on start;
inverse order on stop.
!
Can be overridden to stop individual services or
containers.

23. MyApp
Start order:
1. ZooKeeper
2. Kafka
3. MyApp
Kafka
ZK
Stop order:
1. MyApp
2. Kafka
3. ZooKeeper
Works on subsets of services too.

24. So how do you wield
this power?
A bit clunkily, with YAML (and a bit of Jinja2).
!
!
!
(sorry)

25. # Yay, YAML!
name: lspe
!
registries:
# Define custom image registries for
# private registries, with credentials.
!
ships:
# Declare each target host.
# (Docker daemon locations)
!
services:
# Declare each service, their
# instances, dependencies and
# configuration

26. registries:
# Quay.io with Maestro robot account
quay.io:
registry: https://quay.io/v1/
email: maestro@signalfuse.com
username: signalfuse+maestro
password: {{ env.SUPER_SECRET }}
When starting a container, Maestro will automatically
login and pull the image from the right place if the image
name matches a configured registry.

27. ships:
# Local virtual machine
vm:
ip: 192.168.10.2
docker_port: 4243
timeout: 10
# Slow VM is slow
# A shorter form…
vm2: {ip: 192.168.10.3, timeout: 5}
Ships carry containers and are referred to by name in the
configuration.

28. services:
# ZooKeeper
zookeeper:
image: quay.io/signalfuse/zookeeper:3.4.5
!
# Our zoo isn’t too wild,
# only one keeper is enough.
zk-node-1:
ship: vm
ports:
client: 2181
peer: 2888/tcp
leader_election: “3888/tcp:3888/tcp”
# Keep persistent data on the host.
volumes:
/var/lib/zookeeper: /data/zookeeper
# Environment can be passed-in too.
env:
JVM_FLAGS: “-Xmx1g”

29. # Kafka
kafka:
image: quay.io/signalfuse/kafka:0.8.0
requires: [ zookeeper ]
env:
ZOOKEEPER_BASE: /lspe/kafka
RETENTION_HOURS: 48
broker-1:
ship: vm
ports: {broker: 9092, jmx: “7199:17199”}
# Keep persistent data on the host.
volumes:
/var/lib/kafka: /data/kafka
env:
BROKER_ID: 0
More flexibility in port mappings, volume bindings, and
environment variables definition not shown here.

30. See README.md for full
syntax details and features
https://github.com/signalfuse/maestro-ng/blob/master/README.md

31. Demo time!
Be prepared for it to fail, because demos always do.

32. What’s next?
More flexible service status detection (not only port pinging)
Soft and hard service dependencies
Parallel startup of independent services and instances of a service

33. That’s it!
Thanks for listening! :)
github.com/dotcloud/docker-py
github.com/signalfuse/maestro-ng

34. SignalFuse is hiring
world class engineers!
jobs@signalfuse.com