SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.
SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.
Successfully reported this slideshow.
Activate your 14 day free trial to unlock unlimited reading.
1.
Taking control of chaos
(with Docker and puppet)
- Tomas Doran
@bobtfish
18/11/2013
2.
Docker - why should you care?
• Isolation
– Run each daemon in it’s own container
• Security
– Containers are fully independent
– Build a new container for a new application version!
• Ease of development
– Same containers in Vagrant on laptop as in production!
– Build pipeline for whole application environment
• Ease of deployment
– Deploy from your own private registry
– Roll back by just deploying the last version
3.
Taking control of chaos
• My personal server
• Lots of responsibilities
–
–
–
–
–
–
Postfix
Dovecot
DNS
Irc + tmux
tor
A zillion web apps (ruby, perl, php, python,
nodejs!)
• 0 time spent on it
9.
Convergence and immutability
• Exactly one run
– Should be building machines clean every time
– Doing exactly one puppet run
• Always rebuild
– Unless you rebuild regularly
– You don’t know you can rebuild
• Immutable instances
– Never change config on a server, replace instance!
• Hard if you only have 1 server!
16.
Single
$864
Postfix
Dovecot
Postgrey Spamassasin
17.
Service
$864
Postfix
Dovecot
Postgrey Spamassasin Mysql
18.
Independently!
$864
Postfix
Dovecot
Postgrey Spamassasin Mysql
irssi
19.
For real.
$864
Postfix
Dovecot
Postgrey Spamassasin Mysql
irssi
playground
(sshd)
20.
A cunning plan
• Build puppet code for installing
service on the old server
• Run same puppet code inside a
container to install packages / build
config
• Add shims to start service inside
container
21.
Data management
• All mutable data is an lvm volume
mounted from the host
• All lvm volumes also get bind
mounted read only
• Share unix domain sockets this way
• Server for socket creates
• Clients mount ro version
24.
What’s inside a container?
supervisord:
• The ‘real’ process
• mcollective
• sshd
25.
Code structure
• profile::dovecot
– All the things needed to run dovecot
– Parameterizeable as needed for two different deploy
environments.
• container::dovecot
– Main entry point when building the container
– Delegates most of real setup to profile::dovecot
– Adds all the container specific overrides
– Adds supervisor service(s) for this container
• run_container::dovecot
– Wraps docker::run {}
– Manages the associated lvm volumes
– Adds firewall rules
26.
profile::dovecot
!
– All the things needed to run dovecot
– Parameterizeable as needed for two different deploy environments.
27.
container::dovecot
!
– Main entry point when building the container
– Delegates most of real setup to profile::dovecot
– Adds all the container specific overrides
– Adds supervisor service(s) for this container
29.
Building containers
• profile::docker::build_container
– define - writes out Dockerfile + support files
– Runs docker build .
– Manages dependencies so base containers get built first
• profile::docker::with_socket::mysql
– /socket/mysql
– Symlink into /var/lib/mysql
• profile::docker::with_supervisord
– Setup supervisor with default (mcollective + ssh) tasks
– Default /start script to invoke supervisord
30.
profile::docker::build_container
!
– define - writes out Dockerfile + support files
– Runs docker build .
– Manages dependencies so base containers get built first
31.
profile::docker::build_container
!
– define - writes out Dockerfile + support files
– Runs docker build .
– Manages dependencies so base containers get built first
32.
profile::docker::build_container
!
– define - writes out Dockerfile + support files
– Runs docker build .
– Manages dependencies so base containers get built first
33.
container::with_socket::mysql
– /socket/mysql
– Symlink into /var/lib/mysql
34.
container::with_supervisord
!
– Setup supervisor with default (mcollective + ssh) tasks
– Default /start script to invoke supervisord
35.
The Dockerfile
• Drop facts
– /etc/facter/facts.d/is_container.txt
– /etc/facter/facts.d/container_name
• Copy in code
– ADD support/puppet /etc/puppet
• Run puppet
– Masterless
– No real manifest, just:
if $::is_container {..
36.
Drop facts
– /etc/facter/facts.d/is_container.txt
!
!
!
!
39.
site.pp
– Masterless
– No node manifest, just:
if $::is_container {..
40.
Issues
• Docker is kinda buggy
– Just went 1.0, being fixed fast!
– No sane exit status to docker build
– AUFS 42 layer limit
• Forge modules + packages assume
service management
– No upstart inside containers - fails everywhere
• Debian packages are inconsistent
– Lots of packages don’t use invoke-rc.d
42.
Loads of TODOs
• HAProxy all the things!
– Currently just bind containers to local ports
– Container replacement is not invisible
– Run haproxy on the real host
– Dynamically regenerate its config based on running
containers
• Registry
– Build containers in Vagrant, push up to prod
• PAAS web stuff
– www all still lives on old server :(
– Gonna fix this real soon
43.
Open sores!
• Open source all the things!
– garethr++
– (I owe you patches)
– Forked and changed a million modules
– Will cleanup and upstream some changes
• My profile::docker code
– Don’t know how to make this really generically
reusable
– We need sub-modules?
– Happy to share chunks