Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Taking Control of Chaos with Docker and Puppet

20,390 views

Published on

"Taking Control of Chaos (with Docker and Puppet)" by Tomas Doran (@bobtfish) of Yelp at Puppet Camp London 2014. Find the video here: http://puppetlabs.com/community/puppet-camp

Published in: Technology
  • Be the first to comment

Taking Control of Chaos with Docker and Puppet

  1. 1. Taking control of chaos (with Docker and puppet) - Tomas Doran @bobtfish 18/11/2013
  2. 2. Docker - why should you care? • Isolation – Run each daemon in it’s own container • Security – Containers are fully independent – Build a new container for a new application version! • Ease of development – Same containers in Vagrant on laptop as in production! – Build pipeline for whole application environment • Ease of deployment – Deploy from your own private registry – Roll back by just deploying the last version
  3. 3. Taking control of chaos • My personal server • Lots of responsibilities – – – – – – Postfix Dovecot DNS Irc + tmux tor A zillion web apps (ruby, perl, php, python, nodejs!) • 0 time spent on it
  4. 4. My personal server
  5. 5. Several years later…. Label for your image (if needed) $864
  6. 6. Reality Label for your image (if needed) $864
  7. 7. This year’s model Label for your image (if needed) $864
  8. 8. Half the answer $864
  9. 9. Convergence and immutability • Exactly one run – Should be building machines clean every time – Doing exactly one puppet run • Always rebuild – Unless you rebuild regularly – You don’t know you can rebuild • Immutable instances – Never change config on a server, replace instance! • Hard if you only have 1 server!
  10. 10. The other half $864
  11. 11. Compartmentalization $864
  12. 12. Every service? $864
  13. 13. Yes $864 Postfix
  14. 14. Really $864 Postfix Dovecot
  15. 15. Every $864 Postfix Dovecot Postgrey
  16. 16. Single $864 Postfix Dovecot Postgrey Spamassasin
  17. 17. Service $864 Postfix Dovecot Postgrey Spamassasin Mysql
  18. 18. Independently! $864 Postfix Dovecot Postgrey Spamassasin Mysql irssi
  19. 19. For real. $864 Postfix Dovecot Postgrey Spamassasin Mysql irssi playground (sshd)
  20. 20. A cunning plan • Build puppet code for installing service on the old server • Run same puppet code inside a container to install packages / build config • Add shims to start service inside container
  21. 21. Data management • All mutable data is an lvm volume mounted from the host • All lvm volumes also get bind mounted read only • Share unix domain sockets this way • Server for socket creates • Clients mount ro version
  22. 22. Compartmentalization $864
  23. 23. Containers and volumes
  24. 24. What’s inside a container? supervisord: • The ‘real’ process • mcollective • sshd
  25. 25. Code structure • profile::dovecot – All the things needed to run dovecot – Parameterizeable as needed for two different deploy environments. • container::dovecot – Main entry point when building the container – Delegates most of real setup to profile::dovecot – Adds all the container specific overrides – Adds supervisor service(s) for this container • run_container::dovecot – Wraps docker::run {} – Manages the associated lvm volumes – Adds firewall rules
  26. 26. profile::dovecot ! – All the things needed to run dovecot – Parameterizeable as needed for two different deploy environments.
  27. 27. container::dovecot ! – Main entry point when building the container – Delegates most of real setup to profile::dovecot – Adds all the container specific overrides – Adds supervisor service(s) for this container
  28. 28. run_container::dovecot ! – Wraps docker::run {} – Manages the associated lvm volumes – Adds firewall rules
  29. 29. Building containers • profile::docker::build_container – define - writes out Dockerfile + support files – Runs docker build . – Manages dependencies so base containers get built first • profile::docker::with_socket::mysql – /socket/mysql – Symlink into /var/lib/mysql • profile::docker::with_supervisord – Setup supervisor with default (mcollective + ssh) tasks – Default /start script to invoke supervisord
  30. 30. profile::docker::build_container ! – define - writes out Dockerfile + support files – Runs docker build . – Manages dependencies so base containers get built first
  31. 31. profile::docker::build_container ! – define - writes out Dockerfile + support files – Runs docker build . – Manages dependencies so base containers get built first
  32. 32. profile::docker::build_container ! – define - writes out Dockerfile + support files – Runs docker build . – Manages dependencies so base containers get built first
  33. 33. container::with_socket::mysql – /socket/mysql – Symlink into /var/lib/mysql
  34. 34. container::with_supervisord ! – Setup supervisor with default (mcollective + ssh) tasks – Default /start script to invoke supervisord
  35. 35. The Dockerfile • Drop facts – /etc/facter/facts.d/is_container.txt – /etc/facter/facts.d/container_name • Copy in code – ADD support/puppet /etc/puppet • Run puppet – Masterless – No real manifest, just: if $::is_container {..
  36. 36. Drop facts – /etc/facter/facts.d/is_container.txt ! ! ! !
  37. 37. Drop facts – /etc/facter/facts.d/is_container.txt ! ! ! ! ! ! – /etc/facter/facts.d/container_name.txt
  38. 38. Puppet code – ADD support/puppet /etc/puppet – RUN bundle exec rake puppet
  39. 39. site.pp – Masterless – No node manifest, just: if $::is_container {..
  40. 40. Issues • Docker is kinda buggy – Just went 1.0, being fixed fast! – No sane exit status to docker build – AUFS 42 layer limit • Forge modules + packages assume service management – No upstart inside containers - fails everywhere • Debian packages are inconsistent – Lots of packages don’t use invoke-rc.d
  41. 41. It’s still awesome $864
  42. 42. Loads of TODOs • HAProxy all the things! – Currently just bind containers to local ports – Container replacement is not invisible – Run haproxy on the real host – Dynamically regenerate its config based on running containers • Registry – Build containers in Vagrant, push up to prod • PAAS web stuff – www all still lives on old server :( – Gonna fix this real soon
  43. 43. Open sores! • Open source all the things! – garethr++ – (I owe you patches) – Forked and changed a million modules – Will cleanup and upstream some changes • My profile::docker code – Don’t know how to make this really generically reusable – We need sub-modules? – Happy to share chunks
  44. 44. Sorry!
  45. 45. Questions? http://www.yelp.com/careers?jvi=ogVTXfwL $864 http://containercats.tumblr.com/

×