• Like
Docker puppetcamp london 2013
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Docker puppetcamp london 2013

  • 986 views
Published

My talk at London Puppet camp - about using Docker to 'take control of chaos' that was my home equipment.

My talk at London Puppet camp - about using Docker to 'take control of chaos' that was my home equipment.

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
986
On SlideShare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
12
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Taking control of chaos (with Docker and puppet) - Tomas Doran @bobtfish 18/11/2013
  • 2. Docker - why should you care? • Isolation – Run each daemon in it’s own container • Security – Containers are fully independent – Build a new container for a new application version! • Ease of development – Same containers in Vagrant on laptop as in production! – Build pipeline for whole application environment • Ease of deployment – Deploy from your own private registry – Roll back by just deploying the last version
  • 3. Taking control of chaos • My personal server • Lots of responsibilities – – – – – – Postfix Dovecot DNS Irc + tmux tor A zillion web apps (ruby, perl, php, python, nodejs!) • 0 time spent on it
  • 4. My personal server
  • 5. Several years later…. Label for your image (if needed) $864
  • 6. Reality Label for your image (if needed) $864
  • 7. This year’s model Label for your image (if needed) $864
  • 8. Half the answer $864
  • 9. Convergence and immutability • Exactly one run – Should be building machines clean every time – Doing exactly one puppet run • Always rebuild – Unless you rebuild regularly – You don’t know you can rebuild • Immutable instances – Never change config on a server, replace instance! • Hard if you only have 1 server!
  • 10. The other half $864
  • 11. Compartmentalization $864
  • 12. Every service? $864
  • 13. Yes $864 Postfix
  • 14. Really $864 Postfix Dovecot
  • 15. Every $864 Postfix Dovecot Postgrey
  • 16. Single $864 Postfix Dovecot Postgrey Spamassasin
  • 17. Service $864 Postfix Dovecot Postgrey Spamassasin Mysql
  • 18. Independently! $864 Postfix Dovecot Postgrey Spamassasin Mysql irssi
  • 19. For real. $864 Postfix Dovecot Postgrey Spamassasin Mysql irssi playground (sshd)
  • 20. A cunning plan • Build puppet code for installing service on the old server • Run same puppet code inside a container to install packages / build config • Add shims to start service inside container
  • 21. Data management • All mutable data is an lvm volume mounted from the host • All lvm volumes also get bind mounted read only • Share unix domain sockets this way • Server for socket creates • Clients mount ro version
  • 22. Compartmentalization $864
  • 23. Containers and volumes
  • 24. What’s inside a container? supervisord: • The ‘real’ process • mcollective • sshd
  • 25. Code structure • profile::dovecot – All the things needed to run dovecot – Parameterizeable as needed for two different deploy environments. • container::dovecot – Main entry point when building the container – Delegates most of real setup to profile::dovecot – Adds all the container specific overrides – Adds supervisor service(s) for this container • run_container::dovecot – Wraps docker::run {} – Manages the associated lvm volumes – Adds firewall rules
  • 26. profile::dovecot ! – All the things needed to run dovecot – Parameterizeable as needed for two different deploy environments.
  • 27. container::dovecot ! – Main entry point when building the container – Delegates most of real setup to profile::dovecot – Adds all the container specific overrides – Adds supervisor service(s) for this container
  • 28. run_container::dovecot ! – Wraps docker::run {} – Manages the associated lvm volumes – Adds firewall rules
  • 29. Building containers • profile::docker::build_container – define - writes out Dockerfile + support files – Runs docker build . – Manages dependencies so base containers get built first • profile::docker::with_socket::mysql – /socket/mysql – Symlink into /var/lib/mysql • profile::docker::with_supervisord – Setup supervisor with default (mcollective + ssh) tasks – Default /start script to invoke supervisord
  • 30. profile::docker::build_container ! – define - writes out Dockerfile + support files – Runs docker build . – Manages dependencies so base containers get built first
  • 31. profile::docker::build_container ! – define - writes out Dockerfile + support files – Runs docker build . – Manages dependencies so base containers get built first
  • 32. profile::docker::build_container ! – define - writes out Dockerfile + support files – Runs docker build . – Manages dependencies so base containers get built first
  • 33. container::with_socket::mysql – /socket/mysql – Symlink into /var/lib/mysql
  • 34. container::with_supervisord ! – Setup supervisor with default (mcollective + ssh) tasks – Default /start script to invoke supervisord
  • 35. The Dockerfile • Drop facts – /etc/facter/facts.d/is_container.txt – /etc/facter/facts.d/container_name • Copy in code – ADD support/puppet /etc/puppet • Run puppet – Masterless – No real manifest, just: if $::is_container {..
  • 36. Drop facts – /etc/facter/facts.d/is_container.txt ! ! ! !
  • 37. Drop facts – /etc/facter/facts.d/is_container.txt ! ! ! ! ! ! – /etc/facter/facts.d/container_name.txt
  • 38. Puppet code – ADD support/puppet /etc/puppet – RUN bundle exec rake puppet
  • 39. site.pp – Masterless – No node manifest, just: if $::is_container {..
  • 40. Issues • Docker is kinda buggy – Just went 1.0, being fixed fast! – No sane exit status to docker build – AUFS 42 layer limit • Forge modules + packages assume service management – No upstart inside containers - fails everywhere • Debian packages are inconsistent – Lots of packages don’t use invoke-rc.d
  • 41. It’s still awesome $864
  • 42. Loads of TODOs • HAProxy all the things! – Currently just bind containers to local ports – Container replacement is not invisible – Run haproxy on the real host – Dynamically regenerate its config based on running containers • Registry – Build containers in Vagrant, push up to prod • PAAS web stuff – www all still lives on old server :( – Gonna fix this real soon
  • 43. Open sores! • Open source all the things! – garethr++ – (I owe you patches) – Forked and changed a million modules – Will cleanup and upstream some changes • My profile::docker code – Don’t know how to make this really generically reusable – We need sub-modules? – Happy to share chunks
  • 44. Sorry!
  • 45. Questions? http://www.yelp.com/careers?jvi=ogVTXfwL $864 http://containercats.tumblr.com/