Puppet Camp Chicago 2014: Docker and Puppet: 1+1=3 (Intermediate)

1,647 views
1,540 views

Published on

"Docker and Puppet: 1+1=3" presented by Jerome Petazzoni, Docker at Puppet Camp Chicago 2014

Published in: Software
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,647
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
59
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide

Puppet Camp Chicago 2014: Docker and Puppet: 1+1=3 (Intermediate)

  1. 1. Docker and Puppet 1+1=3
  2. 2. @jpetazzo ● Wrote dotCloud PAAS deployment tools – EC2, LXC, Puppet, Python, Shell, ØMQ... ● Docker contributor – Docker-in-Docker, VPN-in-Docker, router-in-Docker... CONTAINERIZE ALL THE THINGS! ● Runs Docker in production, and helps others to do the same
  3. 3. What is Docker? The quick elevator pitch
  4. 4. Docker Engine + Docker Hub = Docker Platform
  5. 5. Docker Engine
  6. 6. The Docker Engine ● Open Source ● Written in Go ● Runs containers ● On any modern Linux machine (Intel 64 bits for now)
  7. 7. Containers ?
  8. 8. Containers ● Software delivery mechanism (a bit like a package!) ● Put your application in a container, run it anywhere ● A bit like a VM, but ...
  9. 9. I have four words for you ● CONTAINERS boot faster (than VMs) ● CONTAINERS have less overhead (more consolidation) ● CONTAINERS bring native performance (on bare metal) ● CONTAINERS are cloud-compatible (can run in VMs)
  10. 10. Docker Engine recap ● Approximation: it's an hypervisor to run containers ● Approximation: containers are like VMs, but lighter ● Docker makes containers available to everybody (not just veterans from the last emacs/vim war)
  11. 11. Docker Hub
  12. 12. Docker Hub ● Services operated by Docker Inc. ● Library of ready-to-use container images ● Registry for your container images (public or private) ● Automated builds (triggered by pushes to GitHub/Bitbucket) ● Free for public/open source code, $$ otherwise
  13. 13. Building containers
  14. 14. Dockerfile FROM ubuntu:14.04 MAINTAINER Docker Team <education@docker.com> RUN apt-get update RUN apt-get install -y nginx RUN echo 'Hi, I am in your container' >/usr/share/nginx/html/index.html CMD [ "nginx", "-g", "daemon off;" ] EXPOSE 80
  15. 15. FROM ubuntu RUN apt-get -y update RUN apt-get install -y g++ RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe ... RUN apt-get install -y libmozjs185-dev libicu-dev libtool ... RUN apt-get install -y make wget RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf- RUN cd /tmp/apache-couchdb-* && ./configure && make install RUN printf "[httpd]nport = 8101nbind_address = 0.0.0.0" > /usr/local/etc/couchdb/local.d/docker.ini EXPOSE 8101 CMD ["/usr/local/bin/couchdb"] docker build -t jpetazzo/couchdb .
  16. 16. Dockerfile vs. Shell scripts
  17. 17. Shell scripts ● OK-ish for simple stacks ● Tricky to handle all possible situations (that's why we have proper config management) ● Though choice when rebuilding: – from scratch (but it takes forever!) – iteratively (but might behave differently!)
  18. 18. Dockerfile vs. Configuration Management
  19. 19. Configuration Management: the Good ● Deals with low-level stuff ● Abstracts some details (distro, sometimes OS) ● Ensures convergence to a known state ● Library of reusable, composable templates
  20. 20. Configuration Management: the Bad ● Steep learning curve ● Generally requires an agent (or something to trigger e.g. « puppet apply ») ● Resource-intensive (it's OK to run the agent on a 64 GB server, it's less OK to run 100 agents on said server)
  21. 21. Configuration Management ● Reusability is just as good as modules are (i.e. YMMV) ● Not as deterministic as you think ● Rollbacks are harder than you think { 'openssl' : ensure => present } { 'openssl' : ensure => '1.2.3-no-heartbleed-pls' }
  22. 22. Dockerfile to the rescue
  23. 23. Dockerfile ● Doesn't have to deal with « low-level stuff » (hardware, drivers... handled by the host) ● Doesn't need all the goodness of CM (because it doesn't have to converge) ● Partial rebuilds are fast (layered caching rebuilds only what is needed) ● Allows inheritance and composition (FROM <mycustombase>; see also: ONBUILD) ● Easy learning curve (if you know Shell, you already know Dockerfile)
  24. 24. But... ● Doesn't deal with « low-level stuff » (hardware, drivers...) ● Doesn't define resource dependencies (no before/after) ● Doesn't define what runs where
  25. 25. Puppet to the rescue
  26. 26. Before/After ● Use Puppet to setup hardware (or virtual hardware), install packages, deploy code, run services. ● Use Puppet to setup hardware (or virtual hardware), install Docker, run containers. ● Use Dockerfiles to install packages, deploy code, run services.
  27. 27. Do one thing, and do it well
  28. 28. First things first https://github.com/garethr/garethr-docker https://forge.puppetlabs.com/garethr/docker
  29. 29. Installing Docker with Puppet include 'docker' class { 'docker': version => '0.8.1' }
  30. 30. Warm up our image collection # download the registry image docker::image { 'stackbrew/registry': } # don't download all ubuntu, # just 'precise' docker::image { 'ubuntu': image_tag => 'precise' }
  31. 31. Run containers docker::run { 'slavedb': image => 'jpetazzo/postgresql' command => '…' ports => ['5432', '22'], links => ['masterdb:master'], use_name => true, volumes => ['/var/lib/postgresql'], volumes_from => '420fc7e8aa20', memory_limit => 100000000, # bytes username => 'postgres', hostname => 'sdb.prod.dckr.io', env => ['FUZZINESS=42', FOO=BAR', 'FOO2=BAR2'], dns => ['8.8.8.8', '8.8.4.4'], restart_service => true }
  32. 32. Can I use Puppet to build Docker container images?
  33. 33. YES
  34. 34. Should I use Puppet to build Docker container images?
  35. 35. NO
  36. 36. OK, let's do it anyway
  37. 37. My other VM is a container ● write a Dockerfile to install Puppet ● start tons of containers ● run Puppet in them (agent, or one-shot apply) Good if you want a mix of containers/VM/metal But slower to deploy, and uses more resources
  38. 38. Sample Dockerfile FROM ubuntu:12.04 RUN apt-get install -qy wget RUN mkdir /puppet WORKDIR /puppet RUN wget -q http://apt.puppetlabs.com/puppetlabs-release-precise.deb RUN dpkg -i puppetlabs-release-precise.deb RUN apt-get update -q RUN apt-get install -qy puppet-common CMD puppet agent --no-daemonize --verbose
  39. 39. Lightweight, portable VMs ● Start containers instead of VMs – I can start 10 containers on this puny laptop! – You can start those 10 containers too! (Even though you have a totally different laptop!) – We can start those containers in the Cloud! ● Deploy sshd, syslogd, crond, etc. – You can... But do you have to?
  40. 40. The revolution will be containerized ● write a Dockerfile to install Puppet ● … and run Puppet as part of build process ● deploy fully baked, « golden » images Faster to deploy Easier to rollback
  41. 41. Sample Dockerfile FROM ubuntu:12.04 RUN apt-get install -qy wget RUN mkdir /puppet WORKDIR /puppet RUN wget -q http://apt.puppetlabs.com/puppetlabs-release-precise.deb RUN dpkg -i puppetlabs-release-precise.deb RUN apt-get update -q RUN apt-get install -qy puppet-common ENV FACTER_HOSTNAME database42 ADD ./site.pp /puppet/site.pp RUN puppet apply site.pp
  42. 42. Beyond Golden Containers
  43. 43. Get rid of sshd, crond, syslogd... ● Remote access: nsenter https://github.com/jpetazzo/nsenter ● Cron: use a separate container ● Logs: use a data container http://blog.docker.com/2014/06/why-you-dont-need-to-run-sshd-in-docker/
  44. 44. Why? ● Separate orthogonal concerns (don't rebuild your app to change logging, remote access, and other unrelated things) ● Have different policies in prod/dev/QA/etc ● Ship lighter containers
  45. 45. Thoughts...
  46. 46. What if we could... ● Run the Puppet agent outside of the container ● Run a single agent for many containers ● Share the cost of the agent
  47. 47. Thank you!
  48. 48. Shameless promo + Q&A Tonight: Docker and Mesos meet-up, at BrainTree (requires cloning+teleportation) The rest of the week: A bunch of talks about Docker & Containers (requires a LinuxCon pass) http://docker.com/ @docker @jpetazzo

×