Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
doing the old thing the new way 
by @rafecolton
brief prologue 
“rafe” (rafecolton on the internets) 
software engineer, platform @ modcloth 
using docker in prod since v...
obligatory slide with a bunch of logos
the stack that was
the stack that was
the stack that was
the stack that would be
motivations 
• simplify application architecture 
• support a variety of application languages 
• make provisioning and de...
motivations 
• simplify application architecture 
• support a variety of application languages 
• make provisioning and de...
motivations 
• simplify application architecture 
• support a variety of application languages 
• make provisioning and de...
motivations 
goals 
bonus points 
• chatops 
• actual button for provisioning and deployment 
does docker 
facilitate such...
the stack that would be
case study: modcloth.com/style-gallery 
Clear and Simple Statement.
case study: modcloth.com/style-gallery
case study: modcloth.com/style-gallery
case study: modcloth.com/style-gallery 
webserver 
nginx ruby 
smartos 
rails
case study: modcloth.com/style-gallery 
webserver workers 
nginx ruby 
smartos 
rails 
sidekiq 
cron ruby 
smartos 
rails
case study: modcloth.com/style-gallery 
webserver 
nginx ruby 
smartos 
rails 
sidekiq 
workers 
cron ruby 
smartos 
rails...
case study: modcloth.com/style-gallery 
webserver 
rails 
ruby nginx 
docker 
ubuntu
case study: modcloth.com/style-gallery 
webserver 
rails 
ruby nginx 
docker 
ubuntu 
cron
case study: modcloth.com/style-gallery 
webserver 
rails 
ruby nginx 
docker 
ubuntu 
cron 
supervisord
case study: modcloth.com/style-gallery 
webserver 
rails 
ruby nginx 
docker 
ubuntu 
cron 
supervisord 
sidekiq 
workers ...
case study: modcloth.com/style-gallery 
webserver 
rails 
ruby nginx 
cron 
supervisord 
docker 
ubuntu 
sidekiq 
workers ...
challenges 
• overall complexity 
• maintainability 
• image consistency 
• container reliability 
• log aggregation 
• mo...
challenges 
• overall complexity 
• maintainability 
• image consistency 
• container reliability 
• log aggregation 
• mo...
case study: modcloth.com/style-gallery 
webserver 
rails 
ruby 
rails 
docker cron 
ubuntu 
nginx 
sidekiq 
workers 
ruby ...
the stack
observations 
docker is an excellent packaging and distribution system
observations 
docker is an excellent packaging and distribution system 
containers are the canonical building block for a ...
begin github search… 
projects for orchestrating containers: 
• docker/fig 
• deis/deis 
• flynn/flynn 
• coreos/fleet 
• ...
begin github search… 
• docker/fig 
• deis/deis 
• flynn/flynn 
• coreos/fleet 
• ansible/ansible 
• opscode/chef 
• progr...
• docker/fig 
• deis/deis 
• flynn/flynn 
• coreos/fleet 
• ansible/ansible 
• opscode/chef 
• progrium/dokku 
• newrelic/...
begin github search… 
projects for building containers: 
• rafecolton/docker-builder 
• mitchellh/packer 
• swipely/dockly...
observation 
everybody is building containers differently.
observation 
everybody is building containers differently. 
how do we build production-ready containers?
writing a good Dockerfile 
lesson 0: getting started
writing a good Dockerfile 
lesson 0: getting started use a docker hub base
writing a good Dockerfile 
lesson 0: getting started 
set your env
writing a good Dockerfile 
lesson 1: order matters 
deps before bundling
writing a good Dockerfile 
lesson 1: order matters 
ADD only Gemfile* first
writing a good Dockerfile 
lesson 1: order matters 
`ADD .` as late as possible
writing a good Dockerfile 
lesson 2: optimize for size, repeatability 
combine RUN commands 
whenever possible
writing a good Dockerfile 
lesson 2: optimize for size, repeatability 
RUN dependent 
steps together
writing a good Dockerfile 
lesson 3: use a standard entrypoint 
use a *simple* 
entrypoint script
writing a good Dockerfile 
lesson 3: use a standard entrypoint 
operate on docker-specific 
environment variables
writing a good Dockerfile 
lesson 3: use a standard entrypoint 
wrap verbose 
CMD options
writing a good Dockerfile 
lesson 3: use a standard entrypoint 
exec "$@" # give yourself a shell
the image development lifecycle 
build 
*type things* push 
tag
the image development lifecycle 
> docker build -t myapp:latest . 
> export latest="$(docker images | grep myapp:latest | ...
the image development lifecycle: docker-builder 
push 
build 
tag 
> docker-builder build . 
*type things*
teh pipeline 
docker build 
server* 
app 
app 
app 
app 
*https://github.com/rafecolton/docker-builder
conclusion 
building containers is like writing ruby code:
conclusion 
building containers is like writing ruby code: 
it’s easy to do 
it’s hard to do correctly
so what did we learn? (or, through what did you sleep?) 
stuff: 
• the complexity will come naturally (so don’t force it) ...
so what did we learn? (or, through what did you sleep?) 
stuff: 
• the complexity will come naturally (so don’t force it) ...
so what did we learn? (or, through what did you sleep?) 
stuff: 
• the complexity will come naturally (so don’t force it) ...
brief epilogue 
goal: move to linux 
=> all apps (less one) now employing docker/ansible/linux
brief epilogue 
goal: move to linux 
=> all apps (less one) now employing docker/ansible/linux 
goal: consolidated, plugga...
brief epilogue 
goal: move to linux 
=> all apps (less one) now employing docker/ansible/linux 
goal: consolidated, plugga...
brief epilogue 
goal: move to linux 
=> all apps (less one) now employing docker/ansible/linux 
goal: consolidated, plugga...
brief epilogue 
I’m job hunting…
brief epilogue 
I’m job hunting… 
…and I haven’t shot anything yet. 
so if you’re hiring, come talk to me after the show. ...
thank you
Upcoming SlideShare
Loading in …5
×

Dockercon EU 2014

898 views

Published on

My talk from Dockercon EU in Amsterdam, Dec 2014. Original abstract:

The ModCloth Platform team has been building a Docker-based continuous delivery pipeline. This presentation discusses that project and how we build containers at ModCloth. The topics include what goes into our containers; how to optimize builds to use the Docker build cache effectively; useful development workflows (including using fig); and the key decision to treat containers as processes instead of mini-vms. This presentation will also discuss (and demo!) the workflow we’ve adopted for building containers and how we’ve integrated container builds with our CI.

Published in: Software
  • Be the first to comment

Dockercon EU 2014

  1. 1. doing the old thing the new way by @rafecolton
  2. 2. brief prologue “rafe” (rafecolton on the internets) software engineer, platform @ modcloth using docker in prod since v0.7.0 *todo: explain “doing the old thing the new way”
  3. 3. obligatory slide with a bunch of logos
  4. 4. the stack that was
  5. 5. the stack that was
  6. 6. the stack that was
  7. 7. the stack that would be
  8. 8. motivations • simplify application architecture • support a variety of application languages • make provisioning and deployment more accessible
  9. 9. motivations • simplify application architecture • support a variety of application languages • make provisioning and deployment more accessible goals • push-button provisioning and deployment • consolidated, pluggable platform • move to linux
  10. 10. motivations • simplify application architecture • support a variety of application languages • make provisioning and deployment more accessible goals • push-button provisioning and deployment • consolidated, pluggable platform • move to linux bonus points • chatops • actual button for provisioning and deployment
  11. 11. motivations goals bonus points • chatops • actual button for provisioning and deployment does docker facilitate such a solution? • simplify application architecture • support a variety of application languages • make provisioning and deployment more accessible • push-button provisioning and deployment • consolidated, pluggable platform • move to linux
  12. 12. the stack that would be
  13. 13. case study: modcloth.com/style-gallery Clear and Simple Statement.
  14. 14. case study: modcloth.com/style-gallery
  15. 15. case study: modcloth.com/style-gallery
  16. 16. case study: modcloth.com/style-gallery webserver nginx ruby smartos rails
  17. 17. case study: modcloth.com/style-gallery webserver workers nginx ruby smartos rails sidekiq cron ruby smartos rails
  18. 18. case study: modcloth.com/style-gallery webserver nginx ruby smartos rails sidekiq workers cron ruby smartos rails how complex could it be?
  19. 19. case study: modcloth.com/style-gallery webserver rails ruby nginx docker ubuntu
  20. 20. case study: modcloth.com/style-gallery webserver rails ruby nginx docker ubuntu cron
  21. 21. case study: modcloth.com/style-gallery webserver rails ruby nginx docker ubuntu cron supervisord
  22. 22. case study: modcloth.com/style-gallery webserver rails ruby nginx docker ubuntu cron supervisord sidekiq workers rails
  23. 23. case study: modcloth.com/style-gallery webserver rails ruby nginx cron supervisord docker ubuntu sidekiq workers rails nad nodejs rsyslogd sshd
  24. 24. challenges • overall complexity • maintainability • image consistency • container reliability • log aggregation • monitoring
  25. 25. challenges • overall complexity • maintainability • image consistency • container reliability • log aggregation • monitoring lessons • don’t do the new thing the old way • consider division of responsibility
  26. 26. case study: modcloth.com/style-gallery webserver rails ruby rails docker cron ubuntu nginx sidekiq workers ruby nad nodejs rsyslogd
  27. 27. the stack
  28. 28. observations docker is an excellent packaging and distribution system
  29. 29. observations docker is an excellent packaging and distribution system containers are the canonical building block for a continuous delivery pipeline
  30. 30. begin github search… projects for orchestrating containers: • docker/fig • deis/deis • flynn/flynn • coreos/fleet • ansible/ansible • opscode/chef • progrium/dokku • newrelic/centurion
  31. 31. begin github search… • docker/fig • deis/deis • flynn/flynn • coreos/fleet • ansible/ansible • opscode/chef • progrium/dokku • newrelic/centurion • mesosphere/marathon • airbnb/chronos • GoogleCloudPlatform/kubernetes • openshift/geard • VoltFramework/volt projects for orchestrating containers:
  32. 32. • docker/fig • deis/deis • flynn/flynn • coreos/fleet • ansible/ansible • opscode/chef • progrium/dokku • newrelic/centurion • mesosphere/marathon • airbnb/chronos • GoogleCloudPlatform/kubernetes • openshift/geard • VoltFramework/volt • octohost/octohost • makeusabrew/decking • signalfuse/maestro-ng • shipyard/shipyard • DevTable/gantryd • mcuadros/dockership • longshoreman/longshoreman • marmelab/gaudi • etc. begin github search… projects for orchestrating containers:
  33. 33. begin github search… projects for building containers: • rafecolton/docker-builder • mitchellh/packer • swipely/dockly • ???
  34. 34. observation everybody is building containers differently.
  35. 35. observation everybody is building containers differently. how do we build production-ready containers?
  36. 36. writing a good Dockerfile lesson 0: getting started
  37. 37. writing a good Dockerfile lesson 0: getting started use a docker hub base
  38. 38. writing a good Dockerfile lesson 0: getting started set your env
  39. 39. writing a good Dockerfile lesson 1: order matters deps before bundling
  40. 40. writing a good Dockerfile lesson 1: order matters ADD only Gemfile* first
  41. 41. writing a good Dockerfile lesson 1: order matters `ADD .` as late as possible
  42. 42. writing a good Dockerfile lesson 2: optimize for size, repeatability combine RUN commands whenever possible
  43. 43. writing a good Dockerfile lesson 2: optimize for size, repeatability RUN dependent steps together
  44. 44. writing a good Dockerfile lesson 3: use a standard entrypoint use a *simple* entrypoint script
  45. 45. writing a good Dockerfile lesson 3: use a standard entrypoint operate on docker-specific environment variables
  46. 46. writing a good Dockerfile lesson 3: use a standard entrypoint wrap verbose CMD options
  47. 47. writing a good Dockerfile lesson 3: use a standard entrypoint exec "$@" # give yourself a shell
  48. 48. the image development lifecycle build *type things* push tag
  49. 49. the image development lifecycle > docker build -t myapp:latest . > export latest="$(docker images | grep myapp:latest | head -n 1 | awk '{print $3}’)" > docker tag $latest "$(git rev-parse -q HEAD)" # sha > docker tag $latest "$(git describe --always --dirty --tags)" # tag > docker tag $latest "$(git rev-parse -q --abbrev-ref HEAD)" # branch > for image in $(docker images | grep myapp | awk '{print $1 ":" $2}' | head -n 4) ; push do docker push $image ; done build tag *type things*
  50. 50. the image development lifecycle: docker-builder push build tag > docker-builder build . *type things*
  51. 51. teh pipeline docker build server* app app app app *https://github.com/rafecolton/docker-builder
  52. 52. conclusion building containers is like writing ruby code:
  53. 53. conclusion building containers is like writing ruby code: it’s easy to do it’s hard to do correctly
  54. 54. so what did we learn? (or, through what did you sleep?) stuff: • the complexity will come naturally (so don’t force it) • be intentional about your Dockerfile
  55. 55. so what did we learn? (or, through what did you sleep?) stuff: • the complexity will come naturally (so don’t force it) • be intentional about your Dockerfile • docker is an excellent packaging and distribution system • containers are the canonical building blocks
  56. 56. so what did we learn? (or, through what did you sleep?) stuff: • the complexity will come naturally (so don’t force it) • be intentional about your Dockerfile • docker is an excellent packaging and distribution system • containers are the canonical building blocks • consider division of responsibility between the host and the container • don’t do the new thing the old way (do the old thing the new way!)
  57. 57. brief epilogue goal: move to linux => all apps (less one) now employing docker/ansible/linux
  58. 58. brief epilogue goal: move to linux => all apps (less one) now employing docker/ansible/linux goal: consolidated, pluggable platform => shared monitoring, log aggregation, & load balancing services
  59. 59. brief epilogue goal: move to linux => all apps (less one) now employing docker/ansible/linux goal: consolidated, pluggable platform => shared monitoring, log aggregation, & load balancing services goal: push-button provisioning and deployment => it works, minimal magic… and it’s well documented
  60. 60. brief epilogue goal: move to linux => all apps (less one) now employing docker/ansible/linux goal: consolidated, pluggable platform => shared monitoring, log aggregation, & load balancing services goal: push-button provisioning and deployment => it works, minimal magic… and it’s well documented could easily be maintained by only two people… dun dun dun
  61. 61. brief epilogue I’m job hunting…
  62. 62. brief epilogue I’m job hunting… …and I haven’t shot anything yet. so if you’re hiring, come talk to me after the show. twitter: @rafecolton github: rafecolton rafecolton.com
  63. 63. thank you

×