LXC, Docker,
and the future of
software delivery
Linuxcon – New Orleans, 2013
Jérôme Petazzoni, dotCloud Inc.
Outline
● Why Linux Containers?
● What are Linux Containers exactly?
● What do we need on top of LXC?
● Why Docker?
● What...
Why Linux Containers?
What are
we trying
to solve?
The Matrix From Hell
The Matrix From Hell
Many payloads
● backend services (API)
● databases
● distributed stores
● webapps
Many payloads
● Go
● Java
● Node.js
● PHP
● Python
● Ruby
● …
Many payloads
● CherryPy
● Django
● Flask
● Plone
● ...
Many payloads
● Apache
● Gunicorn
● uWSGI
● ...
Many payloads
+ your code
Many targets
● your local development environment
● your coworkers' developement environment
● your Q&A team's test enviro...
Many targets
● BSD
● Linux
● OS X
● Windows
Many targets
● BSD
● Linux
● OS X
● Windows
The Matrix From Hell
Static
website ? ? ? ? ? ? ?
Web
frontend ? ? ? ? ? ? ?
background
workers ? ? ? ? ? ? ?
User DB ? ? ...
Real-world analogy:
containers
Many products
● clothes
● electronics
● raw materials
● wine
● …
Many transportation methods
● ships
● trains
● trucks
● ...
Another matrix from hell
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
Solution to the transport problem:
the intermodal shipping container
Solution to the transport problem:
the intermodal shipping container
● 90% of all cargo now shipped in a standard containe...
Solution to the deployment problem:
the Linux container
Linux containers...
● run everywhere
– regardless of kernel version
– regardless of host distro
– (but container and host ...
What are Linux Containers exactly?
High level approach:
it's a lightweight VM
● own process space
● own network interface
● can run stuff as root
● can have ...
Low level approach:
it's chroot on steroids
● can also not have its own /sbin/init
● container = isolated process(es)
● sh...
Separation of concerns:
Dave the Developer
● inside my container:
– my code
– my libraries
– my package manager
– my app
–...
Separation of concerns:
Oscar the Ops guy
● outside the container:
– logging
– remote access
– network configuration
– mon...
How does it work?
Isolation with namespaces
● pid
● mnt
● net
● uts
● ipc
● user
How does it work?
Isolation with cgroups
● memory
● cpu
● blkio
● devices
Efficiency: almost no overhead
● processes are isolated,
but run straight on the host
● CPU performance = native performan...
Efficiency: storage-friendly
● unioning filesystems
(AUFS, overlayfs)
● snapshotting filesystems
(BTRFS, ZFS)
● copy-on-wr...
Efficiency: storage-friendly
● provisioning now takes a few milliseconds
● … and a few kilobytes
● creating a new base/ima...
Docker
What's Docker?
● Open Source engine to commoditize LXC
● using copy-on-write for quick provisioning
● allowing to create a...
Yes, but...
● « I don't need Docker; I can do all that stuff
with LXC tools, rsync, some scripts! »
● correct on all accou...
Docker: authoring images
● you can author « images »
– either with « run+commit » cycles, taking
snapshots
– or with a Doc...
Dockerfile example
FROM ubuntu
RUN apt-get -y update
RUN apt-get install -y g++
RUN apt-get install -y erlang-dev erlang-m...
Docker: sharing images
● you can push/pull images to/from a registry
(public or private)
● you can search images through a...
Docker: not sharing images
● private registry
– for proprietary code
– or security credentials
– or fast local access
Typical workflow
● code in local environment
(« dockerized » or not)
● each push to the git repo triggers a hook
● the hoo...
Hybrid clouds
● Docker is part of OpenStack « Havana »,
as a Nova driver + Glance translator
● typical workflow:
– code on...
Docker: the community
● Docker: >160 contributors
● latest milestone (0.6): 40 contributors
● GitHub repository: >600 forks
Docker: the ecosystem
● CoreOS (full distro based on Docker)
● Deis (PAAS; available)
● Dokku (mini-Heroku in 100 lines of...
Docker roadmap
● Today: Docker 0.6
– LXC
– AUFS
● Tomorrow: Docker 0.7
– LXC
– device-mapper thin snapshots (target: RHEL)...
Thank you! Questions?
http://docker.io/
https://github.com/dotcloud/docker
@docker
@jpetazzo
Upcoming SlideShare
Loading in...5
×

LXC, Docker, and the future of software delivery | LinuxCon 2013

10,474

Published on

Presentation of Docker and LXCs at LinuxCon 2013 in New Orleans, by Jerome Petazzoni

Published in: Technology
0 Comments
35 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
10,474
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
220
Comments
0
Likes
35
Embeds 0
No embeds

No notes for slide

Transcript of "LXC, Docker, and the future of software delivery | LinuxCon 2013"

  1. 1. LXC, Docker, and the future of software delivery Linuxcon – New Orleans, 2013 Jérôme Petazzoni, dotCloud Inc.
  2. 2. Outline ● Why Linux Containers? ● What are Linux Containers exactly? ● What do we need on top of LXC? ● Why Docker? ● What is Docker exactly? ● Where is it going?
  3. 3. Why Linux Containers? What are we trying to solve?
  4. 4. The Matrix From Hell
  5. 5. The Matrix From Hell
  6. 6. Many payloads ● backend services (API) ● databases ● distributed stores ● webapps
  7. 7. Many payloads ● Go ● Java ● Node.js ● PHP ● Python ● Ruby ● …
  8. 8. Many payloads ● CherryPy ● Django ● Flask ● Plone ● ...
  9. 9. Many payloads ● Apache ● Gunicorn ● uWSGI ● ...
  10. 10. Many payloads + your code
  11. 11. Many targets ● your local development environment ● your coworkers' developement environment ● your Q&A team's test environment ● some random demo/test server ● the staging server(s) ● the production server(s) ● bare metal ● virtual machines ● shared hosting + your dog's Raspberry Pi
  12. 12. Many targets ● BSD ● Linux ● OS X ● Windows
  13. 13. Many targets ● BSD ● Linux ● OS X ● Windows
  14. 14. The Matrix From Hell Static website ? ? ? ? ? ? ? Web frontend ? ? ? ? ? ? ? background workers ? ? ? ? ? ? ? User DB ? ? ? ? ? ? ? Analytics DB ? ? ? ? ? ? ? Queue ? ? ? ? ? ? ? Development VM QA Server Single Prod Server Onsite Cluster Public Cloud Contributor’s laptop Customer Servers
  15. 15. Real-world analogy: containers
  16. 16. Many products ● clothes ● electronics ● raw materials ● wine ● …
  17. 17. Many transportation methods ● ships ● trains ● trucks ● ...
  18. 18. Another matrix from hell ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
  19. 19. Solution to the transport problem: the intermodal shipping container
  20. 20. Solution to the transport problem: the intermodal shipping container ● 90% of all cargo now shipped in a standard container ● faster and cheaper to load and unload on ships (by an order of magnitude) ● less theft, less damage ● freight cost used to be >25% of final goods cost, now <3% ● 5000 ships deliver 200M containers per year
  21. 21. Solution to the deployment problem: the Linux container
  22. 22. Linux containers... ● run everywhere – regardless of kernel version – regardless of host distro – (but container and host architecture must match) ● run anything – if it can run on the host, it can run in the container – i.e., if it can run on a Linux kernel, it can run
  23. 23. What are Linux Containers exactly?
  24. 24. High level approach: it's a lightweight VM ● own process space ● own network interface ● can run stuff as root ● can have its own /sbin/init (different from the host)
  25. 25. Low level approach: it's chroot on steroids ● can also not have its own /sbin/init ● container = isolated process(es) ● share kernel with host ● no device emulation (neither HVM nor PV)
  26. 26. Separation of concerns: Dave the Developer ● inside my container: – my code – my libraries – my package manager – my app – my data
  27. 27. Separation of concerns: Oscar the Ops guy ● outside the container: – logging – remote access – network configuration – monitoring
  28. 28. How does it work? Isolation with namespaces ● pid ● mnt ● net ● uts ● ipc ● user
  29. 29. How does it work? Isolation with cgroups ● memory ● cpu ● blkio ● devices
  30. 30. Efficiency: almost no overhead ● processes are isolated, but run straight on the host ● CPU performance = native performance ● memory performance = a few % shaved off for (optional) accounting ● network performance = small overhead; can be optimized to zero overhead
  31. 31. Efficiency: storage-friendly ● unioning filesystems (AUFS, overlayfs) ● snapshotting filesystems (BTRFS, ZFS) ● copy-on-write (thin snapshots with LVM or device-mapper) This wasn't part of LXC at first; but you definitely want it!
  32. 32. Efficiency: storage-friendly ● provisioning now takes a few milliseconds ● … and a few kilobytes ● creating a new base/image/whateveryoucallit takes a few seconds
  33. 33. Docker
  34. 34. What's Docker? ● Open Source engine to commoditize LXC ● using copy-on-write for quick provisioning ● allowing to create and share images ● propose a standard format for containers
  35. 35. Yes, but... ● « I don't need Docker; I can do all that stuff with LXC tools, rsync, some scripts! » ● correct on all accounts; but it's also true for apt, dpkg, rpm, yum, etc. ● the whole point is to commoditize, i.e. make it ridiculously easy to use
  36. 36. Docker: authoring images ● you can author « images » – either with « run+commit » cycles, taking snapshots – or with a Dockerfile (=source code for a container) – both ways, it's ridiculously easy ● you can run them – anywhere – multiple times
  37. 37. Dockerfile example FROM ubuntu RUN apt-get -y update RUN apt-get install -y g++ RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe ... RUN apt-get install -y libmozjs185-dev libicu-dev libtool ... RUN apt-get install -y make wget RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf- RUN cd /tmp/apache-couchdb-* && ./configure && make install RUN printf "[httpd]nport = 8101nbind_address = 0.0.0.0" > /usr/local/etc/couchdb/local.d/docker.ini EXPOSE 8101 CMD ["/usr/local/bin/couchdb"]
  38. 38. Docker: sharing images ● you can push/pull images to/from a registry (public or private) ● you can search images through a public index ● dotCloud maintains a collection of base images (Ubuntu, Fedora...) ● satisfaction guaranteed or your money back
  39. 39. Docker: not sharing images ● private registry – for proprietary code – or security credentials – or fast local access
  40. 40. Typical workflow ● code in local environment (« dockerized » or not) ● each push to the git repo triggers a hook ● the hook tells a build server to clone the code and run « docker build » (using the Dockerfile) ● the containers are tested (nosetests, Jenkins...), and if the tests pass, pushed to the registry ● production servers pull the containers and run them ● for network services, load balancers are updated
  41. 41. Hybrid clouds ● Docker is part of OpenStack « Havana », as a Nova driver + Glance translator ● typical workflow: – code on local environment – push container to Glance-backed registry – run and manage containers using OpenStack APIs ● Docker confirmed to work with: Digital Ocean, EC2, Joyent, Linode, and many more (not praising a specific vendor, just pointing that it « just works »)
  42. 42. Docker: the community ● Docker: >160 contributors ● latest milestone (0.6): 40 contributors ● GitHub repository: >600 forks
  43. 43. Docker: the ecosystem ● CoreOS (full distro based on Docker) ● Deis (PAAS; available) ● Dokku (mini-Heroku in 100 lines of bash) ● Flynn (PAAS; in development) ● Maestro (orchestration from a simple YAML file) ● OpenStack integration ● Shipper (fabric-like orchestration) And many more
  44. 44. Docker roadmap ● Today: Docker 0.6 – LXC – AUFS ● Tomorrow: Docker 0.7 – LXC – device-mapper thin snapshots (target: RHEL) ● The day after: Docker 1.0 – LXC, libvirt, qemu, KVM, OpenVZ, chroot… – multiple storage back-ends – plugins
  45. 45. Thank you! Questions? http://docker.io/ https://github.com/dotcloud/docker @docker @jpetazzo
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×