Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Deploy services 
reliably & consistently 
• If it works locally, it will work on the server 
• With exactly the same behav...
Just like the real thing 
• Work in dev environment 
• Other services (databases etc.) in containers 
• Whenever you want ...
Better! 
Faster! 
Stronger!
15 Months Later: An Incredible Platform and Ecosystem 
Community 
460+ Contributors 250+ Meetups on Docker 
2.75M Download...
55 People and a Turtle 
Now up to 55 
people 
(and our pet 
turtle, Gordon)
…to Build, Ship, and Run 
Docker Hub 
Build Ship 
Run 
Source 
Dev 
QA 
Staging 
Linux OS 
Doocckkeerr 
Linux OS 
Linux OS...
An Open Platform… 
API 
Engine Hub 
open source software at the heart 
of the Docker platform 
cloud-based platform servic...
An Introduction to 
the Docker Engine
It’s an image builder.
Quagga Quagga 
Haproxy Haproxy 
HTTP Service HTTP Service HTTP Service 
Database 
BGP / OSPF 
Database 
{ 
{ 
{ 
image #1:...
BGP / OSPF 
Quagga Quagga 
HTTP Service HTTP Service HTTP Service 
Zookeeper 
Haproxy Haproxy 
Zookeeper 
{ 
{ 
image #1: ...
Do it! 
‣ Satisfied with your local build? 
‣ Push it to a registry (public or private) 
‣ Run it (automatically!) in CI/C...
Demo!
Installing Docker on EC2 
#!/bin/bash -x 
aws ec2 run-instances  
--image-id ami-e55a648c  
--key-name mykey  
--user-data...
#!/bin/bash -x 
aws ec2 run-instances  
--image-id ami-e55a648c  
--key-name my-key  
--user-data "#include https://get.do...
An ecosystem 
libcontainer libchan 
Engine + + DockerHub 
libswarm
libcontainer… 
a story of Linux 
namespaces
libcontainer… 
• Standalone project 
• Contributors: 
• RedHat 
• Google 
• Parallels (OpenVZ) 
• Ubuntu / LXC
User namespace
Security
misconceptions 
• Docker is not secure 
• Docker should be compared to VM security
a security product 
• Docker Engine is a security product. 
• It provides a wrapper around processes 
• Provides a path to...
a work in progress… 
That said… 
Security was not a priority for the 1.0 release 
Security is a priority post-1.0 
Big iss...
libchan 
a lightweight communication 
protocol for distributed systems 
Tuesday, June 10, 14 25
libchan 
Like Go channels over the network 
Simple message passing 
Synchronization d 
without sharing state 
Raw socket p...
libchan 
Available transports: 
SPDY/TLS, 
websocket, 
raw TCP, 
high-perf unix sockets (with fd passing), 
in-memory go c...
libswarm 
A minimalist toolkit 
to compose network services 
Tuesday, June 10, 14 39
libswarm 
A standard interface to combine and organize 
services in a distributed system. 
Compose complex architectures f...
or Consul or Helios or Centurion 
Shipper or Geard or Mesos or Coreos/Fleet 
EC2 Rackspace GCE Orchard Tutum 
Tuesday, Jun...
Images… 
and a new way of looking 
at infrastructure
Images on HW 
is usually mutable 
Hardware 
Image 
Linux 
Installs Chef 
Chef 
Image' 
Linux 
Chef 
Creates 
Replaces 
Run...
Ephemeral environments 
are (somewhat) immutable. 
Hypervisor 
Image 
Linux 
Chef 
Runs 
Image' 
Linux 
Chef 
Chef 
Runs 
...
Containers are like ephemeral VMs 
Docker 
Image 
Linux 
Chef 
Runs 
Image' 
Linux 
Chef 
Chef 
Runs 
Configures 
Containe...
Chef-for-runtime 
$ cat Dockerfile 
FROM fedora 
RUN yum update;  
yum -y install chef 
ADD http://x/receipes.tar.gz /opt/...
Containers 
are 
THINGS
X
X
Pets vs Cattle 
Servers vs Things
LET US 
BAKE 
IMAGES! 
Let us 
images!
Burning configuration 
into images. 
Image 
Linux 
Docker Initiates Creates Container 
Chef 
Build Creates 
Runs 
Chef 
Co...
Bakery Chef 
$ cat Dockerfile 
FROM fedora 
RUN yum update;  
yum -y install chef" 
ADD http://x/receipes.tar.gz /opt/chef...
Expanded view: 
Burning configuration into 
images. 
Docker Initiates Image tag 
Image' 
Linux 
Chef 
Chef 
Build 
Creates...
Anatomy of a Docker 
+Chef build & run 
Image 
Linux 
Docker Initiates Creates Container 
Chef 
Chef 
Runs 
Configures 
Bu...
For All The Things! 
$ cat Dockerfile 
FROM fedora 
RUN yum update;  
yum -y install chef 
ADD http://x/receipes.tar.gz /o...
Does it converge? 
$ docker build —rm . 
$ echo $? # pass or fail 
(This is great use of Docker as an 
alternative to VMs ...
Managing Docker 
at scale
Creating Containers 
is Easy
Managing them 
SUCKS 
needs improvement
Management Ecosystem 
Mesos 
ClusterHQ Clocker Flynn
PaaS ecosystem
Configuration / Infrastructure 
Management 
• Chef 
• Puppet 
• Salt 
• Ansible 
• CFEngine 
• etc…
Container Inventory 
• discoverd / sdutil 
• serf 
• skydock 
• others?
Change 
Management Tools
Chef 
# using https://github.com/bflad/chef-docker 
$ cat cookbooks/docker-registry/default.rb 
# Pull latest image 
docke...
Chef container 
$ knife container docker init docker 
-r 'recipe[apache2]' -z -b
Puppet 
docker::run { 'helloworld': 
image => 'base', 
command => '/bin/sh -c "while true; do echo 
hello world; sleep 1; ...
Ansible 
- hosts: web 
sudo: yes 
tasks: 
- name: ensure redis container is running 
docker: image=crosbymichael/redis nam...
Orchestration
fig - local orchestration 
——fig.yml—— 
web: 
build: . 
command: python app.py 
ports: 
- "5000:5000" 
volumes: 
- .:/code...
figleaf - containerized fig 
Testing / dev: 
$ docker run --privileged -v $PWD:/opt/figapp ewindisch/figleaf 
Production -...
Orchestration 7KH'RFNHUSOXJLQIRU+HDW 
for Docker 
%XVLQJWKHSOXJLQ+HDWFDQWDONGLUHFWOWR'RFNHU 
with OpenStack Heat 
DockerIn...
Heat Workflow 
Heat API 
VM 
Docker 
Nova resource Nova 
Docker resource 
Container1 
Container2 
Container3 
HOT
heat_template_version: 2013-05-23 
description: shared volumes example 
resources: 
my_instance: 
type: OS::Nova::Server 
...
OpenStack’s new 
container service… 
nova-api 
nova-compute 
instance 
instance 
keystone 
docker 
containers-api 
swarm-p...
OpenStack’s new 
container service… 
nova-api 
nova-compute 
instance 
instance 
keystone 
docker 
containers-api 
swarm-p...
OpenStack’s new 
container service… 
nova-api 
nova-compute 
instance 
instance 
keystone 
docker 
containers-api 
swarm-p...
OSC%Client%Use%Case% 
$%source%./openrc% 
$%osc%containerIcreate%IIport%pub:222:22%IIdaemon%II 
image%uuid%IIcmd%“/usr/sbi...
Docker%Client%Use%Case% 
$%source%./openrc% 
$%export%DOCKER_SERVER=h:ps://…% 
$%docker%run%Ip%pub:222:22%Id%foo%/usr/sbin...
insert here 
Mesos, 
Clocker, 
Fleet, 
Flynn, 
Deis, 
Kubernetes, 
etc
Conclusion…
X
X
Containers 
are 
THINGS
Docker is a valuable 
component in your 
security story.
Q  A 
@ewindisch
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
Upcoming SlideShare
Loading in …5
×

The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration

2,129 views

Published on

An introduction to Docker, its ecosystem, and information on deploying and orchestrating docker.

Published in: Software

The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration

  1. 1. Deploy services reliably & consistently • If it works locally, it will work on the server • With exactly the same behavior • Regardless of versions • Regardless of distros • Regardless of dependencies
  2. 2. Just like the real thing • Work in dev environment • Other services (databases etc.) in containers • Whenever you want to test « for real »: • Build in seconds • Run instantly
  3. 3. Better! Faster! Stronger!
  4. 4. 15 Months Later: An Incredible Platform and Ecosystem Community 460+ Contributors 250+ Meetups on Docker 2.75M Downloads 6.7K Projects on GitHub Official Repos & 14K+ Dockerized Apps Support Enterprise Support Robust Documentation Implementation, Integration, Training Network of Partners The Docker Platform ! Docker Engine Docker Hub ! Build, Ship, and Run Partners Content Users
  5. 5. 55 People and a Turtle Now up to 55 people (and our pet turtle, Gordon)
  6. 6. …to Build, Ship, and Run Docker Hub Build Ship Run Source Dev QA Staging Linux OS Doocckkeerr Linux OS Linux OS Doocckkeerr Infrastructure Management Physical Infrastructure Management Virtual Cloud Source Code Repository DockerFile Docker Boot2Docker Linux OS Doocckkeerr TEST TEST TEST TEST TEST GCE RAX IBM Mac/Win Dev Machine Analytics DB Prod Machine Doocckkeerr ++ Users Collab Provenance Policy Registries Public Curated Private Docker Hub API Third Party Tools Prod Machine Prod Machine VM DoDcokcekrer VM DoDcokcekrer VM DoDcokcekrer QA Machine
  7. 7. An Open Platform… API Engine Hub open source software at the heart of the Docker platform cloud-based platform services for distributed applications API
  8. 8. An Introduction to the Docker Engine
  9. 9. It’s an image builder.
  10. 10. Quagga Quagga Haproxy Haproxy HTTP Service HTTP Service HTTP Service Database BGP / OSPF Database { { { image #1: your-favorite-lb image #2: apache, nginx, etc image #3: mysql, zookeeper, etc
  11. 11. BGP / OSPF Quagga Quagga HTTP Service HTTP Service HTTP Service Zookeeper Haproxy Haproxy Zookeeper { { image #1: your-favorite-lb image #2: consensus-web-service Zookeeper
  12. 12. Do it! ‣ Satisfied with your local build? ‣ Push it to a registry (public or private) ‣ Run it (automatically!) in CI/CD ‣ Run it in production ‣ Happiness! ‣ Something goes wrong? Rollback painlessly!
  13. 13. Demo!
  14. 14. Installing Docker on EC2 #!/bin/bash -x aws ec2 run-instances --image-id ami-e55a648c --key-name mykey --user-data ”#include https://get.docker.io"
  15. 15. #!/bin/bash -x aws ec2 run-instances --image-id ami-e55a648c --key-name my-key --user-data "#include https://get.docker.io” ip=$(aws ec2 describe-instances --output json --filter Name=instance-state-name,Values=running | python -c 'import json; import sys; print json.load(sys.stdin) [“Reservations”][0]["Instances"][0]["PublicIpAddress"]') ssh ubuntu@$ip sudo docker run cirros
  16. 16. An ecosystem libcontainer libchan Engine + + DockerHub libswarm
  17. 17. libcontainer… a story of Linux namespaces
  18. 18. libcontainer… • Standalone project • Contributors: • RedHat • Google • Parallels (OpenVZ) • Ubuntu / LXC
  19. 19. User namespace
  20. 20. Security
  21. 21. misconceptions • Docker is not secure • Docker should be compared to VM security
  22. 22. a security product • Docker Engine is a security product. • It provides a wrapper around processes • Provides a path toward attestation of arbitrary processes (Trusted Compute) • You can use VMs to wrap containers
  23. 23. a work in progress… That said… Security was not a priority for the 1.0 release Security is a priority post-1.0 Big issues are still be discovered at a rapid clip… Big issues are being resolved at a rapid clip.
  24. 24. libchan a lightweight communication protocol for distributed systems Tuesday, June 10, 14 25
  25. 25. libchan Like Go channels over the network Simple message passing Synchronization d without sharing state Raw socket passing: channels as gateways to any other protocol Nesting: channels can send channels Tuesday, June 10, 14 26
  26. 26. libchan Available transports: SPDY/TLS, websocket, raw TCP, high-perf unix sockets (with fd passing), in-memory go channels. Designed to be simple and portable Tuesday, June 10, 14 27
  27. 27. libswarm A minimalist toolkit to compose network services Tuesday, June 10, 14 39
  28. 28. libswarm A standard interface to combine and organize services in a distributed system. Compose complex architectures from standard building blocks Avoid vendor lock-in by swapping any service out with another Pick services from a built-in library, or write your own with a simple API. Tuesday, June 10, 14 40
  29. 29. or Consul or Helios or Centurion Shipper or Geard or Mesos or Coreos/Fleet EC2 Rackspace GCE Orchard Tutum Tuesday, June 10, 14 38
  30. 30. Images… and a new way of looking at infrastructure
  31. 31. Images on HW is usually mutable Hardware Image Linux Installs Chef Chef Image' Linux Chef Creates Replaces Runs
  32. 32. Ephemeral environments are (somewhat) immutable. Hypervisor Image Linux Chef Runs Image' Linux Chef Chef Runs Configures VM Accesses COW
  33. 33. Containers are like ephemeral VMs Docker Image Linux Chef Runs Image' Linux Chef Chef Runs Configures Container Accesses COW Hypervisor Image Linux Chef Runs Image' Linux Chef Chef Runs Configures VM Accesses COW Hypervisor VM Docker Container
  34. 34. Chef-for-runtime $ cat Dockerfile FROM fedora RUN yum update; yum -y install chef ADD http://x/receipes.tar.gz /opt/chef" ADD solo.rb /etc/chef/solo.rb" CMD chef-solo -c /etc/chef/solo.rb ; ! apachectl start
  35. 35. Containers are THINGS
  36. 36. X
  37. 37. X
  38. 38. Pets vs Cattle Servers vs Things
  39. 39. LET US BAKE IMAGES! Let us images!
  40. 40. Burning configuration into images. Image Linux Docker Initiates Creates Container Chef Build Creates Runs Chef Configures
  41. 41. Bakery Chef $ cat Dockerfile FROM fedora RUN yum update; yum -y install chef" ADD http://x/receipes.tar.gz /opt/chef" ADD solo.rb /etc/chef/solo.rb" RUN chef-solo -c /etc/chef/solo.rb
  42. 42. Expanded view: Burning configuration into images. Docker Initiates Image tag Image' Linux Chef Chef Build Creates Image Linux Chef Creates Runs Creates References 1 2
  43. 43. Anatomy of a Docker +Chef build & run Image Linux Docker Initiates Creates Container Chef Chef Runs Configures Build Creates Runs Chef Configures Stage 1 Stage 2
  44. 44. For All The Things! $ cat Dockerfile FROM fedora RUN yum update; yum -y install chef ADD http://x/receipes.tar.gz /opt/chef" ADD solo-stage1.rb /etc/chef/solo-stage1.rb" ADD solo-stage2.rb /etc/chef/solo-stage2.rb" RUN chef-solo -c /etc/chef/solo-stage1.rb" CMD chef-solo -c /etc/chef/solo-stage2.rb; " apachectl start
  45. 45. Does it converge? $ docker build —rm . $ echo $? # pass or fail (This is great use of Docker as an alternative to VMs for testing Chef recipes targeting non-Docker production systems)
  46. 46. Managing Docker at scale
  47. 47. Creating Containers is Easy
  48. 48. Managing them SUCKS needs improvement
  49. 49. Management Ecosystem Mesos ClusterHQ Clocker Flynn
  50. 50. PaaS ecosystem
  51. 51. Configuration / Infrastructure Management • Chef • Puppet • Salt • Ansible • CFEngine • etc…
  52. 52. Container Inventory • discoverd / sdutil • serf • skydock • others?
  53. 53. Change Management Tools
  54. 54. Chef # using https://github.com/bflad/chef-docker $ cat cookbooks/docker-registry/default.rb # Pull latest image docker_image 'samalba/docker-registry' ! # Run container exposing ports docker_container 'samalba/docker-registry' do detach true port '5000:5000' env 'SETTINGS_FLAVOR=local' volume '/mnt/docker:/docker-storage' end $ knife ec2 server create # yada yada yada
  55. 55. Chef container $ knife container docker init docker -r 'recipe[apache2]' -z -b
  56. 56. Puppet docker::run { 'helloworld': image => 'base', command => '/bin/sh -c "while true; do echo hello world; sleep 1; done"', ports => ['4444', '4555'], links => ['mysql:db'], use_name => true, volumes => ['/var/lib/couchdb', '/var/log'], volumes_from => '6446ea52fbc9', memory_limit => 10485760, # bytes username => 'example', hostname => 'example.com', env => ['FOO=BAR', 'FOO2=BAR2'], dns => ['8.8.8.8', '8.8.4.4'], restart_service => true, }
  57. 57. Ansible - hosts: web sudo: yes tasks: - name: ensure redis container is running docker: image=crosbymichael/redis name=redis - name: ensure redis_ambassador container is running docker: image=svendowideit/ambassador ports=6379:6379 links=redis:redis name=redis_ambassador_ansible
  58. 58. Orchestration
  59. 59. fig - local orchestration ——fig.yml—— web: build: . command: python app.py ports: - "5000:5000" volumes: - .:/code links: - redis redis: image: orchardup/redis
  60. 60. figleaf - containerized fig Testing / dev: $ docker run --privileged -v $PWD:/opt/figapp ewindisch/figleaf Production - image generation: $ echo “FROM ewindisch/figleaf” >> Dockerfile $ docker build -t my_img . $ docker run —privileged my_img figleaf images may be plugged into “dumb” orchestration.
  61. 61. Orchestration 7KH'RFNHUSOXJLQIRU+HDW for Docker %XVLQJWKHSOXJLQ+HDWFDQWDONGLUHFWOWR'RFNHU with OpenStack Heat DockerInc::Docker: :Container VMs Baremetal
  62. 62. Heat Workflow Heat API VM Docker Nova resource Nova Docker resource Container1 Container2 Container3 HOT
  63. 63. heat_template_version: 2013-05-23 description: shared volumes example resources: my_instance: type: OS::Nova::Server properties: key_name: ewindisch_key image: ubuntu-precise flavor: m1.large user_data: #include https://get.docker.io ftp_container: type: DockerInc::Docker::Container properties: docker_endpoint: { get_attr: [my_instance, first_address] } image: mikz/vsftpd ports: [ “21:21” ] volumes: [ “/ftp” ] name: “FTP” apache_container: type: DockerInc::Docker::Container properties: docker_endpoint: { get_attr: [my_instance, first_address] } image: fedora/apache ports: [ “80:80” ] volumes-from: “FTP” cmd: “rm -rf /var/www; ln -s /ftp /var/www; /run-apache.sh”
  64. 64. OpenStack’s new container service… nova-api nova-compute instance instance keystone docker containers-api swarm-proxy docker neutron User
  65. 65. OpenStack’s new container service… nova-api nova-compute instance instance keystone docker containers-api swarm-proxy docker neutron User
  66. 66. OpenStack’s new container service… nova-api nova-compute instance instance keystone docker containers-api swarm-proxy docker neutron User swarmd
  67. 67. OSC%Client%Use%Case% $%source%./openrc% $%osc%containerIcreate%IIport%pub:222:22%IIdaemon%II image%uuid%IIcmd%“/usr/sbin/sshd%–D”% $%osc%containerIshow%DEADBEEF%|%grep%ports% ports:%[12.34.56.78:222]% $%ssh%Ip%222%12.34.56.78% foo$% From: https://wiki.openstack.org/w/images/5/51/Containers_Proposal.pdf
  68. 68. Docker%Client%Use%Case% $%source%./openrc% $%export%DOCKER_SERVER=h:ps://…% $%docker%run%Ip%pub:222:22%Id%foo%/usr/sbin/sshd%ID% DEADBEEF% $%osc%containerIshow%DEADBEEF%|%grep%ports% ports:%[12.34.56.78:222]% $%ssh%Ip%222%12.34.56.78% foo$% From: https://wiki.openstack.org/w/images/5/51/Containers_Proposal.pdf
  69. 69. insert here Mesos, Clocker, Fleet, Flynn, Deis, Kubernetes, etc
  70. 70. Conclusion…
  71. 71. X
  72. 72. X
  73. 73. Containers are THINGS
  74. 74. Docker is a valuable component in your security story.
  75. 75. Q A @ewindisch

×