DOCKER
Gavin Heavyside - ACCU 2015 - @gavinheavyside 1
Gavin Heavyside - ACCU 2015 - @gavinheavyside 2
Gavin Heavyside - ACCU 2015 - @gavinheavyside 3
Gavin Heavyside - ACCU 2015 - @gavinheavyside 4
Gavin Heavyside - ACCU 2015 - @gavinheavyside 5
Gavin Heavyside - ACCU 2015 - @gavinheavyside 6
Gavin Heavyside - ACCU 2015 - @gavinheavyside 7
Gavin Heavyside - ACCU 2015 - @gavinheavyside 8
Goals
Gavin Heavyside - ACCU 2015 - @gavinheavyside 9
ship it
Gavin Heavyside - ACCU 2015 - @gavinheavyside 10
Docker Components
• Engine
• Hub
• Compose
• Swarm
• Machine
Gavin Heavyside - ACCU 2015 - @gavinheavyside 11
Docker Engine
Gavin Heavyside - ACCU 2015 - @gavinheavyside 12
Docker Client-Server
┌───────┐ ┌───────────────────────────────────┐
│Client ├┐ │ ┌──────────────────┐ ┌──────────┐│
└┬───...
Docker Client
attach build commit cp create diff events
exec export history images import info
inspect kill load login log...
Running a Docker
Container
docker run -i -t ubuntu /bin/bash
Gavin Heavyside - ACCU 2015 - @gavinheavyside 15
Running a Docker
Container
docker run -i -t ubuntu /bin/bash
• Pulls the ubuntu image
Gavin Heavyside - ACCU 2015 - @gavin...
Running a Docker
Container
docker run -i -t ubuntu /bin/bash
• Creates a new container
Gavin Heavyside - ACCU 2015 - @gavi...
Running a Docker
Container
docker run -i -t ubuntu /bin/bash
• Allocates a filesystem and mounts a R/W layer
Gavin Heavysid...
Running a Docker
Container
docker run -i -t ubuntu /bin/bash
• Allocates a network / bridge interface
Gavin Heavyside - AC...
Running a Docker
Container
docker run -i -t ubuntu /bin/bash
• Sets up an IP address
Gavin Heavyside - ACCU 2015 - @gavinh...
Running a Docker
Container
docker run -i -t ubuntu /bin/bash
• Executes your process
Gavin Heavyside - ACCU 2015 - @gavinh...
Running a Docker
Container
docker run -i -t ubuntu /bin/bash
• Captures and provides application output
Gavin Heavyside - ...
How it works
(Short Version)
• Written in Go
• Takes advantange of Linux kernel features
• Namespaces
• Control Groups (cg...
Namespaces
• Separation of groups of processes
• Can't 'see' resources in other groups
• PID namespace, network, mount, IP...
Namespaces
• Docker creates a set of namespaces for each
container.
• Isolation layer
• each aspect of a container runs in...
Control Groups (cgroups)
• limit, account, and isolate resources used by a
collection of processes
• CPU, memory, disk I/O...
cgroups
• allow Docker to share available hardware
resources to containers
• set up limits and constraints, if required
Ga...
Setting Resource Limits
docker run -m 256m --cpu-shares 512 yourapp
Gavin Heavyside - ACCU 2015 - @gavinheavyside 28
Union File Systems
• Layer files and dirs
• Can be from different file systems
• Present as a single filesystem
• Can have RO...
┌───────────────────────────────────────┐
│ Writeable Layer Container │
└───────────────────────────────────────┘
┌───────...
Union File Systems
• UnionFS
• aufs
• btrfs
• and more...
Gavin Heavyside - ACCU 2015 - @gavinheavyside 31
libcontainer
• https://github.com/docker/libcontainer
• Default supported container format
• Creates containers with names...
Other container
technologies
• Solaris Zones
• lmctfy
• rkt
• LXC
• BSD Jails
Gavin Heavyside - ACCU 2015 - @gavinheavysid...
Building a Container
• Write a Dockerfile
• build the image with docker build
• run it with docker run
• Share by pushing t...
The Dockerfile
• Plain text file
• Series of directives
• add files
• expose ports
• execute commands
• configure runtime
Gav...
The Dockerfile
FROM busybox
RUN mkdir -p /usr/local/bin
COPY ./hello /usr/local/bin/hello
CMD ["/usr/local/bin/hello"]
Gav...
FROM
FROM ubuntu:14.04
• Base image (& tag) to start building from
MAINTAINER
MAINTAINER Peter V "venkman@1984.com"
• Who ...
RUN
RUN apt-get update && apt-get -y upgrade
• Execute command in a new layer and commit
• defaults to using /bin/sh
• RUN...
CMD
CMD ["executable","param1","param2"]
• Default command to execute
Gavin Heavyside - ACCU 2015 - @gavinheavyside 39
EXPOSE
RUN apt-get install nginx
EXPOSE 80
• Ports for the continer to listen on
• Used for interconnecting linked contain...
ENV
ENV FOO=bar
• Set an environment variable in the container
ADD / COPY
COPY /my/src /opt/container/src
• Copy content t...
USER
USER nginx
• Set the UID for the image and any following
directives
WORKDIR
WORKDIR /path/to/workdir
• set the workin...
ONBUILD
ONBUILD bin/rake db:assets:precompile
• Trigger instruction to run when image is used
as a base for another build
...
Gavin Heavyside - ACCU 2015 - @gavinheavyside 44
Dockerfile Tips
• Choose your base image wisely
• Do the expensive work first
• Take advantage of caching and layering
• Us...
Gavin Heavyside - ACCU 2015 - @gavinheavyside 46
Pulling Images From a Registry
docker pull elasticsearch
docker pull private.globocorp.com/elasticsearch
Gavin Heavyside -...
Tags
docker pull nginx:latest
docker pull nginx:1.7.11
----- ------
| |
repo tag
Gavin Heavyside - ACCU 2015 - @gavinheavy...
Running your own
registry
• registry (Docker < 1.6)
• distribution (Docker 1.6+)
• dogestry
Gavin Heavyside - ACCU 2015 - ...
Storage
• Transient
• Local
• Persistent (portable)
• Probably the hardest thing to get right at
the moment
Gavin Heavysid...
Volumes
Gavin Heavyside - ACCU 2015 - @gavinheavyside 51
VOLUME directive
• Indicates the container wants to use external
storagee
--volumes-from
• mount VOLUME paths from contain...
Persistent Storage
docker run -v /local/path:/container/
path elasticsearch
• local path on filesystem is mounted in
contai...
Linking Containers
docker run -d -p 80:80 --name app1 app1:latest
docker run --link app1:app1 app2:latest
• The code runni...
Tailoring your app for Docker
• Docker works best when containers have a
single responsibility
• not necessarily a single ...
The 12-Factor App
• http://12factor.net
• Codebase
• Dependencies
• Config
• Backing Services
• Build;Release;Run
• Process...
The 12-Factor App
• Port Binding
• Concurrency
• Disposability
• Dev/Prod Parity
• Logs
• Admin Processes
Gavin Heavyside ...
12 Factor - Dependencies
• http://12factor.net/dependencies
• Explicitly declare and isolate dependencies
• No implicit de...
12 Factor - Config
• http://12factor.net/config
• Store config in the environment
• Config is everything that can change betw...
12 Factor - Port Binding
• http://12factor.net/port-binding
• App should be entirely self-contained
• Expose services via ...
12 Factor - Dev/Prod Parity
• http://12factor.net/dev-prod-parity
• Keep development, staging, and production
as similar a...
12 Factor - Logs
• http://12factor.net/logs
• Treat logs as event streams
• Log to stdout
• Collect, rotate, and centralis...
Computation Containers
• A program Q, with preconditions P, will
produce output R
• P and Q can change when we move betwee...
Toolchain in a container
$ docker run --rm -v `pwd`:/src 
-w /src golang:1.4 go build hello.go
Gavin Heavyside - ACCU 2015...
Toolchain in a container
$ docker run --rm -v `pwd`:/src 
-w /src golang:1.4 go build hello.go
BUT - I'm on OS X and my bo...
Toolchain in a container
$ docker run --rm -v `pwd`:/src 
-w /src golang:1.4 go build
$ ./hello
exec format error: hello
$...
Toolchain in a container
$ docker run --rm -v src:/src 
-e "GOOS=darwin" 
-w /src golang:1.4-cross 
go build
$ file hello
...
Choosing a base image
• Enough foundation, but not too much
• Security and hardening, provenance
• Reuseability
• Compatib...
The PID 1 Reaping
Problem
• Unix processes are modelled like a tree
• PID 1 is the top-most process
• Typically this is an...
Gavin Heavyside - ACCU 2015 - @gavinheavyside 70
What to do?
• Nothing
• Specify a different init
• runit
• supervisord
• phusion/baseimage-docker
• other init process
Gav...
Minimalist Host OS
Gavin Heavyside - ACCU 2015 - @gavinheavyside 72
Features of the New
Minimal OSes
• Small and lightweight
• Specialised, not general purpose
• Quick to install and boot
• ...
Features of the New
Minimal OSes
• Read-only system files
• Transactional platform updates
• Backup, rollback
• Delta patch...
Examples of Minimalist
OSes
• Snappy Ubuntu Core
• Project Atomic
• CoreOS
• Docker compatible, pushing own
containers
• R...
CoreOS
• Etcd
• Rkt
• Fleet
• Flannel
Gavin Heavyside - ACCU 2015 - @gavinheavyside 76
Docker on
Windows
Gavin Heavyside - ACCU 2015 - @gavinheavyside 77
Docker Client
Gavin Heavyside - ACCU 2015 - @gavinheavyside 78
Windows Links
• http://azure.microsoft.com/blog/tag/docker/
• http://azure.microsoft.com/blog/2015/04/08/
microsoft-unveil...
Cluster
Management
Gavin Heavyside - ACCU 2015 - @gavinheavyside 80
Cattle, Not Pets
• Not snowflakes, either
• Care about the service, not the server
• Easier said than done
Gavin Heavyside ...
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Host 1 │ │ Host 2 │ │ Host 3 │
│ │ │ │ │ │
│ │ │ │ │ │
│ │ │ │ │ │
│ ...
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Host 1 │ │ Host 2 │ │ Host 3 │
│ ┌──┐ │ │ │ │ │
│ │ │ │ │ │ │ │
│ └──...
┌──────────────┐ ┌──────────────┐
│ Host 1 │ │ Host 2 │
│ ┌──┐ │ │ │
│ │ │ │ │ │
│ └──┘ │ │ │
│ ┌──┐┌──┐┌──┐ │ │ │
│ │ ││ ...
Cluster Management
• Kubernetes
• Docker Swarm
• CoreOS Fleet
• AWS ECS
• Google Container Service
• More
Gavin Heavyside ...
Kubernetes
Gavin Heavyside - ACCU 2015 - @gavinheavyside 86
Kubernetes
• Abstract at the service level, not container
• Compose services from containers
• Dependencies
• CPU, RAM, pl...
Hosted Kubernetes
• Google Container Engine (Alpha)
• Hosted K8 on Google Cloud Platform
• Tectonic (Beta)
• by CoreOS
Gav...
AWS EC2 Container
Service
• Hosted Docker orchestration on EC2 (GA)
• Multi-container dependencies
• Placement and schedul...
Service Discovery
• How do your services talk to each other?
• How do they find each other in a dynamically
allocated clust...
Service Discovery
• Message buses (e.g. rabbitMQ)
• DNS
• Service Discovery Tools
• Load balancing and health checking
Gav...
Service Discovery Tools
• DNS
• SmartStack (nerve, synapse)
• Etcd (and SkyDNS)
• Consul
• More
Gavin Heavyside - ACCU 201...
Consul
• https://consul.io
• K/V, DNS interfaces, ACLs
• Services, health checks, load balancing
• serf gossip protocol, r...
Registrator
• https://github.com/gliderlabs/registrator
• Container watches Docker engine events,
dynamically registers se...
Logs
• Easier if containers log to stdout, saved on the
host
• Can mount log dir as a volume in container if
needed
• Cons...
Monitoring
• Some dedicated tools appearing, hosted and
open source
• Still an area with catching up to do
• Traditional t...
Wrapping Up
Gavin Heavyside - ACCU 2015 - @gavinheavyside 97
Gavin Heavyside - ACCU 2015 - @gavinheavyside 98
Image Credits
• minimalist room: https://www.flickr.com/
photos/colinsite/14089317769
• cluster: https://www.flickr.com/phot...
Image Credits
• goals: https://www.flickr.com/photos/
peterfuchs/1239399915
• complexity: https://www.flickr.com/photos/
bit...
Upcoming SlideShare
Loading in …5
×

Docker at ACCU2015

1,156 views

Published on

Whirlwind tour of Docker and related projects, presented at ACCU2015, Bristol, 22 April 2015

Published in: Software
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,156
On SlideShare
0
From Embeds
0
Number of Embeds
326
Actions
Shares
0
Downloads
14
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Docker at ACCU2015

  1. 1. DOCKER Gavin Heavyside - ACCU 2015 - @gavinheavyside 1
  2. 2. Gavin Heavyside - ACCU 2015 - @gavinheavyside 2
  3. 3. Gavin Heavyside - ACCU 2015 - @gavinheavyside 3
  4. 4. Gavin Heavyside - ACCU 2015 - @gavinheavyside 4
  5. 5. Gavin Heavyside - ACCU 2015 - @gavinheavyside 5
  6. 6. Gavin Heavyside - ACCU 2015 - @gavinheavyside 6
  7. 7. Gavin Heavyside - ACCU 2015 - @gavinheavyside 7
  8. 8. Gavin Heavyside - ACCU 2015 - @gavinheavyside 8
  9. 9. Goals Gavin Heavyside - ACCU 2015 - @gavinheavyside 9
  10. 10. ship it Gavin Heavyside - ACCU 2015 - @gavinheavyside 10
  11. 11. Docker Components • Engine • Hub • Compose • Swarm • Machine Gavin Heavyside - ACCU 2015 - @gavinheavyside 11
  12. 12. Docker Engine Gavin Heavyside - ACCU 2015 - @gavinheavyside 12
  13. 13. Docker Client-Server ┌───────┐ ┌───────────────────────────────────┐ │Client ├┐ │ ┌──────────────────┐ ┌──────────┐│ └┬──────┘├─┬─┼──│ Docker Daemon │ │Container ││ └─┬─────┘ │ │ └─────────────────┬┘ └──────────┘│ └───────┘ │ │ │ ┌──────────┐│ │ ┌───────┐ │ └──│Container ││ │ │Client │─┘ └──────────┘│ │ └───────┘ ┌──────────┐│ │ │Container ││ │ Docker Host └──────────┘│ └───────────────────────────────────┘ Gavin Heavyside - ACCU 2015 - @gavinheavyside 13
  14. 14. Docker Client attach build commit cp create diff events exec export history images import info inspect kill load login logout logs port pause ps pull push rename restart rm rmi run save search start stats stop tag top unpause version wait Gavin Heavyside - ACCU 2015 - @gavinheavyside 14
  15. 15. Running a Docker Container docker run -i -t ubuntu /bin/bash Gavin Heavyside - ACCU 2015 - @gavinheavyside 15
  16. 16. Running a Docker Container docker run -i -t ubuntu /bin/bash • Pulls the ubuntu image Gavin Heavyside - ACCU 2015 - @gavinheavyside 16
  17. 17. Running a Docker Container docker run -i -t ubuntu /bin/bash • Creates a new container Gavin Heavyside - ACCU 2015 - @gavinheavyside 17
  18. 18. Running a Docker Container docker run -i -t ubuntu /bin/bash • Allocates a filesystem and mounts a R/W layer Gavin Heavyside - ACCU 2015 - @gavinheavyside 18
  19. 19. Running a Docker Container docker run -i -t ubuntu /bin/bash • Allocates a network / bridge interface Gavin Heavyside - ACCU 2015 - @gavinheavyside 19
  20. 20. Running a Docker Container docker run -i -t ubuntu /bin/bash • Sets up an IP address Gavin Heavyside - ACCU 2015 - @gavinheavyside 20
  21. 21. Running a Docker Container docker run -i -t ubuntu /bin/bash • Executes your process Gavin Heavyside - ACCU 2015 - @gavinheavyside 21
  22. 22. Running a Docker Container docker run -i -t ubuntu /bin/bash • Captures and provides application output Gavin Heavyside - ACCU 2015 - @gavinheavyside 22
  23. 23. How it works (Short Version) • Written in Go • Takes advantange of Linux kernel features • Namespaces • Control Groups (cgroups) • Union File System • libcontainer Gavin Heavyside - ACCU 2015 - @gavinheavyside 23
  24. 24. Namespaces • Separation of groups of processes • Can't 'see' resources in other groups • PID namespace, network, mount, IPC, and more Gavin Heavyside - ACCU 2015 - @gavinheavyside 24
  25. 25. Namespaces • Docker creates a set of namespaces for each container. • Isolation layer • each aspect of a container runs in own namespace • does not have access outside it • some used by Docker: pid, net, ipc, mnt, uts Gavin Heavyside - ACCU 2015 - @gavinheavyside 25
  26. 26. Control Groups (cgroups) • limit, account, and isolate resources used by a collection of processes • CPU, memory, disk I/O, network, etc. • The basis of many container projects • Docker, LXC, lmctfy, Mesos, and more Gavin Heavyside - ACCU 2015 - @gavinheavyside 26
  27. 27. cgroups • allow Docker to share available hardware resources to containers • set up limits and constraints, if required Gavin Heavyside - ACCU 2015 - @gavinheavyside 27
  28. 28. Setting Resource Limits docker run -m 256m --cpu-shares 512 yourapp Gavin Heavyside - ACCU 2015 - @gavinheavyside 28
  29. 29. Union File Systems • Layer files and dirs • Can be from different file systems • Present as a single filesystem • Can have RO and RW layers Gavin Heavyside - ACCU 2015 - @gavinheavyside 29
  30. 30. ┌───────────────────────────────────────┐ │ Writeable Layer Container │ └───────────────────────────────────────┘ ┌───────────────────────────────────────┐ │ ADD apache Image │ └───────────────────────────────────────┘ ┌───────────────────────────────────────┐ │ ADD emacs Image │ └───────────────────────────────────────┘ ┌───────────────────────────────────────┐ │ FROM debian Base Image │ └───────────────────────────────────────┘ ┌───────────────────────────────────────┐ │ Kernel │ └───────────────────────────────────────┘ Gavin Heavyside - ACCU 2015 - @gavinheavyside 30
  31. 31. Union File Systems • UnionFS • aufs • btrfs • and more... Gavin Heavyside - ACCU 2015 - @gavinheavyside 31
  32. 32. libcontainer • https://github.com/docker/libcontainer • Default supported container format • Creates containers with namespaces, cgroups, capabilities, and filesystem access controls • Manages lifecycle of the container Gavin Heavyside - ACCU 2015 - @gavinheavyside 32
  33. 33. Other container technologies • Solaris Zones • lmctfy • rkt • LXC • BSD Jails Gavin Heavyside - ACCU 2015 - @gavinheavyside 33
  34. 34. Building a Container • Write a Dockerfile • build the image with docker build • run it with docker run • Share by pushing to a registry Gavin Heavyside - ACCU 2015 - @gavinheavyside 34
  35. 35. The Dockerfile • Plain text file • Series of directives • add files • expose ports • execute commands • configure runtime Gavin Heavyside - ACCU 2015 - @gavinheavyside 35
  36. 36. The Dockerfile FROM busybox RUN mkdir -p /usr/local/bin COPY ./hello /usr/local/bin/hello CMD ["/usr/local/bin/hello"] Gavin Heavyside - ACCU 2015 - @gavinheavyside 36
  37. 37. FROM FROM ubuntu:14.04 • Base image (& tag) to start building from MAINTAINER MAINTAINER Peter V "venkman@1984.com" • Who ya gonna call? Gavin Heavyside - ACCU 2015 - @gavinheavyside 37
  38. 38. RUN RUN apt-get update && apt-get -y upgrade • Execute command in a new layer and commit • defaults to using /bin/sh • RUN ["/bin/bash", "-c", "uptime"] Gavin Heavyside - ACCU 2015 - @gavinheavyside 38
  39. 39. CMD CMD ["executable","param1","param2"] • Default command to execute Gavin Heavyside - ACCU 2015 - @gavinheavyside 39
  40. 40. EXPOSE RUN apt-get install nginx EXPOSE 80 • Ports for the continer to listen on • Used for interconnecting linked containers • Doesn't automatically map to the host Gavin Heavyside - ACCU 2015 - @gavinheavyside 40
  41. 41. ENV ENV FOO=bar • Set an environment variable in the container ADD / COPY COPY /my/src /opt/container/src • Copy content to the container filesystem Gavin Heavyside - ACCU 2015 - @gavinheavyside 41
  42. 42. USER USER nginx • Set the UID for the image and any following directives WORKDIR WORKDIR /path/to/workdir • set the working dir for the image and any following directives Gavin Heavyside - ACCU 2015 - @gavinheavyside 42
  43. 43. ONBUILD ONBUILD bin/rake db:assets:precompile • Trigger instruction to run when image is used as a base for another build • Only for direct child of this image • Runs after FROM directive in child build Gavin Heavyside - ACCU 2015 - @gavinheavyside 43
  44. 44. Gavin Heavyside - ACCU 2015 - @gavinheavyside 44
  45. 45. Dockerfile Tips • Choose your base image wisely • Do the expensive work first • Take advantage of caching and layering • Use .dockerignore Gavin Heavyside - ACCU 2015 - @gavinheavyside 45
  46. 46. Gavin Heavyside - ACCU 2015 - @gavinheavyside 46
  47. 47. Pulling Images From a Registry docker pull elasticsearch docker pull private.globocorp.com/elasticsearch Gavin Heavyside - ACCU 2015 - @gavinheavyside 47
  48. 48. Tags docker pull nginx:latest docker pull nginx:1.7.11 ----- ------ | | repo tag Gavin Heavyside - ACCU 2015 - @gavinheavyside 48
  49. 49. Running your own registry • registry (Docker < 1.6) • distribution (Docker 1.6+) • dogestry Gavin Heavyside - ACCU 2015 - @gavinheavyside 49
  50. 50. Storage • Transient • Local • Persistent (portable) • Probably the hardest thing to get right at the moment Gavin Heavyside - ACCU 2015 - @gavinheavyside 50
  51. 51. Volumes Gavin Heavyside - ACCU 2015 - @gavinheavyside 51
  52. 52. VOLUME directive • Indicates the container wants to use external storagee --volumes-from • mount VOLUME paths from container A in container B Gavin Heavyside - ACCU 2015 - @gavinheavyside 52
  53. 53. Persistent Storage docker run -v /local/path:/container/ path elasticsearch • local path on filesystem is mounted in container • persists after the container exits • Portability across machines in a cluster is still a hard problem Gavin Heavyside - ACCU 2015 - @gavinheavyside 53
  54. 54. Linking Containers docker run -d -p 80:80 --name app1 app1:latest docker run --link app1:app1 app2:latest • The code running in the app2 container can now talk to app1 on port 80, using the URI http://app1:80 • Not limited to HTTP! Gavin Heavyside - ACCU 2015 - @gavinheavyside 54
  55. 55. Tailoring your app for Docker • Docker works best when containers have a single responsibility • not necessarily a single process • Some design choices can make you life easier in production Gavin Heavyside - ACCU 2015 - @gavinheavyside 55
  56. 56. The 12-Factor App • http://12factor.net • Codebase • Dependencies • Config • Backing Services • Build;Release;Run • Processes Gavin Heavyside - ACCU 2015 - @gavinheavyside 56
  57. 57. The 12-Factor App • Port Binding • Concurrency • Disposability • Dev/Prod Parity • Logs • Admin Processes Gavin Heavyside - ACCU 2015 - @gavinheavyside 57
  58. 58. 12 Factor - Dependencies • http://12factor.net/dependencies • Explicitly declare and isolate dependencies • No implicit deps "leak in" • Full and explicit dependency spec is applied in all envs, dev and prod Gavin Heavyside - ACCU 2015 - @gavinheavyside 58
  59. 59. 12 Factor - Config • http://12factor.net/config • Store config in the environment • Config is everything that can change between deploys; dev, test, and production Gavin Heavyside - ACCU 2015 - @gavinheavyside 59
  60. 60. 12 Factor - Port Binding • http://12factor.net/port-binding • App should be entirely self-contained • Expose services via port binding • Not just for HTTP • Remember health check endpoints Gavin Heavyside - ACCU 2015 - @gavinheavyside 60
  61. 61. 12 Factor - Dev/Prod Parity • http://12factor.net/dev-prod-parity • Keep development, staging, and production as similar as possible • Fewer moving parts means fewer people, skills, less time to push to production Gavin Heavyside - ACCU 2015 - @gavinheavyside 61
  62. 62. 12 Factor - Logs • http://12factor.net/logs • Treat logs as event streams • Log to stdout • Collect, rotate, and centralise logs outside the app Gavin Heavyside - ACCU 2015 - @gavinheavyside 62
  63. 63. Computation Containers • A program Q, with preconditions P, will produce output R • P and Q can change when we move between environments • Docker containers can form a complete statement of the runtime environment P, and the program to run Q Gavin Heavyside - ACCU 2015 - @gavinheavyside 63
  64. 64. Toolchain in a container $ docker run --rm -v `pwd`:/src -w /src golang:1.4 go build hello.go Gavin Heavyside - ACCU 2015 - @gavinheavyside 64
  65. 65. Toolchain in a container $ docker run --rm -v `pwd`:/src -w /src golang:1.4 go build hello.go BUT - I'm on OS X and my boot2docker host is running Linux Gavin Heavyside - ACCU 2015 - @gavinheavyside 65
  66. 66. Toolchain in a container $ docker run --rm -v `pwd`:/src -w /src golang:1.4 go build $ ./hello exec format error: hello $ file hello hello: ELF 64-bit LSB executable, ... Gavin Heavyside - ACCU 2015 - @gavinheavyside 66
  67. 67. Toolchain in a container $ docker run --rm -v src:/src -e "GOOS=darwin" -w /src golang:1.4-cross go build $ file hello hello: Mach-O 64-bit executable x86_64 $ ./hello Hello, World Gavin Heavyside - ACCU 2015 - @gavinheavyside 67
  68. 68. Choosing a base image • Enough foundation, but not too much • Security and hardening, provenance • Reuseability • Compatibility Gavin Heavyside - ACCU 2015 - @gavinheavyside 68
  69. 69. The PID 1 Reaping Problem • Unix processes are modelled like a tree • PID 1 is the top-most process • Typically this is an init process Gavin Heavyside - ACCU 2015 - @gavinheavyside 69
  70. 70. Gavin Heavyside - ACCU 2015 - @gavinheavyside 70
  71. 71. What to do? • Nothing • Specify a different init • runit • supervisord • phusion/baseimage-docker • other init process Gavin Heavyside - ACCU 2015 - @gavinheavyside 71
  72. 72. Minimalist Host OS Gavin Heavyside - ACCU 2015 - @gavinheavyside 72
  73. 73. Features of the New Minimal OSes • Small and lightweight • Specialised, not general purpose • Quick to install and boot • Smaller surface area to harden and defend • Applications deployed as containers Gavin Heavyside - ACCU 2015 - @gavinheavyside 73
  74. 74. Features of the New Minimal OSes • Read-only system files • Transactional platform updates • Backup, rollback • Delta patches • Signatures and fingerprints Gavin Heavyside - ACCU 2015 - @gavinheavyside 74
  75. 75. Examples of Minimalist OSes • Snappy Ubuntu Core • Project Atomic • CoreOS • Docker compatible, pushing own containers • RancherOS Gavin Heavyside - ACCU 2015 - @gavinheavyside 75
  76. 76. CoreOS • Etcd • Rkt • Fleet • Flannel Gavin Heavyside - ACCU 2015 - @gavinheavyside 76
  77. 77. Docker on Windows Gavin Heavyside - ACCU 2015 - @gavinheavyside 77
  78. 78. Docker Client Gavin Heavyside - ACCU 2015 - @gavinheavyside 78
  79. 79. Windows Links • http://azure.microsoft.com/blog/tag/docker/ • http://azure.microsoft.com/blog/2015/04/08/ microsoft-unveils-new-container- technologies-for-the-next-generation-cloud/ • http://azure.microsoft.com/blog/2015/04/16/ docker-client-for-windows-is-now-available/ Gavin Heavyside - ACCU 2015 - @gavinheavyside 79
  80. 80. Cluster Management Gavin Heavyside - ACCU 2015 - @gavinheavyside 80
  81. 81. Cattle, Not Pets • Not snowflakes, either • Care about the service, not the server • Easier said than done Gavin Heavyside - ACCU 2015 - @gavinheavyside 81
  82. 82. ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ Host 1 │ │ Host 2 │ │ Host 3 │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ ┌──┐┌──┐┌──┐ │ │ ┌──┐┌──┐┌──┐ │ │ ┌──┐┌──┐┌──┐ │ │ │ ││ ││ │ │ │ │ ││ ││ │ │ │ │ ││ ││ │ │ │ └──┘└──┘└──┘ │ │ └──┘└──┘└──┘ │ │ └──┘└──┘└──┘ │ └──────────────┘ └──────────────┘ └──────────────┘ Gavin Heavyside - ACCU 2015 - @gavinheavyside 82
  83. 83. ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ Host 1 │ │ Host 2 │ │ Host 3 │ │ ┌──┐ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └──┘ │ │ │ │ │ │ ┌──┐┌──┐┌──┐ │ │ │ │ │ │ │ ││ ││ │ │ │ │ │ │ │ └──┘└──┘└──┘ │ │ │ │ │ │ ┌──┐┌──┐┌──┐ │ │ ┌──┐┌──┐ │ │ │ │ │ ││ ││ │ │ │ │ ││ │ │ │ │ │ └──┘└──┘└──┘ │ │ └──┘└──┘ │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ Gavin Heavyside - ACCU 2015 - @gavinheavyside 83
  84. 84. ┌──────────────┐ ┌──────────────┐ │ Host 1 │ │ Host 2 │ │ ┌──┐ │ │ │ │ │ │ │ │ │ │ └──┘ │ │ │ │ ┌──┐┌──┐┌──┐ │ │ │ │ │ ││ ││ │ │ │ │ │ └──┘└──┘└──┘ │ │ │ │ ┌──┐┌──┐┌──┐ │ │ ┌──┐┌──┐ │ │ │ ││ ││ │ │ │ │ ││ │ │ │ └──┘└──┘└──┘ │ │ └──┘└──┘ │ └──────────────┘ └──────────────┘ Gavin Heavyside - ACCU 2015 - @gavinheavyside 84
  85. 85. Cluster Management • Kubernetes • Docker Swarm • CoreOS Fleet • AWS ECS • Google Container Service • More Gavin Heavyside - ACCU 2015 - @gavinheavyside 85
  86. 86. Kubernetes Gavin Heavyside - ACCU 2015 - @gavinheavyside 86
  87. 87. Kubernetes • Abstract at the service level, not container • Compose services from containers • Dependencies • CPU, RAM, placement • Container start order • services, load balancing Gavin Heavyside - ACCU 2015 - @gavinheavyside 87
  88. 88. Hosted Kubernetes • Google Container Engine (Alpha) • Hosted K8 on Google Cloud Platform • Tectonic (Beta) • by CoreOS Gavin Heavyside - ACCU 2015 - @gavinheavyside 88
  89. 89. AWS EC2 Container Service • Hosted Docker orchestration on EC2 (GA) • Multi-container dependencies • Placement and scheduling • one-off • service • pluggable (e.g. Mesos) Gavin Heavyside - ACCU 2015 - @gavinheavyside 89
  90. 90. Service Discovery • How do your services talk to each other? • How do they find each other in a dynamically allocated cluster? • Docker container linking only works within a host (so far) Gavin Heavyside - ACCU 2015 - @gavinheavyside 90
  91. 91. Service Discovery • Message buses (e.g. rabbitMQ) • DNS • Service Discovery Tools • Load balancing and health checking Gavin Heavyside - ACCU 2015 - @gavinheavyside 91
  92. 92. Service Discovery Tools • DNS • SmartStack (nerve, synapse) • Etcd (and SkyDNS) • Consul • More Gavin Heavyside - ACCU 2015 - @gavinheavyside 92
  93. 93. Consul • https://consul.io • K/V, DNS interfaces, ACLs • Services, health checks, load balancing • serf gossip protocol, raft consensus algorithm • distributed, highly available Gavin Heavyside - ACCU 2015 - @gavinheavyside 93
  94. 94. Registrator • https://github.com/gliderlabs/registrator • Container watches Docker engine events, dynamically registers services with backends • Etcd, Consul, SkyDNS support • Automatically publish addresses and ports of services across your infrastructure Gavin Heavyside - ACCU 2015 - @gavinheavyside 94
  95. 95. Logs • Easier if containers log to stdout, saved on the host • Can mount log dir as a volume in container if needed • Consider running e.g. logstash on the host, archiving and centralising logs • New syslog support in Docker 1.6 Gavin Heavyside - ACCU 2015 - @gavinheavyside 95
  96. 96. Monitoring • Some dedicated tools appearing, hosted and open source • Still an area with catching up to do • Traditional tools can monitor the health of apps via exposed ports and endpoints Gavin Heavyside - ACCU 2015 - @gavinheavyside 96
  97. 97. Wrapping Up Gavin Heavyside - ACCU 2015 - @gavinheavyside 97
  98. 98. Gavin Heavyside - ACCU 2015 - @gavinheavyside 98
  99. 99. Image Credits • minimalist room: https://www.flickr.com/ photos/colinsite/14089317769 • cluster: https://www.flickr.com/photos/ skiwalker79/3306092836 • wrapping: https://www.flickr.com/photos/ georigami/14253603878 • zombies: https://www.flickr.com/photos/ reana/3238910501 Gavin Heavyside - ACCU 2015 - @gavinheavyside 99
  100. 100. Image Credits • goals: https://www.flickr.com/photos/ peterfuchs/1239399915 • complexity: https://www.flickr.com/photos/ bitterjug/7670055210 • volume: http://en.wikipedia.org/wiki/ Up_to_eleven • containers: https://www.flickr.com/photos/ cseeman/11102312383 Gavin Heavyside - ACCU 2015 - @gavinheavyside 100

×