JDD2014: Docker.io - versioned linux containers for JVM devops - Dominik Dorn

docker.io
versioned Lightweight Linux Containers
dominik dorn @ JDD 2014

Overview
● Containers – Why?
● Containers vs. VMs
● Docker
● ( fig, jDocker, shipyard, Jenkins)
● CoreOs
● Panamax
2014-10-14 Dominik Dorn – docker.io @ JDD14

The Challenge
Static website
User DB
Redis +
redis-sentinel
Web frontend
Queue Analytics DB
Background workers
API endpoint
nginx 1.7.6 + modsecurity + openssl +
bootstrap 2
postgresql + pgv8 + v8
hadoop + hive + thrift +
OpenJDK
PlayFramework + sass
BatchEE, jRedis, libcurl + ffmpeg + libopencv +
nodejs + phantomjs
Spray / Akka, jredis + postgresql-jdbc, anorm
Development VM
QA server
Public Cloud
Disaster recovery
Contributor’s
laptop
Production Servers
Multiplicity of
Stacks
Multiplicity of
hardware
environments
Production
Cluster
Customer Data Center
Do services and
apps interact
appropriately?
smoothly and
Can I migrate
quickly?

The Matrix From Hell
Static website
Web frontend
Background
workers
User DB
Analytics DB
Queue
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
Developm
ent VM
QA
Server
Single
Prod
Server
Onsite
Cluster
Public
Cloud
Contribut
or’s
laptop
Customer
Servers

Multiplicity of
Goods
Multipilicity of
methods for
transporting/stor
ing
Do I worry about
interact (e.g.
how goods
next to spices)
cofee beans
Can I transport
quickly and
(e.g. from boat to
smoothly
train to truck)
Cargo Transport Pre-1960

Also a matrix from hell
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?

Multiplicity Multiplicity of methods for of Goods
transporting/storing
Do I worry about how
coffee beans next to
goods interact (e.g.
Solution: Intermodal Shipping Container
spices)
Can I transport quickly
(e.g. from boat to train
and smoothly
to truck)
…in between, can be
loaded and unloaded,
stacked, transported
efficiently over long
distances, and transferred
from one mode of
transport to another
A standard container
that is loaded with
virtually any goods, and
stays sealed until it
reaches final delivery.

Docker is a shipping container system for code
Static website User DB Web frontend Queue Analytics DB
Develop
ment VM
…that can be manipulated
using standard operations
and run consistently on
virtually any hardware
platform
QA server Public Cloud Contributor’
s laptop
Multiplicity of
Stacks
Multiplicity of
hardware
environments
Production
Cluster
Customer
Data Center
Do services and
apps interact
appropriately?
smoothly and
Can I migrate
quickly
An engine that enables
any payload to be
encapsulated as a
lightweight, portable,
self-sufficient
container…

Docker eliminates the matrix from Hell
Static website
Web frontend
Background
workers
User DB
Analytics DB
Queue
Develop
ment VM
QA
Server
Single
Prod
Server
Onsite
Cluster
Public
Cloud
Contribut
or’s
laptop
Custome
r Servers

Containers vs. VMs
App
A
Bins
/
Libs
Bins
/
Libs
Bins
/
Libs
Hypervisor (Type 2)
Host OS
Server
Gu
est
OS
App
A’
Gu
est
OS
App
B
Gu
est
OS
App
A
Docker
Host OS
Server
App A
Bins/Libs
Bins/Libs
App
B
App B
App
B
App B
VM
Container
Containers are isolated,
but share OS and, where
appropriate, bins/libraries
Gu
est
OS
Gu
est
OS
…result is significantly
faster deployment, much
less overhead, easier
migration, faster restart

Docker – Union File System
● aUFS – another UFS
● GIT for file systems
● Layering
● Copy on Write FS
● Can handle millions of
branches
● Docker-Limitation (June
2014)
● 127 Levels (w/o kernel mod)

Docker – installation
● On MacOS X 10.6 or newer

● On Ubuntu / Debian
wget -qO- https://get.docker.io/ | sh
# does basically this
sudo apt-key adv --keyserver
hkp://keyserver.ubuntu.com:80 --recv-keys
36A1D7869245C8950F966E92D8576A8BA88D21E9
$ sudo sh -c "echo deb https://get.docker.io/ubuntu
docker main
> /etc/apt/sources.list.d/docker.list"
$ sudo apt-get update
$ sudo apt-get install lxc-docker

● If you would like to use Docker as a non-root user, you
should now consider adding your user to the "docker"
group with something like:
sudo usermod -aG docker domdorn
● domdorn@camelion ~ % docker –version
Docker version 1.0.0, build 63fe64c
● Use the install script, ubuntu/debian have old versions
of docker in their Repos!

Docker – create a container
$ docker pull ubuntu
$ docker run -i -t ubuntu /bin/bash
// -i interactive
// -t template
*hackhack*
*hackhack*

Docker – commiting
● domdorn@camelion ~ % docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS
3cf5b7a52e4d ubuntu:14.04 /bin/bash 9 minutes ago Up 9 minutes
● domdorn@camelion ~ % docker commit 3cf5b7a52e4d domdorn/nginx
8b8401a1a847ec4ab9055e4d6a7698e5906727f8661a655abe8b614b7fdc9c61
● domdorn@camelion ~ % docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
domdorn/nginx latest 8b8401a1a847 About a minute ago 349 MB

Docker – running
● domdorn@camelion ~ % docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
domdorn/nginx latest 8b8401a1a847 About a minute ago 349 MB
● docker run -d -t domdorn/nginx /usr/sbin/nginx -g 'daemon off;'
bdf571c2d6b7a9ef08b3d1e4795b0605c6034987478e1dfd4c185ba820c73435
● docker inspect --format '{{ .NetworkSettings.IPAddress }}' bdf571c2d6b
172.17.0.9

Docker – publish ports
● docker run -p 9000:80 -d -t domdorn/nginx /usr/sbin/nginx -g 'daemon off;'
d5efd9916468c6feed90021eaf862be3fdfd36cafa20b623e51efb1cebdb41ca

Docker – using volumes
● Map folders from the host / other containers into a
container (bind-mount)
docker run -v /var/www/www.jsug.at/htdocs:/usr/share/nginx/html --name 'jddhttp' -d -t
domdorn/nginx /usr/sbin/nginx -g 'daemon off;'
29fcdbc6775d4200d14696299900fa88c1bc7da9e19a10866940eded69cd26a4
● docker inspect --format '{{ .NetworkSettings.IPAddress }}' jddhttp
172.17.0.32

Docker – importing Volumes
● docker run --volumes-from=126995e77d9b -i -t ubuntu
cat /usr/share/nginx/html/index.html
<html>
<head>
<title>hello JDD</title>
</head>
<body>
<h1>Hello JDD!</h1>
</body>
</html>

Docker – linking containers
● docker run --link=jddhttp:httpserver -i -t ubuntu ping -c 4 httpserver
PING httpserver (172.17.0.25) 56(84) bytes of data.
64 bytes from httpserver (172.17.0.25): icmp_seq=1 ttl=64 time=0.125 ms
--- httpserver ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.065/0.085/0.125/0.023 ms

Docker - Dockerfile
● Recipe to create Docker Containers
● Commands
● ADD – copies files from the file system into the docker container
● RUN – runs a command to build the image (e.g. apt-get install wget)
● CMD – runs a command when the container gets instantiated ( e.g. apt-get update )
● ENTRYPOINT – default command to run when starting a container (e.g. /usr/sbin/nginx )
● ENV – set environment variables
● EXPOSE – make a port available for incoming connections
● FROM – base docker image, e.g. domdorn/nginx or “ubuntu”
● MAINTAINER – name of the maintainer
● USER – execute the entrypoint under a specific UID
● VOLUME – defines directories that get exposed as volumes
● WORKDIR – sets the workdir for the entrypoint

Fig
● http://orchardup.github.io/fig/index.html
● Define your app's environment with Docker so it
can be reproduced anywhere
Dockerfile:
FROM orchardup/python:2.7
ADD . /code
WORKDIR /code
RUN pip install -r requirements.txt

Fig
● Define the services that make up your app so they can be run
together in an isolated environment:
fig.yml:
web:
build: .
command: python app.py
links:
- db
ports:
- "8000:8000"
db:
image: orchardup/postgresql

Fig
● Then type fig up, and Fig will start and run your
entire app:

jDocker
● https://github.com/nirima/jDocker
● Fluent-API for Docker
● Requires TCP-Socket enabled
● Create/Delete/Start/Stop Docker Containers
● Tests:
https://github.com/nirima/jDocker/blob/master/sr
c/test/java/com/kpelykh/docker/client/test/Docke
rClientTest.java

shipyard
● https://github.com/shipyard/shipyard
● Webapp to manage docker containers
● docker run -i -t -v
/var/run/docker.sock:/docker.sock
shipyard/deploy setup

Shipyard - Management

Shipyard - Hosts

Shipyard - Metrics

jenkins
● https://wiki.jenkins-ci.
org/display/JENKINS/Docker+Plugin
● Container/Slave per Build
● Uses jDocker behind the scenes

Jenkins - Config

CoreOS
● Minimal & optimized OS to run Docker
containers
● https://coreos.com/
● “Linux for Massive Server Deployments
CoreOS enables warehouse-scale computing
on top of a minimal, modern operating system.”

CoreOS

Panamax
● By the Docker founders (Cloudcentric)
● Apache 2 Licensed
● Best practice combination of Docker-Tools
( fig, fleet, supervisord, coreos, etcd, …)
● Web-UI for Docker-Configuration
● Template-System
● Panamax-Store: “AppStore” for Templates
● Github integration

Links / Literature
● http://www.slideshare.net/dotCloud/docker-intr
o-november
● http://shipyard-project.com/
● http://www.coreos.com
● http://orchardup.github.io/fig/index.html
● https://wiki.jenkins-ci.org/display/JENKINS/Doc
ker+Plugin
● http://panamax.io

That's it!
twitter: @domdorn
mail: dominik@dominikdorn.com

JDD2014: Docker.io - versioned linux containers for JVM devops - Dominik Dorn

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to JDD2014: Docker.io - versioned linux containers for JVM devops - Dominik Dorn

Similar to JDD2014: Docker.io - versioned linux containers for JVM devops - Dominik Dorn (20)

Recently uploaded

Recently uploaded (20)

JDD2014: Docker.io - versioned linux containers for JVM devops - Dominik Dorn