William Yeh
Architect @ Gogolook



2015-12-10
Albert Camus






1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
Do you have basic infrastructure? (2015-11-18)

http://www.robustperception.io/do-you-have-basic-infrastructure/
public network
1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
private network
CDN LB API servers DB servers
1.1 -
1.2 -
1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
private network
CDN LB API servers DB servers
public network
.1 -
.2 -
1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
private network
CDN LB API servers DB servers
public network
detection
recovery
diagnosis
public network
1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
private network
CDN LB API servers DB servers
1.1 -
1.2 -
hardware
bare metal
OS
runtime
app
VM
hardware
OS
runtime
app
hypervisor
VM
hardware
OS
runtime
app
hypervisor
VM
hardware
OS
runtime
app
hypervisor
Docker
hardware
OS
runtime
app
hypervisor
container
Docker
hardware
OS
runtime
app
hypervisor
container
We know:

• Container is faster than VM

• Container is lightweight

• Dockerized app anywhere…
What if…

• An OS runs only containers?
Docker
hardware
OS
runtime
app
hypervisor
container
CoreOS

RancherOS

Red Hat Atomic

VMware Photon

Snappy Ubuntu Core

Windows Nano Server
Docker
hardware
OS
runtime
app
hypervisor
container
Container per VM
hardware
OS
runtime
app
hypervisor
container
Container per VM
hardware
OS
runtime
app
hypervisor
container
Intel Clear Linux

http://www.ithome.com.tw/news/96119
Hyper

https://hyper.sh/
Container per VM
hardware
OS
runtime
app
hypervisor
container
Unikernel
hardware
library OS
app
hypervisor
unikernel

app stack
Unikernel
hardware
library OS
app
hypervisor
unikernel

app stack
Unikernels are constructed by
using “library operating
systems,” from which the
developer selects only the
minimal set of services
required for an application to
run.
Unikernel
hardware
library OS
app
hypervisor
unikernel

app stack
MirageOS

https://mirage.io/
Boxfuse

https://boxfuse.com/
ClickOS

Clive

HaLVM

LING

Rump Kernels

OSv
Unikernel image
Immutable infrastructure
public network
1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
private network
CDN ELB API servers DB servers
1.1 -
1.2 -
service consolidation
resourceisolation
native app
VM
container per VM
unikernel
Docker
public network
1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
private network
CDN LB API servers DB servers
1.1 -
1.2 -
public network
private network
API servers
JDK
app server (Tomcat, …)
app (jar, war, …)
Node.js runtime
npm
app source
Python runtime
pip
uWSGI/Gunicorn
app source
public network
private network
DB servers
JDK
Elasticsearch
JDK
CassandraMongoDBMySQL
dependenciesdependencies
public network
private network
LB
HAProxynginx
AWS ELB dependenciesdependencies
JDK
app server (Tomcat, …)
app (jar, war, …)
Node.js runtime
npm
app source
Python runtime
pip
uWSGI/Gunicorn
app source
JDK
Elasticsearch
JDK
CassandraMongoDBMySQL
dependenciesdependencies
HAProxynginx
AWS ELB dependenciesdependencies
JDK
app server (Tomcat, …)
app (jar, war, …)
Node.js runtime
npm
app source
Python runtime
pip
uWSGI/Gunicorn
app source
JDK
Elasticsearch
JDK
CassandraMongoDBMySQL
dependenciesdependencies
HAProxynginx
AWS ELB dependenciesdependencies
DevOps tools

• Configuration management

• Build system

• Deployment pipeline

• Continuous integration
JDK
app server (Tomcat, …)
app (jar, war, …)
Node.js runtime
npm
app source
Python runtime
pip
uWSGI/Gunicorn
app source
JDK
Elasticsearch
JDK
CassandraMongoDBMySQL
dependenciesdependencies
HAProxynginx
AWS ELB dependenciesdependencies
Docker
runtime
app
image
immutable image
versioned image
dev/prod parity
Dockerfile
docker build
docker push
docker pull
public network
1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
private network
CDN ELB API servers DB servers
1.1 -
1.2 -
uniform
Docker
Config management will only be used to install
Docker, an orchestration system, configure
PAM/SSH auth, and tune OS sysctl values.
traditional
DevOps
toolchain
tedious
… Basically anything not having to do with app deployment.
https://blog.containership.io/containers-vs-config-management-e64cbb744a94
public network
1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
private network
CDN LB API servers DB servers
1.1 -
1.2 -
public network
private network
API servers
As the number of machines grows…
how to ensure better allocation?
pets cattle
pets cattle
naming?
pets cattle
dispensable?
pets cattle
naming?
dispensable?
pets cattle
PaaS

Hadoop MapReduce

AWS Lambda

AWS Kinesis

Google Dataflow
public network
private network
API servers
As the number of machines grows…
how to ensure better allocation?
Traditional app
OS
runtime
app
tight
interaction
host

resources
pets
https://prezi.com/e7sdy9rdujgp
public network
private network
API servers
better mobility
cattle
public network
private network
API servers
even better mobilitycontainer
cattle
independent of
underlying
machines
cattle
Docker Swarm
Mesos
Kubernetes
allocation, orchestration
public network
1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
private network
CDN ELB API servers DB servers
1.1 -
1.2 -
Docker Swarm

Mesos

Kubernetes
Config management will only be used to install
Docker, an orchestration system…
traditional
DevOps
toolchain
… Don’t assume too much about underlying infrastructure.
cattlepets
cattle
Long Running Services
• Aurora

• Marathon

• Singularity 

• SSSP

Batch Scheduling
• Chronos

• Jenkins

• JobServer
Big Data Processing
• Cray Chapel

• Dpark

• Exelixi

• Hadoop

• Hama

• MPI

• Spark

• Storm

Data Storage
• Cassandra

• Elasticsearch

• Hypertable
Mesos framework
cattle
Mesos
1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
.1 -
.2 -
private network
CDN LB API servers DB servers
public network
private network
LB API servers DB servers
• Rolling upgrade

• Blue/green deployment

• Canary deployment
Traditional app
• in-place update
• immutable infra
public network
private network
API servers
immutable imagescontainer
cattle
independent of
underlying
machines
cattle
Docker Swarm
Mesos
Kubernetes
allocation, orchestration
kubectl rolling-update my-nginx --image=nginx:1.9.1
• Rolling upgrade
• Canary deployment

• use label
Kubernetes
Mesos
http://blog.qubit.com/opensourcing-bamboo-automated-mesos-marathon-load-balancing 

http://www.slideshare.net/johnadowns/making-developers-happier-with-mesos-docker-and-marathon
Universal Control Plane
https://www.docker.com/universal-control-plane
public network
1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
private network
CDN ELB API servers DB servers
.1 -
.2 -
Docker Swarm

Mesos

Kubernetes
traditional
DevOps
toolchain
Consider the benefits:
• immutable infrastructure
• automated allocation
• automated orchestration
cattlepets
private network
CDN LB API servers DB servers
Key Takeaways
1. How to recreate your system

2. How to safely change your system

3. When something has gone wrong
Do you have basic infrastructure? (2015-11-18)

http://www.robustperception.io/do-you-have-basic-infrastructure/
service consolidation
resourceisolation
native app
VM
container per VM
unikernel
Docker
Immutable infrastructure
uniform
Docker
Config management will only be used to install
Docker, an orchestration system… anything
other than app deployment.
traditional
DevOps
toolchain
tedious
Docker Swarm

Mesos

Kubernetes
traditional
DevOps
toolchain
… Don’t assume too much about underlying infrastructure.
cattlepets
cattle
Docker Swarm
Mesos
Kubernetes
allocation, orchestration
http://send.wtf/docker2015

Docker 對傳統 DevOps 工具鏈的衝擊 (Docker's Impact on traditional DevOps toolchain)