Kubernetes: 
Managing Containers the Google way 
CodeMotion Milan, 28 Nov 2014 
Riccardo Carlesso 
Technical Solution Engi...
Why Containers*? 
1. Packaging 
2. Efficiency and Speed 
3. Security (?) 
(*) Container = Docker flavor container
Static application environment 
No stress deployment and update 
Repeatable portable artifact 
Develop here, run there 
Pi...
Everything at Google runs in a container. 
• Resource isolation 
• Predictability 
• Quality of service 
• Efficient overc...
Docker 
● 15000+ GitHub stars 
● 600+ contributors 
● 100's of projects built on top of Docker 
○ UIs, mini-PaaS, … 
○ git...
Kubernetes 
κυβερνήτης: Greek for “pilot” or “helmsman of a ship” 
the open source cluster manager from Google
Kubernetes 
Inspired by Google's systems and experience: 
Manage Containers, not Machines 
Efficient & Robust: 
• optimize...
Simplified Cluster Management Stack 
Cluster Scheduler 
Scheduled 
Containers 
Scheduled 
Containers 
Scheduled Containers...
Kubernetes Container Stack 
Kubernetes Master 
etcd 
Scheduled 
Containers 
Scheduled 
Containers 
Scheduled Containers 
k...
Kubernetes 
Master/Scheduler 
Kubernetes 
Machine 
Host 
Machine 
Host 
Machine 
Host 
Machine 
Host 
Machine 
Host 
Machi...
Web Server 
Log Roller 
Machine 
Host 
Machine 
Host 
Machine 
Host 
Machine 
Host 
Machine 
Host 
Kubernetes 
Master/Sche...
FE 
FE 
FE 
FE 
BE 
FE 
FE 
BE 
BE 
BE 
BE BE BE 
BE 
BE 
Machine 
Host 
Machine 
Host 
Machine 
Host 
Machine 
Host 
Mach...
labels: 
role: frontend 
FE 
FE 
FE 
FE 
BE 
FE 
FE 
BE 
BE 
BE 
BE BE BE 
BE 
BE 
Machine 
Host 
Machine 
Host 
Machine 
...
labels: 
role: frontend 
stage: production 
Machine 
Host 
Machine 
Host 
Machine 
Host 
BE 
Machine 
Host 
Machine 
Host ...
Replication Controller 
FE FE FE FE 
replicas: 4 
template: 
... 
labels: 
role: frontend 
stage: production 
Machine 
Hos...
Replication Controller 
FE 
replicas: 1 
template: 
... 
labels: 
role: frontend 
stage: production 
Machine 
Host 
Machin...
Replication Controller 
FE FE FE 
replicas: 3 
template: 
... 
labels: 
role: frontend 
stage: production 
Machine 
Host 
...
Declarative Over Imperative 
Imperative: 
"for pod in pod{001..100} ; start $pod 
Declarative: 
"Run 100 copies of this po...
id: backend-service 
port: 9000 
labels: 
role: backend 
stage: production 
Pods are auto-placed: COOL! 
but: How to direc...
Demo (Guestbook) 
Redis 
Master 
Finish 
Master Service 
Redis 
Slave 
Redis 
Slave 
Slave Service 
PHP 
Frontend 
PHP 
Fr...
myDB Service 
[app=myDB] 
Managing LifeCycle 
myDB 
ver=1 
myDB 
ver=1 
Machine 
Host 
Machine 
Host 
Machine 
Host 
Machi...
myDB Service 
[app=myDB] 
Managing LifeCycle 
myDB 
ver=1 
myDB 
ver=1 
Machine 
Host 
Machine 
Host 
Machine 
Host 
Machi...
We’re taking lessons we’ve learned and open sourcing them 
Kubernetes is evolving 
We’re eager to hear from you! 
1 
2 
3 ...
We are just getting started... 
Clone Kubernetes at: 
github.com/GoogleCloudPlatform/kubernetes 
IRC: #google-containers o...
Backup slides 
Full screen QR code for feedback ⇒
Step 1: Create a redis master 
jdk$ ls 
README.md index.php redis-slave 
frontend-controller-jdk.json php-redis redis-slav...
Step 1: master.json 
{ 
"id": "redis-master-2", 
"kind": "Pod", 
"apiVersion": "v1beta1", 
"desiredState": { 
"manifest": ...
Step 2: Create a redis master service 
jdk$ ls 
README.md index.php redis-slave 
frontend-controller-jdk.json php-redis re...
Step 2: master.service 
{ 
"id": "redismaster", 
"kind": "Service", 
"apiVersion": "v1beta1", 
"port": 10000, 
"containerP...
Step 3: Create redis slaves & service 
jdk$ ../../cluster/kubecfg.sh -c redis-slave-controller.json create replicationCont...
Step 3: redis-slave-controller.service 
"id": "redisSlaveController", 
"kind": "ReplicationController", 
"apiVersion": "v1...
Step 4: Create frontend servers 
jdk$ ../../cluster/kubecfg.sh -c frontend-controller.json create replicationControllers 
...
Step 5: Enjoy :-)
Docker demo 
$ docker pull dockerfile/nginx 
...a7767ce9ec2c: Download complete…. 
$ docker run -t -i dockerfile/nginx /bi...
Upcoming SlideShare
Loading in …5
×

Managed Containers, Open Source, and Google - Riccardo Carlesso - Codemotion Milan 2014

2,033 views

Published on

Everything at Google (from Search to Gmail) runs in Linux Application Containers. Google is working with the Open Source community to bring over a decade of experience managing containers at scale to developers. Riccardo will talk through Kubernetes, a new Open Source container manager developed by Google with the help of the community. He will cover the core management patterns it supports, the problems it solves, and where things are going both with the project, and with the Google Cloud Platform service analogs. Kubernetes design: https://github.com/GoogleCloudPlatform/kubernetes/

Published in: Technology

Managed Containers, Open Source, and Google - Riccardo Carlesso - Codemotion Milan 2014

  1. 1. Kubernetes: Managing Containers the Google way CodeMotion Milan, 28 Nov 2014 Riccardo Carlesso Technical Solution Engineer ricc@google.com http://goo.gl/3LpbZW
  2. 2. Why Containers*? 1. Packaging 2. Efficiency and Speed 3. Security (?) (*) Container = Docker flavor container
  3. 3. Static application environment No stress deployment and update Repeatable portable artifact Develop here, run there Pick your cloud solely on its merits Loosely coupled = easier to build and manage Easier to build and manage Compose applications from micro-services Packaging
  4. 4. Everything at Google runs in a container. • Resource isolation • Predictability • Quality of service • Efficient overcommit • Resource accounting Google starts over 2 billion containers per week. Images by Connie Zhou Efficiency
  5. 5. Docker ● 15000+ GitHub stars ● 600+ contributors ● 100's of projects built on top of Docker ○ UIs, mini-PaaS, … ○ github.com/google/cadvisor ○ github.com/GoogleCloudPlatform/heapster ● Very popular GitHub project ○ Most popular Go project of all time ○ $ docker run -i -t ubuntu /bin/bash -p 80:80
  6. 6. Kubernetes κυβερνήτης: Greek for “pilot” or “helmsman of a ship” the open source cluster manager from Google
  7. 7. Kubernetes Inspired by Google's systems and experience: Manage Containers, not Machines Efficient & Robust: • optimized packing • active monitoring => self healing Organizationally Scalable: • Hassle-free maintanance => Enables micro-services Modern Open Source: • Written in Go • Hosted on github • Apache 2.0 licensed
  8. 8. Simplified Cluster Management Stack Cluster Scheduler Scheduled Containers Scheduled Containers Scheduled Containers Node Container Manager Node Container Manager Managed Base OS Managed Base OS
  9. 9. Kubernetes Container Stack Kubernetes Master etcd Scheduled Containers Scheduled Containers Scheduled Containers kubelet + Docker kubelet Managed Base OS Managed Base OS
  10. 10. Kubernetes Master/Scheduler Kubernetes Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent MINIONS
  11. 11. Web Server Log Roller Machine Host Machine Host Machine Host Machine Host Machine Host Kubernetes Master/Scheduler Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Pods
  12. 12. FE FE FE FE BE FE FE BE BE BE BE BE BE BE BE Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler Too Many Pods
  13. 13. labels: role: frontend FE FE FE FE BE FE FE BE BE BE BE BE BE BE BE Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler Labels
  14. 14. labels: role: frontend stage: production Machine Host Machine Host Machine Host BE Machine Host Machine Host BE Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler FE FE FE FE FE FE BE BE BE BE BE BE BE Labels
  15. 15. Replication Controller FE FE FE FE replicas: 4 template: ... labels: role: frontend stage: production Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler
  16. 16. Replication Controller FE replicas: 1 template: ... labels: role: frontend stage: production Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler
  17. 17. Replication Controller FE FE FE replicas: 3 template: ... labels: role: frontend stage: production Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler
  18. 18. Declarative Over Imperative Imperative: "for pod in pod{001..100} ; start $pod Declarative: "Run 100 copies of this pod with a target of <= 2 tasks down at any time" Pros: • Repeatable • "Set it and forget it" • Eventually consistent • Easily updatable Con: • Tracing action/reaction can be difficult. "I made a change, is it done?" Image: "Space Needle under construction, 1961" seattlemunicipalarchives CC-BY-2.0 http://www.flickr. com/photos/seattlemunicipalarchives/6847114249
  19. 19. id: backend-service port: 9000 labels: role: backend stage: production Pods are auto-placed: COOL! but: How to direct FE->BE traffic ? Backend Service BE BE BE BE Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Service FE port: 9000 Load-Balancing Service proxy Service proxy Service proxy Service proxy Service proxy Service proxy
  20. 20. Demo (Guestbook) Redis Master Finish Master Service Redis Slave Redis Slave Slave Service PHP Frontend PHP Frontend PHP Frontend
  21. 21. myDB Service [app=myDB] Managing LifeCycle myDB ver=1 myDB ver=1 Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler Replication Controller [app=myDB; ver=1] replicas=5 myDB ver=1 myDB ver=1 myDB ver=1
  22. 22. myDB Service [app=myDB] Managing LifeCycle myDB ver=1 myDB ver=1 Machine Host Machine Host Machine Host Machine Host myDB ver=2 Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler Replication Controller [app=myDB; ver=1] replicas=3 Replication Controller [app=myDB; ver=2] replicas=2 myDB ver=2 myDB ver=1
  23. 23. We’re taking lessons we’ve learned and open sourcing them Kubernetes is evolving We’re eager to hear from you! 1 2 3 Summing Up
  24. 24. We are just getting started... Clone Kubernetes at: github.com/GoogleCloudPlatform/kubernetes IRC: #google-containers on Freenode Google group: google-containers Reach out: ricc@google.com Kubernetes on GCE (new!): https://cloud.google. com/compute/docs/containers Presentation: http://goo.gl/3LpbZW
  25. 25. Backup slides Full screen QR code for feedback ⇒
  26. 26. Step 1: Create a redis master jdk$ ls README.md index.php redis-slave frontend-controller-jdk.json php-redis redis-slave-controller.json frontend-controller.json redis-master-service.json redis-slave-service.json jdk$ ../../cluster/kubecfg.sh -c redis-master.json create pods ID Image(s) Host Labels Status ---------- ---------- ---------- ---------- ---------- redis-master-2 dockerfile/redis / name=redis-master Waiting jdk-macbookpro2:guestbook jdk$ ../../cluster/kubecfg.sh list pods ID Image(s) Host Labels Status ---------- ---------- ---------- ---------- ---------- redis-master-2 dockerfile/redis kubernetes-minion-2.c.pivotal-gearing-505.internal/ name=redis-master Waiting jdk-macbookpro2:guestbook jdk$ ../../cluster/kubecfg.sh list pods ID Image(s) Host Labels Status ---------- ---------- ---------- ---------- ---------- redis-master-2 dockerfile/redis kubernetes-minion-2.c.pivotal-gearing-505.internal/ name=redis-master Running
  27. 27. Step 1: master.json { "id": "redis-master-2", "kind": "Pod", "apiVersion": "v1beta1", "desiredState": { "manifest": { "version": "v1beta1", "id": "redis-master-2", "containers": [{ "name": "master", "image": "dockerfile/redis", "ports": [{ "containerPort": 6379, "hostPort": 6379 }] }] } }, "labels": { "name": "redis-master" } }
  28. 28. Step 2: Create a redis master service jdk$ ls README.md index.php redis-slave frontend-controller-jdk.json php-redis redis-slave-controller.json frontend-controller.json redis-master-service.json redis-slave-service.json jdk$ ../../cluster/kubecfg.sh -c redis-master-service.json create services ID Labels Selector Port ---------- ---------- ---------- ---------- redismaster name=redis-master 10000 jdk$ … test redis somehow
  29. 29. Step 2: master.service { "id": "redismaster", "kind": "Service", "apiVersion": "v1beta1", "port": 10000, "containerPort": 6379, "selector": { "name": "redis-master" } }
  30. 30. Step 3: Create redis slaves & service jdk$ ../../cluster/kubecfg.sh -c redis-slave-controller.json create replicationControllers ID Image(s) Selector Replicas ---------- ---------- ---------- ---------- redisSlaveController brendanburns/redis-slave name=redisslave 2 jdk$ ../../cluster/kubecfg.sh -c redis-slave-controller.json list pods ID Image(s) Host Labels Status ---------- ---------- ---------- ---------- ---------- redis-master-2 dockerfile/redis kubernetes-minion-2.c.pivotal-gearing-505.internal/ name=redis-master Running 4df5661d-4e0b-11e4-957e-42010af01e53 brendanburns/redis-slave kubernetes-minion-2.c.pivotal-gearing-505.internal/ name=redisslave,replicationController=redisSlaveController Running 4df6be88-4e0b-11e4-957e-42010af01e53 brendanburns/redis-slave kubernetes-minion-4.c.pivotal-gearing-505.internal/ name=redisslave,replicationController=redisSlaveController Running jdk$ ../../cluster/kubecfg.sh -c redis-slave-service.json create services ID Labels Selector Port ---------- ---------- ---------- ---------- redisslave name=redisslave name=redisslave 10001 jdk$ … test redis ...
  31. 31. Step 3: redis-slave-controller.service "id": "redisSlaveController", "kind": "ReplicationController", "apiVersion": "v1beta1", "desiredState": { "replicas": 2, "replicaSelector": {"name": "redisslave"}, "podTemplate": { "desiredState": { "manifest": { "version": "v1beta1", "id": "redisSlaveController", "containers": [{ "name": "slave", "image": "brendanburns/redis-slave", "ports": [{"containerPort": 6379, "hostPort": 6380}] }] } }, "labels": {"name": "redisslave"} }}, "labels": {"name": "redisslave"} }
  32. 32. Step 4: Create frontend servers jdk$ ../../cluster/kubecfg.sh -c frontend-controller.json create replicationControllers ID Image(s) Selector Replicas ---------- ---------- ---------- ---------- frontendController brendanburns/php-redis name=frontend 3
  33. 33. Step 5: Enjoy :-)
  34. 34. Docker demo $ docker pull dockerfile/nginx ...a7767ce9ec2c: Download complete…. $ docker run -t -i dockerfile/nginx /bin/bash [ root@055dbcc687d8]$ cd /usr/share/nginx/html [ root@055dbcc687d8]$ cp index.html jdk.html $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 055dbcc687d8 dockerfile/nginx:latest "/bin/bash" 21 minutes ago Up 21 minutes 443/tcp, 80/tcp kickass_wright …. $ docker commit … jdk1/nginx-modified $ docker push jdk1/nginx-modified $ docker pull jdk1/nginx-modified docker run -p 80:80 jdk1/nginx-modified /usr/sbin/nginx

×