Managed Containers, Open Source, and Google - Riccardo Carlesso - Codemotion Milan 2014

1,959 views

Published on

Everything at Google (from Search to Gmail) runs in Linux Application Containers. Google is working with the Open Source community to bring over a decade of experience managing containers at scale to developers. Riccardo will talk through Kubernetes, a new Open Source container manager developed by Google with the help of the community. He will cover the core management patterns it supports, the problems it solves, and where things are going both with the project, and with the Google Cloud Platform service analogs. Kubernetes design: https://github.com/GoogleCloudPlatform/kubernetes/

Published in: Technology

Managed Containers, Open Source, and Google - Riccardo Carlesso - Codemotion Milan 2014

  1. 1. Kubernetes: Managing Containers the Google way CodeMotion Milan, 28 Nov 2014 Riccardo Carlesso Technical Solution Engineer ricc@google.com http://goo.gl/3LpbZW
  2. 2. Why Containers*? 1. Packaging 2. Efficiency and Speed 3. Security (?) (*) Container = Docker flavor container
  3. 3. Static application environment No stress deployment and update Repeatable portable artifact Develop here, run there Pick your cloud solely on its merits Loosely coupled = easier to build and manage Easier to build and manage Compose applications from micro-services Packaging
  4. 4. Everything at Google runs in a container. • Resource isolation • Predictability • Quality of service • Efficient overcommit • Resource accounting Google starts over 2 billion containers per week. Images by Connie Zhou Efficiency
  5. 5. Docker ● 15000+ GitHub stars ● 600+ contributors ● 100's of projects built on top of Docker ○ UIs, mini-PaaS, … ○ github.com/google/cadvisor ○ github.com/GoogleCloudPlatform/heapster ● Very popular GitHub project ○ Most popular Go project of all time ○ $ docker run -i -t ubuntu /bin/bash -p 80:80
  6. 6. Kubernetes κυβερνήτης: Greek for “pilot” or “helmsman of a ship” the open source cluster manager from Google
  7. 7. Kubernetes Inspired by Google's systems and experience: Manage Containers, not Machines Efficient & Robust: • optimized packing • active monitoring => self healing Organizationally Scalable: • Hassle-free maintanance => Enables micro-services Modern Open Source: • Written in Go • Hosted on github • Apache 2.0 licensed
  8. 8. Simplified Cluster Management Stack Cluster Scheduler Scheduled Containers Scheduled Containers Scheduled Containers Node Container Manager Node Container Manager Managed Base OS Managed Base OS
  9. 9. Kubernetes Container Stack Kubernetes Master etcd Scheduled Containers Scheduled Containers Scheduled Containers kubelet + Docker kubelet Managed Base OS Managed Base OS
  10. 10. Kubernetes Master/Scheduler Kubernetes Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent MINIONS
  11. 11. Web Server Log Roller Machine Host Machine Host Machine Host Machine Host Machine Host Kubernetes Master/Scheduler Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Pods
  12. 12. FE FE FE FE BE FE FE BE BE BE BE BE BE BE BE Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler Too Many Pods
  13. 13. labels: role: frontend FE FE FE FE BE FE FE BE BE BE BE BE BE BE BE Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler Labels
  14. 14. labels: role: frontend stage: production Machine Host Machine Host Machine Host BE Machine Host Machine Host BE Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler FE FE FE FE FE FE BE BE BE BE BE BE BE Labels
  15. 15. Replication Controller FE FE FE FE replicas: 4 template: ... labels: role: frontend stage: production Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler
  16. 16. Replication Controller FE replicas: 1 template: ... labels: role: frontend stage: production Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler
  17. 17. Replication Controller FE FE FE replicas: 3 template: ... labels: role: frontend stage: production Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler
  18. 18. Declarative Over Imperative Imperative: "for pod in pod{001..100} ; start $pod Declarative: "Run 100 copies of this pod with a target of <= 2 tasks down at any time" Pros: • Repeatable • "Set it and forget it" • Eventually consistent • Easily updatable Con: • Tracing action/reaction can be difficult. "I made a change, is it done?" Image: "Space Needle under construction, 1961" seattlemunicipalarchives CC-BY-2.0 http://www.flickr. com/photos/seattlemunicipalarchives/6847114249
  19. 19. id: backend-service port: 9000 labels: role: backend stage: production Pods are auto-placed: COOL! but: How to direct FE->BE traffic ? Backend Service BE BE BE BE Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Service FE port: 9000 Load-Balancing Service proxy Service proxy Service proxy Service proxy Service proxy Service proxy
  20. 20. Demo (Guestbook) Redis Master Finish Master Service Redis Slave Redis Slave Slave Service PHP Frontend PHP Frontend PHP Frontend
  21. 21. myDB Service [app=myDB] Managing LifeCycle myDB ver=1 myDB ver=1 Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler Replication Controller [app=myDB; ver=1] replicas=5 myDB ver=1 myDB ver=1 myDB ver=1
  22. 22. myDB Service [app=myDB] Managing LifeCycle myDB ver=1 myDB ver=1 Machine Host Machine Host Machine Host Machine Host myDB ver=2 Machine Host Machine Host Machine Host Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Container Agent Kubernetes - Master/Scheduler Replication Controller [app=myDB; ver=1] replicas=3 Replication Controller [app=myDB; ver=2] replicas=2 myDB ver=2 myDB ver=1
  23. 23. We’re taking lessons we’ve learned and open sourcing them Kubernetes is evolving We’re eager to hear from you! 1 2 3 Summing Up
  24. 24. We are just getting started... Clone Kubernetes at: github.com/GoogleCloudPlatform/kubernetes IRC: #google-containers on Freenode Google group: google-containers Reach out: ricc@google.com Kubernetes on GCE (new!): https://cloud.google. com/compute/docs/containers Presentation: http://goo.gl/3LpbZW
  25. 25. Backup slides Full screen QR code for feedback ⇒
  26. 26. Step 1: Create a redis master jdk$ ls README.md index.php redis-slave frontend-controller-jdk.json php-redis redis-slave-controller.json frontend-controller.json redis-master-service.json redis-slave-service.json jdk$ ../../cluster/kubecfg.sh -c redis-master.json create pods ID Image(s) Host Labels Status ---------- ---------- ---------- ---------- ---------- redis-master-2 dockerfile/redis / name=redis-master Waiting jdk-macbookpro2:guestbook jdk$ ../../cluster/kubecfg.sh list pods ID Image(s) Host Labels Status ---------- ---------- ---------- ---------- ---------- redis-master-2 dockerfile/redis kubernetes-minion-2.c.pivotal-gearing-505.internal/ name=redis-master Waiting jdk-macbookpro2:guestbook jdk$ ../../cluster/kubecfg.sh list pods ID Image(s) Host Labels Status ---------- ---------- ---------- ---------- ---------- redis-master-2 dockerfile/redis kubernetes-minion-2.c.pivotal-gearing-505.internal/ name=redis-master Running
  27. 27. Step 1: master.json { "id": "redis-master-2", "kind": "Pod", "apiVersion": "v1beta1", "desiredState": { "manifest": { "version": "v1beta1", "id": "redis-master-2", "containers": [{ "name": "master", "image": "dockerfile/redis", "ports": [{ "containerPort": 6379, "hostPort": 6379 }] }] } }, "labels": { "name": "redis-master" } }
  28. 28. Step 2: Create a redis master service jdk$ ls README.md index.php redis-slave frontend-controller-jdk.json php-redis redis-slave-controller.json frontend-controller.json redis-master-service.json redis-slave-service.json jdk$ ../../cluster/kubecfg.sh -c redis-master-service.json create services ID Labels Selector Port ---------- ---------- ---------- ---------- redismaster name=redis-master 10000 jdk$ … test redis somehow
  29. 29. Step 2: master.service { "id": "redismaster", "kind": "Service", "apiVersion": "v1beta1", "port": 10000, "containerPort": 6379, "selector": { "name": "redis-master" } }
  30. 30. Step 3: Create redis slaves & service jdk$ ../../cluster/kubecfg.sh -c redis-slave-controller.json create replicationControllers ID Image(s) Selector Replicas ---------- ---------- ---------- ---------- redisSlaveController brendanburns/redis-slave name=redisslave 2 jdk$ ../../cluster/kubecfg.sh -c redis-slave-controller.json list pods ID Image(s) Host Labels Status ---------- ---------- ---------- ---------- ---------- redis-master-2 dockerfile/redis kubernetes-minion-2.c.pivotal-gearing-505.internal/ name=redis-master Running 4df5661d-4e0b-11e4-957e-42010af01e53 brendanburns/redis-slave kubernetes-minion-2.c.pivotal-gearing-505.internal/ name=redisslave,replicationController=redisSlaveController Running 4df6be88-4e0b-11e4-957e-42010af01e53 brendanburns/redis-slave kubernetes-minion-4.c.pivotal-gearing-505.internal/ name=redisslave,replicationController=redisSlaveController Running jdk$ ../../cluster/kubecfg.sh -c redis-slave-service.json create services ID Labels Selector Port ---------- ---------- ---------- ---------- redisslave name=redisslave name=redisslave 10001 jdk$ … test redis ...
  31. 31. Step 3: redis-slave-controller.service "id": "redisSlaveController", "kind": "ReplicationController", "apiVersion": "v1beta1", "desiredState": { "replicas": 2, "replicaSelector": {"name": "redisslave"}, "podTemplate": { "desiredState": { "manifest": { "version": "v1beta1", "id": "redisSlaveController", "containers": [{ "name": "slave", "image": "brendanburns/redis-slave", "ports": [{"containerPort": 6379, "hostPort": 6380}] }] } }, "labels": {"name": "redisslave"} }}, "labels": {"name": "redisslave"} }
  32. 32. Step 4: Create frontend servers jdk$ ../../cluster/kubecfg.sh -c frontend-controller.json create replicationControllers ID Image(s) Selector Replicas ---------- ---------- ---------- ---------- frontendController brendanburns/php-redis name=frontend 3
  33. 33. Step 5: Enjoy :-)
  34. 34. Docker demo $ docker pull dockerfile/nginx ...a7767ce9ec2c: Download complete…. $ docker run -t -i dockerfile/nginx /bin/bash [ root@055dbcc687d8]$ cd /usr/share/nginx/html [ root@055dbcc687d8]$ cp index.html jdk.html $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 055dbcc687d8 dockerfile/nginx:latest "/bin/bash" 21 minutes ago Up 21 minutes 443/tcp, 80/tcp kickass_wright …. $ docker commit … jdk1/nginx-modified $ docker push jdk1/nginx-modified $ docker pull jdk1/nginx-modified docker run -p 80:80 jdk1/nginx-modified /usr/sbin/nginx

×