More Related Content
Similar to 2016 08-30 Kubernetes talk for Waterloo DevOps
2016 08-30 Kubernetes talk for Waterloo DevOps
- 1. Google Cloud Platform1 Kubernetes Container orchestration and beyond Craig Box, Google Cloud Platform @craigbox
- 2.
- 3. Google Cloud Platform3 The Anatomy of a Large-Scale Hypertextual Web Search Engine 1996, Sergey Brin and Lawrence Page Computer Science Department, Stanford University, Stanford, CA 94305
- 4.
- 5. Google Cloud Platform5 Serve web pages
- 6. Google Cloud Platform6 Build an index
- 7. Google Cloud Platform7 Babysitter
- 8. Google Cloud Platform8 Global Work Queue
- 9. Google Cloud Platform9 "Borg"
- 10. Google Cloud Platform10 A datacenter is not a collection of computers, a datacenter is a computer.
- 11. Google Cloud Platform11 Containers
- 12. Google Cloud Platform12 Containers
- 13. Google Cloud Platform13 Hypervisor Guest environment app code libraries guest kernel This is vanilla virtualization
- 14. Google Cloud Platform14 machine image locked into a platform It has downsides: Not portable, and opaque Hypervisor Guest environment app code libraries guest kernel
- 15. Google Cloud Platform15 It has downsides: No Isolation Hypervisor Guest environment app code libraries guest kernel dependency??? app code
- 16. Google Cloud Platform16 It has downsides: Little Reuse Hypervisor Guest environment app code libraries guest kernel Guest environment app code libraries guest kernel Guest environment app code libraries guest kernelredundant
- 17. Google Cloud Platform17 Containers create a better abstraction layer Hypervisor Guest environment app code libraries guest kernel cut here
- 18. Google Cloud Platform18 Node environment Much better: Portable, isolated, static app environments Hypervisor node kernel app code libraries app code libraries app code libraries container 1 container 2 container 3
- 19. Google Cloud Platform19 • Lightweight Linux environment • Hermetically sealed, deployable application • Introspectable, runnable artifact • Recently popularized by Docker Containers
- 20. Google Cloud Platform20 Resource isolation Predictability Quality of Service Batch vs. interactive Containers at Google Resource accounting Efficient overcommit
- 21. Google Cloud Platform21 Containers at Google We launch over 2 Billion containers per week. Google Cloud Platform 21
- 22. Google Cloud Platform22 Repeatable, runnable artifact Loosely coupled Static application environment Images by Connie Zhou ● Portability ● Develop here, run there ● Pick your cloud solely on its merits ● Easier to build and manage ● Compose applications from micro-services ● Mix in and extend third party services ● Reliable deployments ● No stress deployment and update Why do developers care?
- 23. Google Cloud Platform23 Highly automatable = path to active management ● Efficiency: optimized packing, better scaling ● Performance: active environment tuning ● Continuous integration: easy and reliable ● Robustness: active monitoring, self healing Why do developers care?
- 24. Google Cloud Platform24 Containers have taken the world by storm! • 2013: LXC is a niche technology • 2016: “Container” wins buzzword bingo It’s a huge paradigm shift • VMs are out, containers are in It’s happening way faster than anyone could have predicted • Adoption is up 5x in 12 months1 The right tech at the right time 1: https://www.datadoghq.com/docker-adoption/
- 26. Distributed systems toolkit Makingreliable, scalable, agile distributed systems a CS101 exercise.
- 27. Google Cloud Platform27 Greek for “Helmsman”: the root of the word “governor” and “cybernetic” ● Container orchestrator ● Builds on Docker containers ○ also supporting other container technologies ● Multiple cloud and bare-metal environments ● Supports existing OSS apps ○ cannot require apps becoming cloud-native ● Inspired and informed by Google’s ● experiences and internal systems ● 100% Open source, written in Go Let users manage applications, not machines κυβερνήτης
- 28. Google Cloud Platform28 machine-1 machine-2 machine-3 “Physical” Infrastructure frontend middleware backend
- 29. Google Cloud Platform29 frontend middleware backend Kubernetes API: Unified Compute Substrate Logical Infrastructure
- 30. Google Cloud Platform30 Cluster Federation
- 31. Google Cloud Platform31 kubelet UI kubeletCLI API users master nodes Single Kubernetes Cluster etcd kubelet scheduler controllers apiserver
- 32. Google Cloud Platform32 UI All you really care about API Container Cluster
- 33. Google Cloud Platform33 Pods Small group of containers & volumes Tightly coupled same node The atom of cluster scheduling & placement Each pod has its own IP address shared namespace: share IP address & localhost Ephemeral can die and be replaced Example: data puller & web server Pod File Puller Web Server Volume Consumers Content Manager
- 34. Google Cloud Platform34 Volumes Pod-scoped storage Support many types of volume plugins Empty dir (and tmpfs) Host path Git repository GCE Persistent Disk AWS Elastic Block Store Azure File Storage iSCSI Flocker NFS vSphere GlusterFS Ceph File and RBD Cinder FibreChannel Secret, ConfigMap, DownwardAPI Flex (exec a binary) ...
- 35. Google Cloud Platform35 ReplicaSets* A simple control loop Runs out-of-process wrt API server One job: ensure N copies of a pod grouped by a selector too few? start some too many? kill some Layered on top of the public Pod API Replicated pods are fungible No implied order or identity * The evolution of ReplicationControllers ReplicaSet - name = “my-rc” - selector = {“App”: “MyApp”} - template = { ... } - replicas = 4 API Server How many? 3 Start 1 more OK How many? 4
- 36. Google Cloud Platform36 Services A group of pods that act as one == Service group == selector Defines access policy only “load balanced” for now Gets a stable virtual IP and port called the service portal also a DNS name VIP is captured by kube-proxy watches the service constituency updates when backends change Hide complexity - ideal for non-native apps Portal (VIP) Client
- 37. Google Cloud Platform37 Customer pain points High Availability mitigate zone/provider outages Geographic scaling low latency local serving Application Migration Vendor lock-in Automating migrations & rollbacks Capacity Overflow Utilization / Cost Performance Policy Enforcement Data must be stored and processed within specified political jurisdictions Cross-cluster Load Balancer Your paying customer Cluster 1 Cluster 2 Cluster 3
- 38. Google Cloud Platform38 UI CLI API Control Plane Clusters Federation API Users Kubernetes on Kubernetes on Kubernetes on Premise Federation
- 39. Google Cloud Platform39 Create the Kubernetes Clusters Cluster 1 us-east1-b Cluster 2 us-central1-b Cluster 3 europe-west1-b Cluster 4 asia-east1-b API API API API API CLI UI
- 40. Google Cloud Platform40 Deploy the Federated Control Plane on an existing Kubernetes cluster ● Federation: ○ Namespace ○ API Server Service with public VIP ○ API Server Deployment Pod with 2 replicas ○ Controller Manager Pod with 1 replica ○ Database key/value store Cluster 1 us-east1-b Cluster 2 us-central1-b Cluster 3 europe-west1-b Cluster 4 asia-east1-b --context=federation-cluster
- 41. Google Cloud Platform41 Add Cluster(s) to Federation Cluster 1 us-east1-b Cluster 2 us-central1-b Cluster 3 europe-west1-b Cluster 4 asia-east1-b API API API API Federation Control Plane kubectl --context=federation-cluster create -f clusters/gce-asia-east1.yaml apiVersion: federation/v1beta1 kind: Cluster metadata: name: gce-asia-east1 spec: serverAddressByClientCIDRs: - clientCIDR: "0.0.0.0/0" serverAddress: "https://xxx.xxx.194.68" secretRef: name: gce-asia-east1CLI
- 42. Google Cloud Platform42 Deploying a Federated Service Cluster 1 us-east1-b Cluster 2 us-central1-b Cluster 3 europe-west1-b Cluster 4 asia-east1-b API API API API Federation Control Plane API CLI kubectl --context=federation-cluster create -f services/nginx.yaml NGINX NGINX NGINX NGINX
- 43. Google Cloud Platform43 Each Service Shard is exposed via External Load Balancer Cluster 1 us-east1-b Cluster 2 us-central1-b Cluster 3 europe-west1-b Cluster 4 asia-east1-b Federation Control Plane API CLI NGINX NGINX NGINX NGINX
- 44. Google Cloud Platform44 Deploy Federated Service Backends Cluster 1 us-east1-b Cluster 2 us-central1-b Cluster 3 europe-west1-b Cluster 4 asia-east1-b DNS NGINX NGINX NGINX NGINX DNS DNS DNS
- 45. Google Cloud Platform45 Cross Cluster Service Discovery (External Request) Cluster 1 us-east1-b Cluster 2 us-central1-b Cluster 3 europe-west1-b Cluster 4 asia-east1-b DNS NGINX NGINX NGINX NGINX DNS DNS DNS External DNS nginx.default.federation.svc.federation.com C3 C4 C3 C4 1 2 3 4
- 46. Google Cloud Platform46 Cross Cluster Service Discovery (External Request) Cluster 1 us-east1-b Cluster 2 us-central1-b Cluster 3 europe-west1-b Cluster 4 asia-east1-b DNS NGINX NGINX NGINX NGINX DNS DNS DNS C3 C4 X External DNS
- 47. Google Cloud Platform47 Q2 2016 Q3 2016 Q4 2016 (**) 2017 and beyond (**) Federation Control plane (beta1) - Public facing Multi Region/Could - Cross Cluster Service Discovery (internal/external DNS) - Service object API support Federation Control plane (beta2) - Multi Region Ingress(L7) Load Balancing across clusters for GCP only Federation Control plane (beta3) - Cross provider Multi Region Ingress(L7) Load Balancing - GKE IAM Integration Federation Control plane (GA) - Non Public Facing Cross Cluster Service Discovery - Full support for Kubernetes API objects - UI support for Federated Clusters - Federated IAM - GKE Hosted Control Plane Updated 2016-08-03 (Dataproc HA schedule update), douglasdollars@ (**) - this is a proposed roadmap. Items listed here are subject to change. Cluster Federation roadmap
- 48. Google Cloud Platform48 ● Kubernetes Cluster Federation Sneak Peak (link) ● Kubernetes Cluster Federation using GKE (link) ● Cluster Federation Admin Guide (link) ● Cross Cluster Service Discovery Deployment Guide (link) ● Cross Cluster Services - Achieving Higher Availability for your Kubernetes Applications (link) Also, ● Participate with us on the Kubernetes #sig-federation ● Post issues or feature requests on GitHub ● Join us in the #federation channel on Slack Want to learn more?
- 49. Google Cloud Platform49 Stateful services
- 50. Google Cloud Platform50 Stateful services (1.2) - ReplicaSets ensure N replicas of a pod template exist - Only attribute that differs between pods is the name - Pods have no stable network identity other than an assigned pod IP - e.g. no fixed and predictable hostname - Best for ‘shared-nothing, zero-coordination systems’ Photo credit: http://delectabledeliciousness.blogspot.co.uk
- 51. Google Cloud Platform51 Persistent Volume Framework Administrator Developer Developer owned PV PV PV Pool of Persistent Volumes Registers PVs in the pool claim claim Claims a PV from the pool References claim in pod Pod PV Administrator owned
- 52. Google Cloud Platform52 Init Containers ● Alpha feature ● Run container(s) before the long-running main pod container(s) are launched ● Executed in sequential order ● Only if init containers succeed do the main container(s) start ● Use cases: ○ download files / clone repos ○ write out config files ○ etc init-1 init-2 main-container
- 53. 53 ● New Alpharesource (apps/v1alpha1) ○ Not available on GKE ● Stable and unique identity associated with that instance of the storage ● Consistent network identity ● Predictable number of instances to ensure that systems can form a quorum ● Ability to migrate from node to node with stable network identity (DNS name) ● Scale up/down in a controlled fashion PetSet
- 54. Google Cloud Platform54 PetSet PetSet pet-0. pet.default... PetSet Controller PV-0 PVC-0 API Server Watches Creates claim from template BindsMounts Creates and waits until ready Service 1:1 mapping
- 55. Google Cloud Platform55 PetSet PetSet pet-0. pet.default... PetSet Controller PV-0 PVC-0 API Server Service pet-1. pet.default... PV-1 PVC-1 pet-2. pet.default... PV-2 PVC-2
- 56. Google Cloud Platform56 In closing
- 57. Google Cloud Platform57 Top 0.01% of all Github projects 1200+ external projects based on Kubernetes 800+ unique contributors Community 30,000+ commits Broad industry support
- 58. Google Cloud Platform58 Kubernetes Adoption Startup to Enterprise Adoption ● 2,000+ external projects based on Kubernetes ● Usage spans all platforms, from on-prem to all public clouds ● Google Container Engine (GKE) usage is doubling every 90 days
- 59. Google Cloud Platform59 Kubernetes & Container Engine Containers are powering a revolution For Developers: Easy dev-test environments Easy to compose micro-services Update your cluster with one command For Ops/IT: Kubernetes is managed for you Deploying becomes file-copy Highly portable Google Cloud Platform 59
- 60. Google Cloud Platform60 Thank you #kubernetes @craigbox Thanks to Matt Bates from jetstack.io and Steve Watt from Red Hat