Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
What's new in Kubernetes
1. Google Cloud Platform
What’s new in Kubernetes
Docker & Bay Area OpenSource meetup
February 16, 2016
Daniel Smith <dbsmith@google.com>
Senior Software Engineer
2. Google Cloud Platform
Kubernetes
Greek for “Helmsman”; also the root of the
words “governor” and “cybernetic”
• Runs and manages containers
• Inspired and informed by Google’s experiences
and internal systems
• Supports multiple cloud and bare-metal
environments
• Supports multiple container runtimes
• 100% Open source, written in Go
Manage applications, not machines
3. Google Cloud Platform
Google has been developing
and using containers to
manage applications for
over 10 years.
Images by Connie Zhou
7. Google Cloud Platform
Pods
Small group of containers & volumes
Tightly coupled
The atom of scheduling & placement
Shared namespace
• share IP address & localhost
• share IPC, etc.
Managed lifecycle
• bound to a node, restart in place
• can die, cannot be reborn with same ID
Example: data puller & web server
Consumers
Content
Manager
File
Puller
Web
Server
Volume
Pod
8. Google Cloud Platform
Volumes
Very similar to Docker’s concept
Pod scoped storage
Share the pod’s lifetime & fate
Support many types of volume plugins
• Empty dir (and tmpfs)
• Host path
• Git repository
• GCE Persistent Disk
• AWS Elastic Block Store
• Azure File Storage
• iSCSI
• Flocker
• NFS
• GlusterFS
• Ceph File and RBD
• Cinder
• FibreChannel
• Secret, ConfigMap, DownwardAPI
• Flex (exec a binary)
• ...
10. Google Cloud Platform
ReplicationControllers
A simple control loop
Runs out-of-process wrt API server
Has 1 job: ensure N copies of a pod
• if too few, start some
• if too many, kill some
• grouped by a selector
Cleanly layered on top of the core
• all access is by public APIs
Replicated pods are fungible
• No implied order or identity
ReplicationController
- name = “my-rc”
- selector = {“App”: “MyApp”}
- podTemplate = { ... }
- replicas = 4
API Server
How
many?
3
Start 1
more
OK
How
many?
4
12. Google Cloud Platform
Services
A group of pods that work together
• grouped by a selector
Defines access policy
• “load balanced” or “headless”
Gets a stable virtual IP and port
• sometimes called the service portal
• also a DNS name
VIP is managed by kube-proxy
• watches all services
• updates iptables when backends change
Hides complexity - ideal for non-native apps
Client
Virtual IP
13. Google Cloud Platform
External Services
Services IPs are only available inside the
cluster
Need to receive traffic from “the outside
world”
Builtin: Service “type”
• NodePort: expose on a port on every node
• LoadBalancer: provision a cloud load-balancer
DiY load-balancer solutions
• socat (for nodePort remapping)
• haproxy
• nginx
15. Google Cloud Platform
Ingress (L7)
Services are assumed L3/L4
Lots of apps want HTTP/HTTPS
Ingress maps incoming traffic to backend
services
• by HTTP host headers
• by HTTP URL paths
HAProxy, NGINX, AWS and GCE
implementations in progress
Now with SSL!
Status: BETA in Kubernetes v1.2
URL Map
Client
38. Google Cloud Platform
ConfigMaps
Problem: how to manage app configuration
• ...without making overly-brittle container images
12-factor says config comes from the
environment
• Kubernetes is the environment
Manage config via the Kubernetes API
Inject config as a virtual volume into your Pods
• late-binding, live-updated (atomic)
• also available as env vars
Status: GA in Kubernetes v1.2
node
API
Pod Config
Map
39. Google Cloud Platform
Secrets
Problem: how to grant a pod access to a
secured something?
• don’t put secrets in the container image!
12-factor says config comes from the
environment
• Kubernetes is the environment
Manage secrets via the Kubernetes API
Inject secrets as virtual volumes into your Pods
• late-binding, tmpfs - never touches disk
• also available as env vars
node
API
Pod Secret
41. Google Cloud Platform
Rolling Updates
ReplicationController
- replicas: 3
- selector:
- app: MyApp
- version: v1
Service
- app: MyApp
42. Google Cloud Platform
Rolling Updates
ReplicationController
- replicas: 3
- selector:
- app: MyApp
- version: v1
Service
- app: MyApp
# Update pods of frontend-v1 using new replication controller data in frontend-v2.json.
$ kubectl rolling-update frontend-v1 -f frontend-v2.json
# Update pods of frontend-v1 using JSON data passed into stdin.
$ cat frontend-v2.json | kubectl rolling-update frontend-v1 -f -
# Update the pods of frontend-v1 to frontend-v2 by just changing the image, and switching
the
# name of the replication controller.
$ kubectl rolling-update frontend-v1 frontend-v2 --image=image:v2
# Update the pods of frontend by just changing the image, and keeping the old name
$ kubectl rolling-update frontend --image=image:v2
52. Google Cloud Platform
Deployments
Rolling update is too imperative
Deployment manages RC changes for you
• stable object name
• updates are done server-side rather than client
• kubectl edit or kubectl apply is all you need
Aggregates stats
Can have multiple updates in flight
Status: BETA in Kubernetes v1.2
...
55. Google Cloud Platform
Jobs
Run-to-completion, as opposed to run-forever
• Express parallelism vs. required completions
• Workflow: restart on failure
• Build/test: don’t restart on failure
Aggregates success/failure counts
Built for batch and big-data work
Status: GA in Kubernetes v1.2
...
60. Google Cloud Platform
DaemonSets
Problem: how to run a Pod on every node
• or a subset of nodes
Similar to ReplicationController
• principle: do one thing, don’t overload
“Which nodes?” is a selector
Use familiar tools and patterns
Status: BETA in Kubernetes v1.2
Pod
62. Google Cloud Platform
Graceful Termination
Give pods time to clean up
• finish in-flight operations
• log state
• flush to disk
• 30 seconds by default
Catch SIGTERM, cleanup, exit ASAP
Pod status “Terminating”
Declarative: ‘DELETE’ manifests as an object
field in the API
64. Google Cloud Platform
HorizontalPodAutoScalers
Automatically scale ReplicationControllers to a
target utilization
• CPU utilization for now
• Probably more later
Operates within user-defined min/max bounds
Set it and forget it
Status: GA in Kubernetes v1.2
...
Stats
66. Google Cloud Platform
Cluster Scaling
Add nodes when needed
• e.g. CPU usage too high
• nodes self-register with API server
Remove nodes when not needed
• e.g. CPU usage too low
Status: Works on GCE, need other
implementations
...
67. Google Cloud Platform
New and coming soon
• Cron (scheduled jobs)
• Custom metrics
• “Apply” a config (even more declarative)
• Interactive containers
• Bandwidth shaping
• Third-party API objects
• Scalability: 1000 nodes, 100+ pods/node
• Performance
• Machine-generated Go clients (less deps!)
• Volume usage stats
• Multi-zone (AZ) support
• Multi-scheduler support
• Node affinity and anti-affinity
• Multi-cluster federation
• API federation
• More volume types
• Private Docker registry
• External DNS integration
• Volume classes and auto-provisioning
• Node fencing
• DiY Cloud Provider plugins
• More container runtimes (e.g. Hyper)
• Better auth{n,z}
• Network policy (micro-segmentation)
• Big data integrations
• Device scheduling (e.g. GPUs)
68. Google Cloud Platform
Kubernetes status & plans
Open sourced in June, 2014
• v1.0 in July, 2015
• v1.1 in November, 2015
• v1.2 ... soon!
Google Container Engine (GKE)
• hosted Kubernetes - don’t think about cluster setup
PaaSes:
• RedHat OpenShift, Deis, Stratos
Distros:
• CoreOS Tectonic, Mirantis Murano (OpenStack),RedHat
Atomic, Mesos
Hitting a ~3 month release cadence
69. Google Cloud Platform
The Goal: Read-write open source
Containers are a new way of working
Requires new concepts and new tools
Google has a lot of experience...
...but we are listening to users!
Your input does make a difference!
72. Google Cloud Platform
Kubernetes is Open
- open community
- open design
- open source
- open to ideas
http://kubernetes.io
https://github.com/kubernetes/kubernetes
slack: kubernetes
twitter: @kubernetesio
