How we manage thousands of clusters with minimal effort

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

!"#$%&%#
!"#$#%$&'(')%$*+",-'(.-$
"/$01,-*%2-$#3*+$&3(3&'1$%//"2*$$
Hardik Dodiya, Samarth Deyagond
22/06/2021
Open Source...

Meet your speakers
Hardik Dodiya
Software Developer @ Gardener on Metal
Samarth Deyagond
Software Developer @ Gardener

Motivation
OSS Project Gardener: Managed Kubernetes-as-a-Service at Scale
• primarily runs in our own infra & also possibl...

Founder
@ Botanist Quest
(Hypothetical Application)
Product Manager
@ Botanist Quest
Who are we?
Botanist Quest
Hardik
&
S...

Control plane
Nodes
Botanist
Quest
Botanist
Quest
Botanist
Quest
What do we have so far?

How we plan to host Botanist Quest

WAIT . . .
q Are we ready to scale?
q Multi cloud support?
q Homogeneity & transparency?
q Also employ our own infrastruct...

Well yeah . .

q Excessive under-utilized control-plane machines.
q Operational complexity
q Cluster management complexity
HOLD ON

Kubeception . . .
Deploy control-plane as a workload of other cluster ?
ü Under utilized dedicated Control plane machines....

Control plane of
cluster #1
in its own namespace
Control plane of
cluster #2
in its own namespace
Management
cluster

ü Excessive under-utilized control-plane machines.
q Lifecycle of the control-planes.
q Lifecycle of the underlying machin...

Cluster CRD Machine CRD
Solve it the Kubernetes way …

Machine CRD creates machines
for workload cluster
Management cluster
Nodes of the cluster
where customers host their
workl...

Let’s prototype it then…

Demo Time

ü Excessive under-utilized control plane machines
ü Lifecycle management of the control-planes.
ü Lifecycle of the underly...

Well, let me think …

ü Excessive under-utilized control plane machines
ü Lifecycle management of the control-planes.
ü Lifecycle of the underly...

Phase 1
Cluster with
dedicated
control plane

Cluster
Controller
Manager
Management
cluster
Cluster nodes
decoupled
with their
control plane
Cluster nodes
decoupled
wit...

Phase 3
Provisioning management
clusters across different
locations

Deploying control planes of management clusters
as a workload on Super Management Cluster
Management
Cluster nodes
hosting...

ü Excessive under-utilized control plane machines
ü Lifecycle management of the control-planes.
ü Lifecycle of the underly...

Can we now already manage thousands of clusters
effortlessly?

Create cluster
object
•In super
management
cluster using
cluster CRD
Assign the
management
cluster
•To host the
control pl...

Kube-
APIServer
Kube-
scheduler
Kube-
controller-
manager
Kubelet
Kubelet
Node
Node
pods
déjà vu…

Cluster-agg-
APIServer
Cluster
scheduler
Cluster
controller
manager
Clusterlet
Clusterlet
Management cluster
Management cl...

Gardener-
APIServer
Gardener
scheduler
Gardener
controller
manager
Gardenlet
Gardenlet
Seed Cluster
Seed cluster
Shoot
Con...

ü Excessive under-utilized control plane machines
ü Lifecycle management of the control-planes.
ü Lifecycle of the underly...

Batteries included but swappable…
Cloud
Providers
Cloud
Providers
Gardener
Container
Runtimes
Network
plugins
Operating
Sy...

Seed hosting
Control
Planes of shoot
Super Management
Cluster hosting
Control
Planes of Seed
Gardener
Controller
Manager
S...

Demo Time

Horizontal Vertical Pod Autoscaler
Etcd backup and restore
Automatic Seed provisioning
Scales control-plane vertically and...

Adopters

• External Contributions
• - Find the approximate number of external contributions.

Relation with Cluster API . . .
Kubeadm-
controlplane-provider
Gardener-controlplane-
provider
To be implemented …

TODOSamarthGIF:
- Scrolling plenty of clusters in the dev landscape
credits: https://twitter.com/timebertt

Join the Community
Slack GitHub YouTube Twitter
Blog News Community Adopters Documentation

v
Scan to join the Community
Samarth Deyagond
https://github.com/AxiomSamarth
Hardik Dodiya
https://github.com/hardikdr

How we manage thousands of clusters with minimal effort

Share SlideShare

Facebook
Twitter
LinkedIn

Embed

Size (px)

Show related SlideShares at end

WordPress Shortcode

Link

LinkedIn
Facebook
Twitter
Email

Replay

Top clipped slide

1 of 46 Ad

1 of 46 Ad