Speed Up Your Kubernetes Upgrades for Your Kafka Clusters With Vane...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Vanessa Vuibert
Sta
ff
Production Engineer
Speed Up Your Kubernetes Upgrades For
Your Ka
f
ka Clusters

• Bug
f
ixes
• New features
• Mandatory at some point
Kubernetes (k8s) upgrades
are important!

710 14
Ka
f
ka at Shopify
Brokers Ka
f
ka clusters
2 1PB
k8s upgrades per year Average storage usage

• Upgrade control plane
• Upgrade node pools
• Drain node pools
In-place k8s upgrades

• Cannot rollback
control plane
upgrade
• Leftover resources
• Cannot skip
versions

Create a new k8s cluster
and migrate your workloads.

Stateless app migration

Stateless app migration

Stateless app migration

Stateless app migration

Migrating a stateful
app is more complex.

Stateful app migration

Stateful app migration

Stateful app migration

Stateful app migration

Stretch the Ka
f
ka cluster between
old and new k8s clusters.

1. Copy the data over
the network
2. Reuse the persistent
disks

• Deploy StatefulSet in new k8s cluster with
di
ff
erent broker IDs
• Use LinkedIn Cruise Control to remove brokers
• Remo...

Option 1: Copy the data over the network

Option 1: Copy the data over the network

Option 1: Copy the data over the network

Option 1: Copy the data over the network

60% 20%
Top expense items
GCP bill in storage GCP bill in network
1PB 3GB/s
Average storage usage Average incoming tra
ff
...

• Create persistent volumes (PVs)
• Create persistent volume claims (PVCs)
• Deploy StatefulSet in new k8s cluster
• Gradu...

Option 2: Reuse the disks

Option 2: Reuse the disks

Option 2: Reuse the disks

Option 2: Reuse the disks

Option 2: Reuse the disks

Option 2: Reuse the disks

Option 2: Reuse the disks

Option 2: Reuse the disks

Ka
f
ka k8s migrations
used to take
6
months copying data
over the network
now only takes
2
weeks reusing disks
Time savin...

How to reuse the
persistent disks.

• New k8s cluster creation
• Create PVs and PVCs
• Rollback strategy
Preparation

Create new k8s cluster

Ka
f
ka StatefulSet: cross-cluster

Ka
f
ka StatefulSet: persistent storage

Ka
f
ka StatefulSet: PV and PVC

Create PVs

Create PVs and PVCs

Source preparation: rollback strategy

• Migrate half of Zookeeper servers
• Migrate Ka
f
ka
• Finish Zookeeper migration
Zookeeper and Ka
f
ka migration

Zookeeper StatefulSet: single-cluster

Zookeeper StatefulSet: cross-cluster

Zookeeper and Ka
f
ka migration

Zookeeper and Ka
f
ka migration

Zookeeper and Ka
f
ka migration

Zookeeper and Ka
f
ka migration

Zookeeper and Ka
f
ka migration

• Recreate k8s
clusters instead of
in-place upgrades
• Reuse disks

Thanks!

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

2024 Trend Updates: What Really Wor... by Search Engine Jou... 1054718 views
Storytelling For The Web: Integrate... by Chiara Aliotta 970719 views
Artificial Intelligence, Data and C... by OECD Directorate ... 905165 views
How to Leverage AI to Boost Employe... by SocialHRCamp 382887 views
2024 State of Marketing Report – by... by Marius Sescu 216911 views
Everything You Need To Know About C... by Expeed Software 234552 views

View on Slideshare

1 of 55 Ad

View on Slideshare

1 of 55 Ad