Blue turns green! Approaches and technologies for sustainable K8s clusters #CloudNativeNight #GSD

Green Software Development
Green Software DevelopmentGreen Software Development
qaware.de
Blue turns green!
Approaches and technologies for sustainable
Kubernetes Clusters
Mario-Leander Reimer
mario-leander.reimer@qaware.de
@LeanderReimer
https://cisweb.lancaster.ac.uk/
2
Mario-Leander Reimer
Managing Director | CTO
@LeanderReimer
#cloudnativenerd #qaware
#gernperDude
3
Mario-Leander Reimer
Managing Director | CTO
@LeanderReimer
#cloudnativenerd #qaware
#gernperDude
4
IT
Create sustainability impact
in other industries
Show responsibility for own
consumption of resources
sustainable
kubernetes
cncf/tag-env-sustainability
TAG Environmental Sustainability
🌳🌍♻
https://tag-env-sustainability.cncf.io/landscape/
Shared Responsibility means it is our responsibility to use the
required cloud resources in a sustainable manner.
8
QAware
Sustainability Engineering to optimize the CO2 footprint of
your K8s clusters and workloads.
9
QAware
Monitoring
What is the energy
consumption and
CO2 footprint of
my cluster and
workloads?
Measure,
Measure,
Measure,
Act.
Elasticity
Oversized or
underutilized
resources?
Infrastructure and
workloads should
be proportionate
to actual demand.
Architecture
Where is the
cluster running?
On which
hardware?
Take sustainability
into account when
building the
infrastructure.
Waste
Shutdown or
deplete the
wasted
resources.
Not required,
redundant or
forgotten
environments or
workloads?
10
QAware
ELECTRICITYMAPS.COM
Business requirements and sustainability goals need to be
aligned and reconciled.
The choice of region for the cluster and workloads has a significant impact on their KPIs,
including performance, cost, and carbon footprint.
■ Ensure that the region meets the required legal regulations (e.g. GDPR)
■ Ensure that the region supports the required services and features.
– Which hardware classes are supported?
– Databases? Storage? …
■ Calculate costs for the cluster and workloads for the selected region.
■ Test network latency between your end user locations and region!
11
QAware
https://docs.aws.amazon.com/wellarchitected/latest/sustainability-pillar/sus_sus_region_a2.html
AWS Graviton 2 can save up to 65%
on energy consumption
AWS Graviton 2 can save up to 15%
on costs
Cluster Rightsizing mit Cluster-Autoscaler und Karpenter
12
QAware
Node 1 Node 2 Node 3
X
Node 4
Workload Rightsizing with VPA and Goldilocks
13
QAware
https://github.com/FairwindsOps/goldilocks
apiVersion: "autoscaling.k8s.io/v1"
kind: VerticalPodAutoscaler
metadata:
name: hamster-vpa
spec:
targetRef:
apiVersion: "apps/v1"
kind: Deployment
name: hamster
updatePolicy:
# updateMode: "Auto"
updateMode: "Off"
resourcePolicy
:
containerPolicies
:
- containerName: '*'
minAllowed:
cpu: 100m
memory: 50Mi
maxAllowed:
cpu: 1
memory: 500Mi
controlledResources
: ["cpu", "memory"]
defined and tune requests and
limits for CPU and RAM
perform realistic
performance tests
analyse measurements of
resource consumption
https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler
kube-green
An operator to reduce CO2 footprint of your clusters
14
QAware
apiVersion: kube-green.com/v1alpha1
kind: SleepInfo
metadata:
name: non-working-hours
spec:
weekdays: "1-5"
sleepAt: "18:00"
wakeUpAt: "08:00"
timeZone: "Europe/Rome"
suspendCronJobs : true
excludeRef:
- apiVersion: "apps/v1"
kind: Deployment
name: no-sleep-deployment
- matchLabels :
kube-green.dev/exclude : "true" https://kube-green.dev/docs/FAQ/#how-many-co2-is-produced-by-pod
High-Level View on Scaling in Kubernetes
15
QAware
Resource Metrics, Custom Metrics, External Metrics API
16
QAware
Sustainable Computing Projects
■ Kepler (Kubernetes-based Efficient Power Level Exporter) uses eBPF to probe energy
related system stats and exports as Prometheus metrics.
■ PEAKS (Power Efficiency Aware Kubernetes Scheduler) uses metrics exported by
Kepler to help Kubernetes schedule to improve energy efficiency by placing Pods on
optimal nodes.
■ CLEVER (Container Level Energy-efficient VPA Recommender) uses metrics exported
by Kepler to recommend Vertical Pod Autoscaler the resource profiles to improve
energy efficiency by running workloads.
17
QAware
https://www.sustainable-computing.io
Monitor and Measure the Energy Consumption of Nodes and
Pods using Kepler
18
QAware
Carbon-aware Scaling mit KEDA
■ https://keda.sh
■ KEDA ist ein Kubernetes-basierter ereignisgesteuerter
und zweck-orientierter Autoscaler.
■ Unterstützt 50+ Scaler Out-of-the-box, e.g. Cron,
RabbitMQ, …
■ Carbon Aware Scheduling
- Kubernetes Carbon Intensity Exporter
- Carbon Aware KEDA Operator
19
QAware
Carbon aware Spatial Shifting with Karmada
20
QAware https://rossfairbanks.com/2023/07/12/carbon-aware-spatial-shifting-with-karmada/
karmada control plane
Cluster A
API Server
Cluster B
API Server
Agent
karmada API server
etcd
karmada
scheduler
karmada API server
Cluster
Controller
Policy
Controller
Binding
Controller
Execution
Controller
W1 W1 W1 W1
Carbon aware Spatial Shifting with Karmada
21
QAware https://rossfairbanks.com/2023/07/12/carbon-aware-spatial-shifting-with-karmada/
karmada control plane
Cluster A
API Server
Cluster B
API Server
Agent
karmada API server
etcd
karmada
scheduler
karmada API server
Cluster
Controller
Policy
Controller
Binding
Controller
Execution
Controller
W1 W1 W1 W1
lreimer/green-kubernetes
qaware.de
QAware GmbH
Aschauer Straße 32
81549 München
Tel. +49 89 232315-0
info@qaware.de
twitter.com/qaware
linkedin.com/company/qaware-gmbh
xing.com/companies/qawaregmbh
slideshare.net/qaware
github.com/qaware
1 of 23

Recommended

Blue turns green! Approaches and technologies for sustainable K8s clusters. by
Blue turns green! Approaches and technologies for sustainable K8s clusters.Blue turns green! Approaches and technologies for sustainable K8s clusters.
Blue turns green! Approaches and technologies for sustainable K8s clusters.QAware GmbH
32 views26 slides
A Hitchhiker’s Guide to the Cloud Native Stack. #CDS17 by
A Hitchhiker’s Guide to the Cloud Native Stack. #CDS17A Hitchhiker’s Guide to the Cloud Native Stack. #CDS17
A Hitchhiker’s Guide to the Cloud Native Stack. #CDS17Mario-Leander Reimer
563 views52 slides
A hitchhiker‘s guide to the cloud native stack by
A hitchhiker‘s guide to the cloud native stackA hitchhiker‘s guide to the cloud native stack
A hitchhiker‘s guide to the cloud native stackQAware GmbH
946 views52 slides
Cloud-native .NET Microservices mit Kubernetes by
Cloud-native .NET Microservices mit KubernetesCloud-native .NET Microservices mit Kubernetes
Cloud-native .NET Microservices mit KubernetesQAware GmbH
1.1K views45 slides
Kubernetes 101 and Fun by
Kubernetes 101 and FunKubernetes 101 and Fun
Kubernetes 101 and FunQAware GmbH
692 views35 slides
Kubernetes 101 and Fun by
Kubernetes 101 and FunKubernetes 101 and Fun
Kubernetes 101 and FunMario-Leander Reimer
1.6K views35 slides

More Related Content

Similar to Blue turns green! Approaches and technologies for sustainable K8s clusters #CloudNativeNight #GSD

Cloud-Native Operations with Kubernetes and CI/CD by
Cloud-Native Operations with Kubernetes and CI/CDCloud-Native Operations with Kubernetes and CI/CD
Cloud-Native Operations with Kubernetes and CI/CDVMware Tanzu
1.9K views80 slides
GitOps & the deployment branching models - DevOps D-day Marseille 2021 by
GitOps & the deployment branching models - DevOps D-day Marseille 2021GitOps & the deployment branching models - DevOps D-day Marseille 2021
GitOps & the deployment branching models - DevOps D-day Marseille 2021SoKube
369 views27 slides
Running Kubernetes with Amazon EKS - AWS Online Tech Talks by
Running Kubernetes with Amazon EKS - AWS Online Tech TalksRunning Kubernetes with Amazon EKS - AWS Online Tech Talks
Running Kubernetes with Amazon EKS - AWS Online Tech TalksAmazon Web Services
5.4K views31 slides
Expert Tips for Successful Kubernetes Deployments on AWS by
Expert Tips for Successful Kubernetes Deployments on AWSExpert Tips for Successful Kubernetes Deployments on AWS
Expert Tips for Successful Kubernetes Deployments on AWSAmazon Web Services
970 views59 slides
Deep-Dive-with-Cloud-Monitoring-with-Amazon-EKS-and-Prometheus by
Deep-Dive-with-Cloud-Monitoring-with-Amazon-EKS-and-PrometheusDeep-Dive-with-Cloud-Monitoring-with-Amazon-EKS-and-Prometheus
Deep-Dive-with-Cloud-Monitoring-with-Amazon-EKS-and-PrometheusAmazon Web Services
1.5K views43 slides
深探如何使用-Amazon-EKS-與-Prometheus-進行雲端監控 by
深探如何使用-Amazon-EKS-與-Prometheus-進行雲端監控深探如何使用-Amazon-EKS-與-Prometheus-進行雲端監控
深探如何使用-Amazon-EKS-與-Prometheus-進行雲端監控Amazon Web Services
522 views43 slides

Similar to Blue turns green! Approaches and technologies for sustainable K8s clusters #CloudNativeNight #GSD(20)

Cloud-Native Operations with Kubernetes and CI/CD by VMware Tanzu
Cloud-Native Operations with Kubernetes and CI/CDCloud-Native Operations with Kubernetes and CI/CD
Cloud-Native Operations with Kubernetes and CI/CD
VMware Tanzu1.9K views
GitOps & the deployment branching models - DevOps D-day Marseille 2021 by SoKube
GitOps & the deployment branching models - DevOps D-day Marseille 2021GitOps & the deployment branching models - DevOps D-day Marseille 2021
GitOps & the deployment branching models - DevOps D-day Marseille 2021
SoKube369 views
Running Kubernetes with Amazon EKS - AWS Online Tech Talks by Amazon Web Services
Running Kubernetes with Amazon EKS - AWS Online Tech TalksRunning Kubernetes with Amazon EKS - AWS Online Tech Talks
Running Kubernetes with Amazon EKS - AWS Online Tech Talks
Amazon Web Services5.4K views
Expert Tips for Successful Kubernetes Deployments on AWS by Amazon Web Services
Expert Tips for Successful Kubernetes Deployments on AWSExpert Tips for Successful Kubernetes Deployments on AWS
Expert Tips for Successful Kubernetes Deployments on AWS
Deep-Dive-with-Cloud-Monitoring-with-Amazon-EKS-and-Prometheus by Amazon Web Services
Deep-Dive-with-Cloud-Monitoring-with-Amazon-EKS-and-PrometheusDeep-Dive-with-Cloud-Monitoring-with-Amazon-EKS-and-Prometheus
Deep-Dive-with-Cloud-Monitoring-with-Amazon-EKS-and-Prometheus
Amazon Web Services1.5K views
深探如何使用-Amazon-EKS-與-Prometheus-進行雲端監控 by Amazon Web Services
深探如何使用-Amazon-EKS-與-Prometheus-進行雲端監控深探如何使用-Amazon-EKS-與-Prometheus-進行雲端監控
深探如何使用-Amazon-EKS-與-Prometheus-進行雲端監控
Completing the Microservices Puzzle: Kubernetes, Prometheus and FreshTracks.io by CA Technologies
Completing the Microservices Puzzle: Kubernetes, Prometheus and FreshTracks.ioCompleting the Microservices Puzzle: Kubernetes, Prometheus and FreshTracks.io
Completing the Microservices Puzzle: Kubernetes, Prometheus and FreshTracks.io
CA Technologies513 views
MongoDB World 2019: Mastering MongoDB in Kubernetes by MongoDB
MongoDB World 2019: Mastering MongoDB in KubernetesMongoDB World 2019: Mastering MongoDB in Kubernetes
MongoDB World 2019: Mastering MongoDB in Kubernetes
MongoDB408 views
Episode 2: Deploying Kubernetes at Scale by Mesosphere Inc.
Episode 2: Deploying Kubernetes at ScaleEpisode 2: Deploying Kubernetes at Scale
Episode 2: Deploying Kubernetes at Scale
Mesosphere Inc.431 views
Why we don’t use the Term DevOps: the Journey to a Product Mindset - DevOpsCo... by Henning Jacobs
Why we don’t use the Term DevOps: the Journey to a Product Mindset - DevOpsCo...Why we don’t use the Term DevOps: the Journey to a Product Mindset - DevOpsCo...
Why we don’t use the Term DevOps: the Journey to a Product Mindset - DevOpsCo...
Henning Jacobs1.4K views
Webinar: Operating Kubernetes at Scale by Mesosphere Inc.
Webinar: Operating Kubernetes at ScaleWebinar: Operating Kubernetes at Scale
Webinar: Operating Kubernetes at Scale
Mesosphere Inc.671 views
How Honestbee Does CI/CD on Kubernetes - Vincent DeSmet by DevOpsDaysJKT
How Honestbee Does CI/CD on Kubernetes - Vincent DeSmetHow Honestbee Does CI/CD on Kubernetes - Vincent DeSmet
How Honestbee Does CI/CD on Kubernetes - Vincent DeSmet
DevOpsDaysJKT365 views
Amazon Web Services User Group Sydney - March 2018 by PolarSeven Pty Ltd
Amazon Web Services User Group Sydney - March 2018Amazon Web Services User Group Sydney - March 2018
Amazon Web Services User Group Sydney - March 2018
PolarSeven Pty Ltd397 views
Kubernetes deployment strategies - CNCF Webinar by Etienne Tremel
Kubernetes deployment strategies - CNCF WebinarKubernetes deployment strategies - CNCF Webinar
Kubernetes deployment strategies - CNCF Webinar
Etienne Tremel3.1K views
Use GitLab with Chaos Engineering to Harden your Applications + OpenEBS 1.3 ... by MayaData Inc
 Use GitLab with Chaos Engineering to Harden your Applications + OpenEBS 1.3 ... Use GitLab with Chaos Engineering to Harden your Applications + OpenEBS 1.3 ...
Use GitLab with Chaos Engineering to Harden your Applications + OpenEBS 1.3 ...
MayaData Inc246 views
Kubernetes Docker Container Implementation Ppt PowerPoint Presentation Slide ... by SlideTeam
Kubernetes Docker Container Implementation Ppt PowerPoint Presentation Slide ...Kubernetes Docker Container Implementation Ppt PowerPoint Presentation Slide ...
Kubernetes Docker Container Implementation Ppt PowerPoint Presentation Slide ...
SlideTeam232 views
Episode 4: Operating Kubernetes at Scale with DC/OS by Mesosphere Inc.
Episode 4: Operating Kubernetes at Scale with DC/OSEpisode 4: Operating Kubernetes at Scale with DC/OS
Episode 4: Operating Kubernetes at Scale with DC/OS
Mesosphere Inc.407 views
20200113 - IBM Cloud Côte d'Azur - DeepDive Kubernetes by IBM France Lab
20200113 - IBM Cloud Côte d'Azur - DeepDive Kubernetes20200113 - IBM Cloud Côte d'Azur - DeepDive Kubernetes
20200113 - IBM Cloud Côte d'Azur - DeepDive Kubernetes
IBM France Lab361 views
Application Autoscaling Made Easy with Kubernetes Event-Driven Autoscaling (K... by Codit
Application Autoscaling Made Easy with Kubernetes Event-Driven Autoscaling (K...Application Autoscaling Made Easy with Kubernetes Event-Driven Autoscaling (K...
Application Autoscaling Made Easy with Kubernetes Event-Driven Autoscaling (K...
Codit1.4K views

More from Green Software Development

Green Software Big Picture by
Green Software Big PictureGreen Software Big Picture
Green Software Big PictureGreen Software Development
19 views1 slide
Green Cloud - Measure cloud emissions by
Green Cloud - Measure cloud emissionsGreen Cloud - Measure cloud emissions
Green Cloud - Measure cloud emissionsGreen Software Development
51 views18 slides
CO2 Footprint of ChatGPT by
CO2 Footprint of ChatGPTCO2 Footprint of ChatGPT
CO2 Footprint of ChatGPTGreen Software Development
112 views10 slides
Tools and best practices for sustainable software by
Tools and best practices for sustainable softwareTools and best practices for sustainable software
Tools and best practices for sustainable softwareGreen Software Development
66 views29 slides
The challenge of putting software sustainability research into practice by
The challenge of putting software sustainability research into practiceThe challenge of putting software sustainability research into practice
The challenge of putting software sustainability research into practiceGreen Software Development
46 views23 slides
Carbon Hack 2022 by
Carbon Hack 2022Carbon Hack 2022
Carbon Hack 2022Green Software Development
9 views12 slides

Recently uploaded

DSD-INT 2023 Delft3D FM Suite 2024.01 1D2D - Beta testing programme - Geertsema by
DSD-INT 2023 Delft3D FM Suite 2024.01 1D2D - Beta testing programme - GeertsemaDSD-INT 2023 Delft3D FM Suite 2024.01 1D2D - Beta testing programme - Geertsema
DSD-INT 2023 Delft3D FM Suite 2024.01 1D2D - Beta testing programme - GeertsemaDeltares
17 views13 slides
Quality Engineer: A Day in the Life by
Quality Engineer: A Day in the LifeQuality Engineer: A Day in the Life
Quality Engineer: A Day in the LifeJohn Valentino
6 views18 slides
Navigating container technology for enhanced security by Niklas Saari by
Navigating container technology for enhanced security by Niklas SaariNavigating container technology for enhanced security by Niklas Saari
Navigating container technology for enhanced security by Niklas SaariMetosin Oy
14 views34 slides
Gen Apps on Google Cloud PaLM2 and Codey APIs in Action by
Gen Apps on Google Cloud PaLM2 and Codey APIs in ActionGen Apps on Google Cloud PaLM2 and Codey APIs in Action
Gen Apps on Google Cloud PaLM2 and Codey APIs in ActionMárton Kodok
5 views55 slides
Keep by
KeepKeep
KeepGeniusee
75 views10 slides
DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t... by
DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t...DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t...
DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t...Deltares
9 views26 slides

Recently uploaded(20)

DSD-INT 2023 Delft3D FM Suite 2024.01 1D2D - Beta testing programme - Geertsema by Deltares
DSD-INT 2023 Delft3D FM Suite 2024.01 1D2D - Beta testing programme - GeertsemaDSD-INT 2023 Delft3D FM Suite 2024.01 1D2D - Beta testing programme - Geertsema
DSD-INT 2023 Delft3D FM Suite 2024.01 1D2D - Beta testing programme - Geertsema
Deltares17 views
Quality Engineer: A Day in the Life by John Valentino
Quality Engineer: A Day in the LifeQuality Engineer: A Day in the Life
Quality Engineer: A Day in the Life
John Valentino6 views
Navigating container technology for enhanced security by Niklas Saari by Metosin Oy
Navigating container technology for enhanced security by Niklas SaariNavigating container technology for enhanced security by Niklas Saari
Navigating container technology for enhanced security by Niklas Saari
Metosin Oy14 views
Gen Apps on Google Cloud PaLM2 and Codey APIs in Action by Márton Kodok
Gen Apps on Google Cloud PaLM2 and Codey APIs in ActionGen Apps on Google Cloud PaLM2 and Codey APIs in Action
Gen Apps on Google Cloud PaLM2 and Codey APIs in Action
Márton Kodok5 views
DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t... by Deltares
DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t...DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t...
DSD-INT 2023 Thermobaricity in 3D DCSM-FM - taking pressure into account in t...
Deltares9 views
BushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports by Ra'Fat Al-Msie'deen
BushraDBR: An Automatic Approach to Retrieving Duplicate Bug ReportsBushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports
BushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports
Fleet Management Software in India by Fleetable
Fleet Management Software in India Fleet Management Software in India
Fleet Management Software in India
Fleetable11 views
DSD-INT 2023 Salt intrusion Modelling of the Lauwersmeer, towards a measureme... by Deltares
DSD-INT 2023 Salt intrusion Modelling of the Lauwersmeer, towards a measureme...DSD-INT 2023 Salt intrusion Modelling of the Lauwersmeer, towards a measureme...
DSD-INT 2023 Salt intrusion Modelling of the Lauwersmeer, towards a measureme...
Deltares5 views
DSD-INT 2023 Delft3D FM Suite 2024.01 2D3D - New features + Improvements - Ge... by Deltares
DSD-INT 2023 Delft3D FM Suite 2024.01 2D3D - New features + Improvements - Ge...DSD-INT 2023 Delft3D FM Suite 2024.01 2D3D - New features + Improvements - Ge...
DSD-INT 2023 Delft3D FM Suite 2024.01 2D3D - New features + Improvements - Ge...
Deltares17 views
Airline Booking Software by SharmiMehta
Airline Booking SoftwareAirline Booking Software
Airline Booking Software
SharmiMehta6 views
DSD-INT 2023 Leveraging the results of a 3D hydrodynamic model to improve the... by Deltares
DSD-INT 2023 Leveraging the results of a 3D hydrodynamic model to improve the...DSD-INT 2023 Leveraging the results of a 3D hydrodynamic model to improve the...
DSD-INT 2023 Leveraging the results of a 3D hydrodynamic model to improve the...
Deltares6 views
AI and Ml presentation .pptx by FayazAli87
AI and Ml presentation .pptxAI and Ml presentation .pptx
AI and Ml presentation .pptx
FayazAli8711 views
Software evolution understanding: Automatic extraction of software identifier... by Ra'Fat Al-Msie'deen
Software evolution understanding: Automatic extraction of software identifier...Software evolution understanding: Automatic extraction of software identifier...
Software evolution understanding: Automatic extraction of software identifier...
360 graden fabriek by info33492
360 graden fabriek360 graden fabriek
360 graden fabriek
info3349238 views
Dapr Unleashed: Accelerating Microservice Development by Miroslav Janeski
Dapr Unleashed: Accelerating Microservice DevelopmentDapr Unleashed: Accelerating Microservice Development
Dapr Unleashed: Accelerating Microservice Development
Miroslav Janeski10 views
Team Transformation Tactics for Holistic Testing and Quality (Japan Symposium... by Lisi Hocke
Team Transformation Tactics for Holistic Testing and Quality (Japan Symposium...Team Transformation Tactics for Holistic Testing and Quality (Japan Symposium...
Team Transformation Tactics for Holistic Testing and Quality (Japan Symposium...
Lisi Hocke30 views
Dev-HRE-Ops - Addressing the _Last Mile DevOps Challenge_ in Highly Regulated... by TomHalpin9
Dev-HRE-Ops - Addressing the _Last Mile DevOps Challenge_ in Highly Regulated...Dev-HRE-Ops - Addressing the _Last Mile DevOps Challenge_ in Highly Regulated...
Dev-HRE-Ops - Addressing the _Last Mile DevOps Challenge_ in Highly Regulated...
TomHalpin96 views
2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx by animuscrm
2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx
2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx
animuscrm14 views

Blue turns green! Approaches and technologies for sustainable K8s clusters #CloudNativeNight #GSD

  • 1. qaware.de Blue turns green! Approaches and technologies for sustainable Kubernetes Clusters Mario-Leander Reimer mario-leander.reimer@qaware.de @LeanderReimer https://cisweb.lancaster.ac.uk/
  • 2. 2 Mario-Leander Reimer Managing Director | CTO @LeanderReimer #cloudnativenerd #qaware #gernperDude
  • 3. 3 Mario-Leander Reimer Managing Director | CTO @LeanderReimer #cloudnativenerd #qaware #gernperDude
  • 4. 4 IT Create sustainability impact in other industries Show responsibility for own consumption of resources
  • 8. Shared Responsibility means it is our responsibility to use the required cloud resources in a sustainable manner. 8 QAware
  • 9. Sustainability Engineering to optimize the CO2 footprint of your K8s clusters and workloads. 9 QAware Monitoring What is the energy consumption and CO2 footprint of my cluster and workloads? Measure, Measure, Measure, Act. Elasticity Oversized or underutilized resources? Infrastructure and workloads should be proportionate to actual demand. Architecture Where is the cluster running? On which hardware? Take sustainability into account when building the infrastructure. Waste Shutdown or deplete the wasted resources. Not required, redundant or forgotten environments or workloads?
  • 11. Business requirements and sustainability goals need to be aligned and reconciled. The choice of region for the cluster and workloads has a significant impact on their KPIs, including performance, cost, and carbon footprint. ■ Ensure that the region meets the required legal regulations (e.g. GDPR) ■ Ensure that the region supports the required services and features. – Which hardware classes are supported? – Databases? Storage? … ■ Calculate costs for the cluster and workloads for the selected region. ■ Test network latency between your end user locations and region! 11 QAware https://docs.aws.amazon.com/wellarchitected/latest/sustainability-pillar/sus_sus_region_a2.html AWS Graviton 2 can save up to 65% on energy consumption AWS Graviton 2 can save up to 15% on costs
  • 12. Cluster Rightsizing mit Cluster-Autoscaler und Karpenter 12 QAware Node 1 Node 2 Node 3 X Node 4
  • 13. Workload Rightsizing with VPA and Goldilocks 13 QAware https://github.com/FairwindsOps/goldilocks apiVersion: "autoscaling.k8s.io/v1" kind: VerticalPodAutoscaler metadata: name: hamster-vpa spec: targetRef: apiVersion: "apps/v1" kind: Deployment name: hamster updatePolicy: # updateMode: "Auto" updateMode: "Off" resourcePolicy : containerPolicies : - containerName: '*' minAllowed: cpu: 100m memory: 50Mi maxAllowed: cpu: 1 memory: 500Mi controlledResources : ["cpu", "memory"] defined and tune requests and limits for CPU and RAM perform realistic performance tests analyse measurements of resource consumption https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler
  • 14. kube-green An operator to reduce CO2 footprint of your clusters 14 QAware apiVersion: kube-green.com/v1alpha1 kind: SleepInfo metadata: name: non-working-hours spec: weekdays: "1-5" sleepAt: "18:00" wakeUpAt: "08:00" timeZone: "Europe/Rome" suspendCronJobs : true excludeRef: - apiVersion: "apps/v1" kind: Deployment name: no-sleep-deployment - matchLabels : kube-green.dev/exclude : "true" https://kube-green.dev/docs/FAQ/#how-many-co2-is-produced-by-pod
  • 15. High-Level View on Scaling in Kubernetes 15 QAware
  • 16. Resource Metrics, Custom Metrics, External Metrics API 16 QAware
  • 17. Sustainable Computing Projects ■ Kepler (Kubernetes-based Efficient Power Level Exporter) uses eBPF to probe energy related system stats and exports as Prometheus metrics. ■ PEAKS (Power Efficiency Aware Kubernetes Scheduler) uses metrics exported by Kepler to help Kubernetes schedule to improve energy efficiency by placing Pods on optimal nodes. ■ CLEVER (Container Level Energy-efficient VPA Recommender) uses metrics exported by Kepler to recommend Vertical Pod Autoscaler the resource profiles to improve energy efficiency by running workloads. 17 QAware https://www.sustainable-computing.io
  • 18. Monitor and Measure the Energy Consumption of Nodes and Pods using Kepler 18 QAware
  • 19. Carbon-aware Scaling mit KEDA ■ https://keda.sh ■ KEDA ist ein Kubernetes-basierter ereignisgesteuerter und zweck-orientierter Autoscaler. ■ Unterstützt 50+ Scaler Out-of-the-box, e.g. Cron, RabbitMQ, … ■ Carbon Aware Scheduling - Kubernetes Carbon Intensity Exporter - Carbon Aware KEDA Operator 19 QAware
  • 20. Carbon aware Spatial Shifting with Karmada 20 QAware https://rossfairbanks.com/2023/07/12/carbon-aware-spatial-shifting-with-karmada/ karmada control plane Cluster A API Server Cluster B API Server Agent karmada API server etcd karmada scheduler karmada API server Cluster Controller Policy Controller Binding Controller Execution Controller W1 W1 W1 W1
  • 21. Carbon aware Spatial Shifting with Karmada 21 QAware https://rossfairbanks.com/2023/07/12/carbon-aware-spatial-shifting-with-karmada/ karmada control plane Cluster A API Server Cluster B API Server Agent karmada API server etcd karmada scheduler karmada API server Cluster Controller Policy Controller Binding Controller Execution Controller W1 W1 W1 W1
  • 23. qaware.de QAware GmbH Aschauer Straße 32 81549 München Tel. +49 89 232315-0 info@qaware.de twitter.com/qaware linkedin.com/company/qaware-gmbh xing.com/companies/qawaregmbh slideshare.net/qaware github.com/qaware