SlideShare a Scribd company logo
Flagger
Istio Progressive Delivery Operator
Stefan Prodan @stefanprodan
Istio London November 2018
1
What is Progressive Delivery?
Progressive Delivery is Continuous Delivery with fine-grained control over the
blast radius.
Building blocks
● User segmentation
● Traffic management
● Observability
● Automation
https://redmonk.com/jgovernor/2018/08/06/towards-progressive-delivery/
Introducing Flagger
Flagger is a Kubernetes operator that automates the promotion of
canary deployments using Istio routing for traffic shifting and
Prometheus metrics for canary analysis.
Flagger implements a control loop that gradually shifts traffic to the
canary while measuring key performance indicators. Based on the
KPIs analysis a canary is promoted or aborted.
Get Flagger
helm repo add flagger https://flagger.app
helm upgrade -i flagger flagger/flagger 
--namespace=istio-system 
--set metricsServer=http://prometheus.istio-system:9090 
--set controlLoopInterval=1m
helm upgrade -i flagger-grafana flagger/grafana 
--namespace=istio-system 
--set url=http://prometheus.istio-system:9090
Flagger overview
Key Performance Indicators
The decision to pause the traffic shift, abort or promote a canary is
based on:
● Deployment health status
● Request success rate percentage
● Request latency average value
Observability
Flagger emits Kubernetes events related to the advancement and final status of a canary analysis.
Observability RED + USE methods
Flagger comes with a Grafana dashboard for canary analysis.
Canary CRD
9
apiVersion: flagger.app/v1alpha1
kind: Canary
metadata:
name: podinfo
namespace: test
spec:
# deployment reference
targetRef:
apiVersion: apps/v1
kind: Deployment
name: podinfo
# hpa reference (optional)
autoscalerRef:
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
name: podinfo
service:
# container port
port: 9898
# Istio gateways (optional)
gateways:
- public-gateway.istio-system.svc.cluster.local
# Istio virtual service host names (optional)
hosts:
- app.istio.weavedx.com
Canary CRD - analysis spec
10
canaryAnalysis:
# maximum number of failed metric checks
# before rolling back the canary
threshold: 10
# max traffic percentage routed to canary
maxWeight: 50
# canary increment step
stepWeight: 5
metrics:
- name: istio_requests_total
# minimum req success rate percentage (non 5xx responses)
threshold: 99
interval: 1m
- name: istio_request_duration_seconds_bucket
# maximum req duration P99 (milliseconds)
threshold: 500
interval: 30s
Canary deployment stages
Canary automation for production systems
Flagger control loop stage 1-5
● scan for canary deployments
● create the primary deployment and HPA
● create the ClusterIP services for primary and canary
● create an Istio virtual service with weighted destinations mapped to primary and
canary ClusterIP services
● check primary and canary deployments status
○ halt advancement if a rolling update is underway
○ halt advancement if pods are unhealthy
● increase canary traffic weight percentage from 0% to 5% (step weight)
● check canary HTTP request success rate and latency
○ halt advancement if any metric is under the specified threshold
○ increment the failed checks counter
Flagger control loop stage 6-7
● check if the number of failed checks reached the threshold
○ route all traffic to primary
○ scale to zero the canary deployment
○ mark canary as failed
○ wait for the canary to be updated (revision bump) and start over
● increase canary traffic by 5% (step weight)
○ halt advancement while canary request success rate is under the threshold
○ halt advancement while canary request duration P99 is over the threshold
○ halt advancement if the primary or canary deployment becomes unhealthy
○ halt advancement while canary deployment is being scaled up/down by HPA
Flagger control loop stage 8-12
● promote canary to primary when canary weight reaches max
○ copy canary deployment spec template over primary
● wait for primary rolling update to finish
○ halt advancement if pods are unhealthy
● route all traffic to primary
● scale to zero the canary deployment
● mark canary as finished
● wait for the canary to be updated (revision bump) and start over
GitOps Progressive Delivery Demo
16
Links
17
Flagger
https://github.com/stefanprodan/flagger
GitOps demo repo
https://github.com/stefanprodan/gitops-progressive-delivery
Istio walkthrough
https://github.com/stefanprodan/istio-gke

More Related Content

What's hot

Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Amazon Web Services
 
Terraform Basics
Terraform BasicsTerraform Basics
Terraform Basics
Mohammed Fazuluddin
 
Room 1 - 4 - Phạm Tường Chiến & Trần Văn Thắng - Deliver managed Kubernetes C...
Room 1 - 4 - Phạm Tường Chiến & Trần Văn Thắng - Deliver managed Kubernetes C...Room 1 - 4 - Phạm Tường Chiến & Trần Văn Thắng - Deliver managed Kubernetes C...
Room 1 - 4 - Phạm Tường Chiến & Trần Văn Thắng - Deliver managed Kubernetes C...
Vietnam Open Infrastructure User Group
 
Kubernetes
KubernetesKubernetes
Kubernetes
erialc_w
 
Kubernetes Basics
Kubernetes BasicsKubernetes Basics
Kubernetes Basics
Eueung Mulyana
 
Room 1 - 3 - Lê Anh Tuấn - Build a High Performance Identification at GHTK wi...
Room 1 - 3 - Lê Anh Tuấn - Build a High Performance Identification at GHTK wi...Room 1 - 3 - Lê Anh Tuấn - Build a High Performance Identification at GHTK wi...
Room 1 - 3 - Lê Anh Tuấn - Build a High Performance Identification at GHTK wi...
Vietnam Open Infrastructure User Group
 
Jenkins
JenkinsJenkins
Jenkins
Roger Xia
 
Apache Pulsar Development 101 with Python
Apache Pulsar Development 101 with PythonApache Pulsar Development 101 with Python
Apache Pulsar Development 101 with Python
Timothy Spann
 
Service Mesh on Kubernetes with Istio
Service Mesh on Kubernetes with IstioService Mesh on Kubernetes with Istio
Service Mesh on Kubernetes with Istio
Michelle Holley
 
Introduction to helm
Introduction to helmIntroduction to helm
Introduction to helm
Jeeva Chelladhurai
 
Kubernetes a comprehensive overview
Kubernetes   a comprehensive overviewKubernetes   a comprehensive overview
Kubernetes a comprehensive overview
Gabriel Carro
 
Secret Management with Hashicorp’s Vault
Secret Management with Hashicorp’s VaultSecret Management with Hashicorp’s Vault
Secret Management with Hashicorp’s Vault
AWS Germany
 
Bridging the Security Testing Gap in Your CI/CD Pipeline
Bridging the Security Testing Gap in Your CI/CD PipelineBridging the Security Testing Gap in Your CI/CD Pipeline
Bridging the Security Testing Gap in Your CI/CD Pipeline
DevOps.com
 
Room 2 - 3 - Nguyễn Hoài Nam & Nguyễn Việt Hùng - Terraform & Pulumi Comparin...
Room 2 - 3 - Nguyễn Hoài Nam & Nguyễn Việt Hùng - Terraform & Pulumi Comparin...Room 2 - 3 - Nguyễn Hoài Nam & Nguyễn Việt Hùng - Terraform & Pulumi Comparin...
Room 2 - 3 - Nguyễn Hoài Nam & Nguyễn Việt Hùng - Terraform & Pulumi Comparin...
Vietnam Open Infrastructure User Group
 
GitOps with ArgoCD
GitOps with ArgoCDGitOps with ArgoCD
GitOps with ArgoCD
CloudOps2005
 
Docker Kubernetes Istio
Docker Kubernetes IstioDocker Kubernetes Istio
Docker Kubernetes Istio
Araf Karsh Hamid
 
Terraform
TerraformTerraform
Terraform
Otto Jongerius
 
ELK Stack
ELK StackELK Stack
ELK Stack
Eberhard Wolff
 
Apache Kafka Best Practices
Apache Kafka Best PracticesApache Kafka Best Practices
Apache Kafka Best Practices
DataWorks Summit/Hadoop Summit
 
MuleSoft Online Meetup a Guide to RTF application deployment - October 2020
MuleSoft Online Meetup   a Guide to RTF application deployment  - October 2020MuleSoft Online Meetup   a Guide to RTF application deployment  - October 2020
MuleSoft Online Meetup a Guide to RTF application deployment - October 2020
Royston Lobo
 

What's hot (20)

Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
 
Terraform Basics
Terraform BasicsTerraform Basics
Terraform Basics
 
Room 1 - 4 - Phạm Tường Chiến & Trần Văn Thắng - Deliver managed Kubernetes C...
Room 1 - 4 - Phạm Tường Chiến & Trần Văn Thắng - Deliver managed Kubernetes C...Room 1 - 4 - Phạm Tường Chiến & Trần Văn Thắng - Deliver managed Kubernetes C...
Room 1 - 4 - Phạm Tường Chiến & Trần Văn Thắng - Deliver managed Kubernetes C...
 
Kubernetes
KubernetesKubernetes
Kubernetes
 
Kubernetes Basics
Kubernetes BasicsKubernetes Basics
Kubernetes Basics
 
Room 1 - 3 - Lê Anh Tuấn - Build a High Performance Identification at GHTK wi...
Room 1 - 3 - Lê Anh Tuấn - Build a High Performance Identification at GHTK wi...Room 1 - 3 - Lê Anh Tuấn - Build a High Performance Identification at GHTK wi...
Room 1 - 3 - Lê Anh Tuấn - Build a High Performance Identification at GHTK wi...
 
Jenkins
JenkinsJenkins
Jenkins
 
Apache Pulsar Development 101 with Python
Apache Pulsar Development 101 with PythonApache Pulsar Development 101 with Python
Apache Pulsar Development 101 with Python
 
Service Mesh on Kubernetes with Istio
Service Mesh on Kubernetes with IstioService Mesh on Kubernetes with Istio
Service Mesh on Kubernetes with Istio
 
Introduction to helm
Introduction to helmIntroduction to helm
Introduction to helm
 
Kubernetes a comprehensive overview
Kubernetes   a comprehensive overviewKubernetes   a comprehensive overview
Kubernetes a comprehensive overview
 
Secret Management with Hashicorp’s Vault
Secret Management with Hashicorp’s VaultSecret Management with Hashicorp’s Vault
Secret Management with Hashicorp’s Vault
 
Bridging the Security Testing Gap in Your CI/CD Pipeline
Bridging the Security Testing Gap in Your CI/CD PipelineBridging the Security Testing Gap in Your CI/CD Pipeline
Bridging the Security Testing Gap in Your CI/CD Pipeline
 
Room 2 - 3 - Nguyễn Hoài Nam & Nguyễn Việt Hùng - Terraform & Pulumi Comparin...
Room 2 - 3 - Nguyễn Hoài Nam & Nguyễn Việt Hùng - Terraform & Pulumi Comparin...Room 2 - 3 - Nguyễn Hoài Nam & Nguyễn Việt Hùng - Terraform & Pulumi Comparin...
Room 2 - 3 - Nguyễn Hoài Nam & Nguyễn Việt Hùng - Terraform & Pulumi Comparin...
 
GitOps with ArgoCD
GitOps with ArgoCDGitOps with ArgoCD
GitOps with ArgoCD
 
Docker Kubernetes Istio
Docker Kubernetes IstioDocker Kubernetes Istio
Docker Kubernetes Istio
 
Terraform
TerraformTerraform
Terraform
 
ELK Stack
ELK StackELK Stack
ELK Stack
 
Apache Kafka Best Practices
Apache Kafka Best PracticesApache Kafka Best Practices
Apache Kafka Best Practices
 
MuleSoft Online Meetup a Guide to RTF application deployment - October 2020
MuleSoft Online Meetup   a Guide to RTF application deployment  - October 2020MuleSoft Online Meetup   a Guide to RTF application deployment  - October 2020
MuleSoft Online Meetup a Guide to RTF application deployment - October 2020
 

Similar to Flagger: Istio Progressive Delivery Operator

Weavework Flagger Demo- AWS Container Day 2019 Barcelona
Weavework Flagger Demo- AWS Container Day 2019 BarcelonaWeavework Flagger Demo- AWS Container Day 2019 Barcelona
Weavework Flagger Demo- AWS Container Day 2019 Barcelona
Amazon Web Services
 
AutoSat Fleet Management System Overview
AutoSat Fleet Management System OverviewAutoSat Fleet Management System Overview
AutoSat Fleet Management System Overview
Seva Vanchikov
 
Grafana overview deck - Tech - 2023 May v1.pdf
Grafana overview deck  - Tech - 2023 May v1.pdfGrafana overview deck  - Tech - 2023 May v1.pdf
Grafana overview deck - Tech - 2023 May v1.pdf
BillySin5
 
Change Control Management Supports Continuous Integration and Delivery
Change Control Management Supports Continuous Integration and DeliveryChange Control Management Supports Continuous Integration and Delivery
Change Control Management Supports Continuous Integration and Delivery
SAP Cloud Platform
 
Trunk based development and Canary deployment
Trunk based development and Canary deploymentTrunk based development and Canary deployment
Trunk based development and Canary deployment
Hai Lu
 
Reliability Pipelines With Keptn Quality Gates
Reliability Pipelines With Keptn Quality GatesReliability Pipelines With Keptn Quality Gates
Reliability Pipelines With Keptn Quality Gates
Paul Balogh
 
Disaster Recovery for Multi-Region Apache Kafka Ecosystems at Uber
Disaster Recovery for Multi-Region Apache Kafka Ecosystems at UberDisaster Recovery for Multi-Region Apache Kafka Ecosystems at Uber
Disaster Recovery for Multi-Region Apache Kafka Ecosystems at Uber
confluent
 
PAC 2019 virtual Bruno Audoux
PAC 2019 virtual Bruno Audoux PAC 2019 virtual Bruno Audoux
PAC 2019 virtual Bruno Audoux
Neotys
 
Kong API Gateway.pdf
Kong API Gateway.pdfKong API Gateway.pdf
Kong API Gateway.pdf
AvinashUpadhyaya3
 
Solutions Dept. Presentation
Solutions Dept. PresentationSolutions Dept. Presentation
Solutions Dept. Presentation
Kira Yakunin
 
Programmable Testing for Programmable Telcos, Andreas Granig
Programmable Testing for Programmable Telcos, Andreas GranigProgrammable Testing for Programmable Telcos, Andreas Granig
Programmable Testing for Programmable Telcos, Andreas Granig
Alan Quayle
 
Simplify your SAP CPI development with Figaf
Simplify your SAP CPI development with FigafSimplify your SAP CPI development with Figaf
Simplify your SAP CPI development with Figaf
Daniel Graversen
 
Fmq lqos introduction npi rev 2
Fmq  lqos introduction npi rev 2Fmq  lqos introduction npi rev 2
Fmq lqos introduction npi rev 2
amitsonuksms
 
Automate SAP PI/PO Migration
Automate SAP PI/PO Migration Automate SAP PI/PO Migration
Automate SAP PI/PO Migration
Daniel Graversen
 
How to expose shortcuts in competitive poc
How to expose shortcuts in competitive pocHow to expose shortcuts in competitive poc
How to expose shortcuts in competitive poc
Moti Sagey מוטי שגיא
 
Kafka Practices @ Uber - Seattle Apache Kafka meetup
Kafka Practices @ Uber - Seattle Apache Kafka meetupKafka Practices @ Uber - Seattle Apache Kafka meetup
Kafka Practices @ Uber - Seattle Apache Kafka meetup
Mingmin Chen
 
Solutions dept
Solutions deptSolutions dept
Solutions dept
Kira Yakunin
 
DevOps for SAP CPI presentation
DevOps for SAP CPI presentationDevOps for SAP CPI presentation
DevOps for SAP CPI presentation
Daniel Graversen
 
Pricer Plaza - June 2020.pptx
Pricer Plaza -  June 2020.pptxPricer Plaza -  June 2020.pptx
Pricer Plaza - June 2020.pptx
RiyanPratama21
 
HKNOG 6.0 Next Generation Networks - will automation put us out of jobs?
HKNOG 6.0 Next Generation Networks - will automation put us out of jobs?HKNOG 6.0 Next Generation Networks - will automation put us out of jobs?
HKNOG 6.0 Next Generation Networks - will automation put us out of jobs?
Tom Paseka
 

Similar to Flagger: Istio Progressive Delivery Operator (20)

Weavework Flagger Demo- AWS Container Day 2019 Barcelona
Weavework Flagger Demo- AWS Container Day 2019 BarcelonaWeavework Flagger Demo- AWS Container Day 2019 Barcelona
Weavework Flagger Demo- AWS Container Day 2019 Barcelona
 
AutoSat Fleet Management System Overview
AutoSat Fleet Management System OverviewAutoSat Fleet Management System Overview
AutoSat Fleet Management System Overview
 
Grafana overview deck - Tech - 2023 May v1.pdf
Grafana overview deck  - Tech - 2023 May v1.pdfGrafana overview deck  - Tech - 2023 May v1.pdf
Grafana overview deck - Tech - 2023 May v1.pdf
 
Change Control Management Supports Continuous Integration and Delivery
Change Control Management Supports Continuous Integration and DeliveryChange Control Management Supports Continuous Integration and Delivery
Change Control Management Supports Continuous Integration and Delivery
 
Trunk based development and Canary deployment
Trunk based development and Canary deploymentTrunk based development and Canary deployment
Trunk based development and Canary deployment
 
Reliability Pipelines With Keptn Quality Gates
Reliability Pipelines With Keptn Quality GatesReliability Pipelines With Keptn Quality Gates
Reliability Pipelines With Keptn Quality Gates
 
Disaster Recovery for Multi-Region Apache Kafka Ecosystems at Uber
Disaster Recovery for Multi-Region Apache Kafka Ecosystems at UberDisaster Recovery for Multi-Region Apache Kafka Ecosystems at Uber
Disaster Recovery for Multi-Region Apache Kafka Ecosystems at Uber
 
PAC 2019 virtual Bruno Audoux
PAC 2019 virtual Bruno Audoux PAC 2019 virtual Bruno Audoux
PAC 2019 virtual Bruno Audoux
 
Kong API Gateway.pdf
Kong API Gateway.pdfKong API Gateway.pdf
Kong API Gateway.pdf
 
Solutions Dept. Presentation
Solutions Dept. PresentationSolutions Dept. Presentation
Solutions Dept. Presentation
 
Programmable Testing for Programmable Telcos, Andreas Granig
Programmable Testing for Programmable Telcos, Andreas GranigProgrammable Testing for Programmable Telcos, Andreas Granig
Programmable Testing for Programmable Telcos, Andreas Granig
 
Simplify your SAP CPI development with Figaf
Simplify your SAP CPI development with FigafSimplify your SAP CPI development with Figaf
Simplify your SAP CPI development with Figaf
 
Fmq lqos introduction npi rev 2
Fmq  lqos introduction npi rev 2Fmq  lqos introduction npi rev 2
Fmq lqos introduction npi rev 2
 
Automate SAP PI/PO Migration
Automate SAP PI/PO Migration Automate SAP PI/PO Migration
Automate SAP PI/PO Migration
 
How to expose shortcuts in competitive poc
How to expose shortcuts in competitive pocHow to expose shortcuts in competitive poc
How to expose shortcuts in competitive poc
 
Kafka Practices @ Uber - Seattle Apache Kafka meetup
Kafka Practices @ Uber - Seattle Apache Kafka meetupKafka Practices @ Uber - Seattle Apache Kafka meetup
Kafka Practices @ Uber - Seattle Apache Kafka meetup
 
Solutions dept
Solutions deptSolutions dept
Solutions dept
 
DevOps for SAP CPI presentation
DevOps for SAP CPI presentationDevOps for SAP CPI presentation
DevOps for SAP CPI presentation
 
Pricer Plaza - June 2020.pptx
Pricer Plaza -  June 2020.pptxPricer Plaza -  June 2020.pptx
Pricer Plaza - June 2020.pptx
 
HKNOG 6.0 Next Generation Networks - will automation put us out of jobs?
HKNOG 6.0 Next Generation Networks - will automation put us out of jobs?HKNOG 6.0 Next Generation Networks - will automation put us out of jobs?
HKNOG 6.0 Next Generation Networks - will automation put us out of jobs?
 

More from Weaveworks

Weave AI Controllers (Weave GitOps Office Hours)
Weave AI Controllers (Weave GitOps Office Hours)Weave AI Controllers (Weave GitOps Office Hours)
Weave AI Controllers (Weave GitOps Office Hours)
Weaveworks
 
Flamingo: Expand ArgoCD with Flux (Office Hours)
Flamingo: Expand ArgoCD with Flux (Office Hours)Flamingo: Expand ArgoCD with Flux (Office Hours)
Flamingo: Expand ArgoCD with Flux (Office Hours)
Weaveworks
 
Webinar: Capabilities, Confidence and Community – What Flux GA Means for You
Webinar: Capabilities, Confidence and Community – What Flux GA Means for YouWebinar: Capabilities, Confidence and Community – What Flux GA Means for You
Webinar: Capabilities, Confidence and Community – What Flux GA Means for You
Weaveworks
 
Six Signs You Need Platform Engineering
Six Signs You Need Platform EngineeringSix Signs You Need Platform Engineering
Six Signs You Need Platform Engineering
Weaveworks
 
SRE and GitOps for Building Robust Kubernetes Platforms.pdf
SRE and GitOps for Building Robust Kubernetes Platforms.pdfSRE and GitOps for Building Robust Kubernetes Platforms.pdf
SRE and GitOps for Building Robust Kubernetes Platforms.pdf
Weaveworks
 
Webinar: End to End Security & Operations with Chainguard and Weave GitOps
Webinar: End to End Security & Operations with Chainguard and Weave GitOpsWebinar: End to End Security & Operations with Chainguard and Weave GitOps
Webinar: End to End Security & Operations with Chainguard and Weave GitOps
Weaveworks
 
Flux Beyond Git Harnessing the Power of OCI
Flux Beyond Git Harnessing the Power of OCIFlux Beyond Git Harnessing the Power of OCI
Flux Beyond Git Harnessing the Power of OCI
Weaveworks
 
Automated Provisioning, Management & Cost Control for Kubernetes Clusters
Automated Provisioning, Management & Cost Control for Kubernetes ClustersAutomated Provisioning, Management & Cost Control for Kubernetes Clusters
Automated Provisioning, Management & Cost Control for Kubernetes Clusters
Weaveworks
 
How to Avoid Kubernetes Multi-tenancy Catastrophes
How to Avoid Kubernetes Multi-tenancy CatastrophesHow to Avoid Kubernetes Multi-tenancy Catastrophes
How to Avoid Kubernetes Multi-tenancy Catastrophes
Weaveworks
 
Building internal developer platform with EKS and GitOps
Building internal developer platform with EKS and GitOpsBuilding internal developer platform with EKS and GitOps
Building internal developer platform with EKS and GitOps
Weaveworks
 
GitOps Testing in Kubernetes with Flux and Testkube.pdf
GitOps Testing in Kubernetes with Flux and Testkube.pdfGitOps Testing in Kubernetes with Flux and Testkube.pdf
GitOps Testing in Kubernetes with Flux and Testkube.pdf
Weaveworks
 
Intro to GitOps with Weave GitOps, Flagger and Linkerd
Intro to GitOps with Weave GitOps, Flagger and LinkerdIntro to GitOps with Weave GitOps, Flagger and Linkerd
Intro to GitOps with Weave GitOps, Flagger and Linkerd
Weaveworks
 
Implementing Flux for Scale with Soft Multi-tenancy
Implementing Flux for Scale with Soft Multi-tenancyImplementing Flux for Scale with Soft Multi-tenancy
Implementing Flux for Scale with Soft Multi-tenancy
Weaveworks
 
Accelerating Hybrid Multistage Delivery with Weave GitOps on EKS
Accelerating Hybrid Multistage Delivery with Weave GitOps on EKSAccelerating Hybrid Multistage Delivery with Weave GitOps on EKS
Accelerating Hybrid Multistage Delivery with Weave GitOps on EKS
Weaveworks
 
The Story of Flux Reaching Graduation in the CNCF
The Story of Flux Reaching Graduation in the CNCFThe Story of Flux Reaching Graduation in the CNCF
The Story of Flux Reaching Graduation in the CNCF
Weaveworks
 
Shift Deployment Security Left with Weave GitOps & Upbound’s Universal Crossp...
Shift Deployment Security Left with Weave GitOps & Upbound’s Universal Crossp...Shift Deployment Security Left with Weave GitOps & Upbound’s Universal Crossp...
Shift Deployment Security Left with Weave GitOps & Upbound’s Universal Crossp...
Weaveworks
 
Securing Your App Deployments with Tunnels, OIDC, RBAC, and Progressive Deliv...
Securing Your App Deployments with Tunnels, OIDC, RBAC, and Progressive Deliv...Securing Your App Deployments with Tunnels, OIDC, RBAC, and Progressive Deliv...
Securing Your App Deployments with Tunnels, OIDC, RBAC, and Progressive Deliv...
Weaveworks
 
Flux’s Security & Scalability with OCI & Helm Slides.pdf
Flux’s Security & Scalability with OCI & Helm Slides.pdfFlux’s Security & Scalability with OCI & Helm Slides.pdf
Flux’s Security & Scalability with OCI & Helm Slides.pdf
Weaveworks
 
Flux Security & Scalability using VS Code GitOps Extension
Flux Security & Scalability using VS Code GitOps Extension Flux Security & Scalability using VS Code GitOps Extension
Flux Security & Scalability using VS Code GitOps Extension
Weaveworks
 
Deploying Stateful Applications Securely & Confidently with Ondat & Weave GitOps
Deploying Stateful Applications Securely & Confidently with Ondat & Weave GitOpsDeploying Stateful Applications Securely & Confidently with Ondat & Weave GitOps
Deploying Stateful Applications Securely & Confidently with Ondat & Weave GitOps
Weaveworks
 

More from Weaveworks (20)

Weave AI Controllers (Weave GitOps Office Hours)
Weave AI Controllers (Weave GitOps Office Hours)Weave AI Controllers (Weave GitOps Office Hours)
Weave AI Controllers (Weave GitOps Office Hours)
 
Flamingo: Expand ArgoCD with Flux (Office Hours)
Flamingo: Expand ArgoCD with Flux (Office Hours)Flamingo: Expand ArgoCD with Flux (Office Hours)
Flamingo: Expand ArgoCD with Flux (Office Hours)
 
Webinar: Capabilities, Confidence and Community – What Flux GA Means for You
Webinar: Capabilities, Confidence and Community – What Flux GA Means for YouWebinar: Capabilities, Confidence and Community – What Flux GA Means for You
Webinar: Capabilities, Confidence and Community – What Flux GA Means for You
 
Six Signs You Need Platform Engineering
Six Signs You Need Platform EngineeringSix Signs You Need Platform Engineering
Six Signs You Need Platform Engineering
 
SRE and GitOps for Building Robust Kubernetes Platforms.pdf
SRE and GitOps for Building Robust Kubernetes Platforms.pdfSRE and GitOps for Building Robust Kubernetes Platforms.pdf
SRE and GitOps for Building Robust Kubernetes Platforms.pdf
 
Webinar: End to End Security & Operations with Chainguard and Weave GitOps
Webinar: End to End Security & Operations with Chainguard and Weave GitOpsWebinar: End to End Security & Operations with Chainguard and Weave GitOps
Webinar: End to End Security & Operations with Chainguard and Weave GitOps
 
Flux Beyond Git Harnessing the Power of OCI
Flux Beyond Git Harnessing the Power of OCIFlux Beyond Git Harnessing the Power of OCI
Flux Beyond Git Harnessing the Power of OCI
 
Automated Provisioning, Management & Cost Control for Kubernetes Clusters
Automated Provisioning, Management & Cost Control for Kubernetes ClustersAutomated Provisioning, Management & Cost Control for Kubernetes Clusters
Automated Provisioning, Management & Cost Control for Kubernetes Clusters
 
How to Avoid Kubernetes Multi-tenancy Catastrophes
How to Avoid Kubernetes Multi-tenancy CatastrophesHow to Avoid Kubernetes Multi-tenancy Catastrophes
How to Avoid Kubernetes Multi-tenancy Catastrophes
 
Building internal developer platform with EKS and GitOps
Building internal developer platform with EKS and GitOpsBuilding internal developer platform with EKS and GitOps
Building internal developer platform with EKS and GitOps
 
GitOps Testing in Kubernetes with Flux and Testkube.pdf
GitOps Testing in Kubernetes with Flux and Testkube.pdfGitOps Testing in Kubernetes with Flux and Testkube.pdf
GitOps Testing in Kubernetes with Flux and Testkube.pdf
 
Intro to GitOps with Weave GitOps, Flagger and Linkerd
Intro to GitOps with Weave GitOps, Flagger and LinkerdIntro to GitOps with Weave GitOps, Flagger and Linkerd
Intro to GitOps with Weave GitOps, Flagger and Linkerd
 
Implementing Flux for Scale with Soft Multi-tenancy
Implementing Flux for Scale with Soft Multi-tenancyImplementing Flux for Scale with Soft Multi-tenancy
Implementing Flux for Scale with Soft Multi-tenancy
 
Accelerating Hybrid Multistage Delivery with Weave GitOps on EKS
Accelerating Hybrid Multistage Delivery with Weave GitOps on EKSAccelerating Hybrid Multistage Delivery with Weave GitOps on EKS
Accelerating Hybrid Multistage Delivery with Weave GitOps on EKS
 
The Story of Flux Reaching Graduation in the CNCF
The Story of Flux Reaching Graduation in the CNCFThe Story of Flux Reaching Graduation in the CNCF
The Story of Flux Reaching Graduation in the CNCF
 
Shift Deployment Security Left with Weave GitOps & Upbound’s Universal Crossp...
Shift Deployment Security Left with Weave GitOps & Upbound’s Universal Crossp...Shift Deployment Security Left with Weave GitOps & Upbound’s Universal Crossp...
Shift Deployment Security Left with Weave GitOps & Upbound’s Universal Crossp...
 
Securing Your App Deployments with Tunnels, OIDC, RBAC, and Progressive Deliv...
Securing Your App Deployments with Tunnels, OIDC, RBAC, and Progressive Deliv...Securing Your App Deployments with Tunnels, OIDC, RBAC, and Progressive Deliv...
Securing Your App Deployments with Tunnels, OIDC, RBAC, and Progressive Deliv...
 
Flux’s Security & Scalability with OCI & Helm Slides.pdf
Flux’s Security & Scalability with OCI & Helm Slides.pdfFlux’s Security & Scalability with OCI & Helm Slides.pdf
Flux’s Security & Scalability with OCI & Helm Slides.pdf
 
Flux Security & Scalability using VS Code GitOps Extension
Flux Security & Scalability using VS Code GitOps Extension Flux Security & Scalability using VS Code GitOps Extension
Flux Security & Scalability using VS Code GitOps Extension
 
Deploying Stateful Applications Securely & Confidently with Ondat & Weave GitOps
Deploying Stateful Applications Securely & Confidently with Ondat & Weave GitOpsDeploying Stateful Applications Securely & Confidently with Ondat & Weave GitOps
Deploying Stateful Applications Securely & Confidently with Ondat & Weave GitOps
 

Recently uploaded

openEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain SecurityopenEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain Security
Shane Coughlan
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
Rakesh Kumar R
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptxLORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
lorraineandreiamcidl
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Green Software Development
 
Why Mobile App Regression Testing is Critical for Sustained Success_ A Detail...
Why Mobile App Regression Testing is Critical for Sustained Success_ A Detail...Why Mobile App Regression Testing is Critical for Sustained Success_ A Detail...
Why Mobile App Regression Testing is Critical for Sustained Success_ A Detail...
kalichargn70th171
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOMLORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
lorraineandreiamcidl
 
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteAI Pilot Review: The World’s First Virtual Assistant Marketing Suite
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
Google
 
Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604
Fermin Galan
 
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
Łukasz Chruściel
 
Codeigniter VS Cakephp Which is Better for Web Development.pdf
Codeigniter VS Cakephp Which is Better for Web Development.pdfCodeigniter VS Cakephp Which is Better for Web Development.pdf
Codeigniter VS Cakephp Which is Better for Web Development.pdf
Semiosis Software Private Limited
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
Ayan Halder
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
Aftab Hussain
 
Launch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in MinutesLaunch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in Minutes
Roshan Dwivedi
 
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
mz5nrf0n
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software  Development Company in Noida - DeugloEmpowering Growth with Best Software  Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
Deuglo Infosystem Pvt Ltd
 
Enterprise Resource Planning System in Telangana
Enterprise Resource Planning System in TelanganaEnterprise Resource Planning System in Telangana
Enterprise Resource Planning System in Telangana
NYGGS Automation Suite
 
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR  SPRING BOOT (SPRING IO)GOING AOT WITH GRAALVM FOR  SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
Alina Yurenko
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 
SWEBOK and Education at FUSE Okinawa 2024
SWEBOK and Education at FUSE Okinawa 2024SWEBOK and Education at FUSE Okinawa 2024
SWEBOK and Education at FUSE Okinawa 2024
Hironori Washizaki
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Green Software Development
 

Recently uploaded (20)

openEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain SecurityopenEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain Security
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptxLORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
 
Why Mobile App Regression Testing is Critical for Sustained Success_ A Detail...
Why Mobile App Regression Testing is Critical for Sustained Success_ A Detail...Why Mobile App Regression Testing is Critical for Sustained Success_ A Detail...
Why Mobile App Regression Testing is Critical for Sustained Success_ A Detail...
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOMLORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
 
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteAI Pilot Review: The World’s First Virtual Assistant Marketing Suite
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
 
Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604
 
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
 
Codeigniter VS Cakephp Which is Better for Web Development.pdf
Codeigniter VS Cakephp Which is Better for Web Development.pdfCodeigniter VS Cakephp Which is Better for Web Development.pdf
Codeigniter VS Cakephp Which is Better for Web Development.pdf
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
 
Launch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in MinutesLaunch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in Minutes
 
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software  Development Company in Noida - DeugloEmpowering Growth with Best Software  Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
 
Enterprise Resource Planning System in Telangana
Enterprise Resource Planning System in TelanganaEnterprise Resource Planning System in Telangana
Enterprise Resource Planning System in Telangana
 
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR  SPRING BOOT (SPRING IO)GOING AOT WITH GRAALVM FOR  SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 
SWEBOK and Education at FUSE Okinawa 2024
SWEBOK and Education at FUSE Okinawa 2024SWEBOK and Education at FUSE Okinawa 2024
SWEBOK and Education at FUSE Okinawa 2024
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
 

Flagger: Istio Progressive Delivery Operator

  • 1. Flagger Istio Progressive Delivery Operator Stefan Prodan @stefanprodan Istio London November 2018 1
  • 2. What is Progressive Delivery? Progressive Delivery is Continuous Delivery with fine-grained control over the blast radius. Building blocks ● User segmentation ● Traffic management ● Observability ● Automation https://redmonk.com/jgovernor/2018/08/06/towards-progressive-delivery/
  • 3. Introducing Flagger Flagger is a Kubernetes operator that automates the promotion of canary deployments using Istio routing for traffic shifting and Prometheus metrics for canary analysis. Flagger implements a control loop that gradually shifts traffic to the canary while measuring key performance indicators. Based on the KPIs analysis a canary is promoted or aborted.
  • 4. Get Flagger helm repo add flagger https://flagger.app helm upgrade -i flagger flagger/flagger --namespace=istio-system --set metricsServer=http://prometheus.istio-system:9090 --set controlLoopInterval=1m helm upgrade -i flagger-grafana flagger/grafana --namespace=istio-system --set url=http://prometheus.istio-system:9090
  • 6. Key Performance Indicators The decision to pause the traffic shift, abort or promote a canary is based on: ● Deployment health status ● Request success rate percentage ● Request latency average value
  • 7. Observability Flagger emits Kubernetes events related to the advancement and final status of a canary analysis.
  • 8. Observability RED + USE methods Flagger comes with a Grafana dashboard for canary analysis.
  • 9. Canary CRD 9 apiVersion: flagger.app/v1alpha1 kind: Canary metadata: name: podinfo namespace: test spec: # deployment reference targetRef: apiVersion: apps/v1 kind: Deployment name: podinfo # hpa reference (optional) autoscalerRef: apiVersion: autoscaling/v2beta1 kind: HorizontalPodAutoscaler name: podinfo service: # container port port: 9898 # Istio gateways (optional) gateways: - public-gateway.istio-system.svc.cluster.local # Istio virtual service host names (optional) hosts: - app.istio.weavedx.com
  • 10. Canary CRD - analysis spec 10 canaryAnalysis: # maximum number of failed metric checks # before rolling back the canary threshold: 10 # max traffic percentage routed to canary maxWeight: 50 # canary increment step stepWeight: 5 metrics: - name: istio_requests_total # minimum req success rate percentage (non 5xx responses) threshold: 99 interval: 1m - name: istio_request_duration_seconds_bucket # maximum req duration P99 (milliseconds) threshold: 500 interval: 30s
  • 12. Canary automation for production systems
  • 13. Flagger control loop stage 1-5 ● scan for canary deployments ● create the primary deployment and HPA ● create the ClusterIP services for primary and canary ● create an Istio virtual service with weighted destinations mapped to primary and canary ClusterIP services ● check primary and canary deployments status ○ halt advancement if a rolling update is underway ○ halt advancement if pods are unhealthy ● increase canary traffic weight percentage from 0% to 5% (step weight) ● check canary HTTP request success rate and latency ○ halt advancement if any metric is under the specified threshold ○ increment the failed checks counter
  • 14. Flagger control loop stage 6-7 ● check if the number of failed checks reached the threshold ○ route all traffic to primary ○ scale to zero the canary deployment ○ mark canary as failed ○ wait for the canary to be updated (revision bump) and start over ● increase canary traffic by 5% (step weight) ○ halt advancement while canary request success rate is under the threshold ○ halt advancement while canary request duration P99 is over the threshold ○ halt advancement if the primary or canary deployment becomes unhealthy ○ halt advancement while canary deployment is being scaled up/down by HPA
  • 15. Flagger control loop stage 8-12 ● promote canary to primary when canary weight reaches max ○ copy canary deployment spec template over primary ● wait for primary rolling update to finish ○ halt advancement if pods are unhealthy ● route all traffic to primary ● scale to zero the canary deployment ● mark canary as finished ● wait for the canary to be updated (revision bump) and start over