The document discusses the issue of configuration drifts in Kubernetes (k8s) and the challenges associated with maintaining infrastructure as code (IaC). It highlights the significant impacts of these drifts, such as security vulnerabilities and increased costs, and proposes strategies for drift detection and mitigation using tools like Terraform and GitOps methodologies. The speaker emphasizes the importance of both technological solutions and changes in team behavior to prevent infrastructure drift.

1. Eran Bibi, Co-Founder and CPO, Firefly
Don’t Panic: Get Your K8s
Drifts Under Control

2. E6i
Co-Founder & CPO, Firefly
Former Head of DevOps & SaaS Platform, Aqua
A CLI Type of Guy

3. • Code reviews
• Immutable infrastructure
• Disaster recovery planning
• Shift left security scanning for the infrastructure
• Pre-deployment cost analysis
Why IaC?
kubeadm
Ansible
Terraform
Cloud Console
eksctl
kops
The Hard Way
Infrastructure-as-
code
The Challenge: Infrastructures Tend to

4. 9 out of 10
Large Scale Deployments
Have Configuration Drifts
50% of the Drifts
Were Unnoticed
Toil 60%
Security 20%
Reliability 20%
100% Impact

5. Infrastructure Drift
Drift is what happens whenever the real-world state of your
infrastructure differs from the defined one in your configuration.
The Configuration The Real-world State

6. And Now for Real
The Configuration
The Real-world State

7. K8s drift
The Configuration
The Real-world State

8. Data / Outcome
Drifts: Cause & Effect
Manual changes
(“emergency
fix”)
Environment
discrepancies
Automated apps
Infrastructure Drift:
• Security vulnerabilities
• Increased costs
• Excessive work
• Decreased efficiency
• Reliability risk
Misalignment between
IaC and Cloud APIs

9. K8s drifts: Double Trouble

10. 1st Approach:
Establish a Drift Detection
Strategy
Understand when drifts may become risky
Catch the drifts: implement drift detection automation per your stack/s
Classify and route each response to respective individual or team

11. Drift Detection
using
Terraform plan

12. 2nd Approach: GitOps

13. Takeaways
● Implement tools to detect drifts
○ Terraform Plan
○ driftctl
○ kubediff
● Consider GitOps methodology as a mitigation strategy
○ ArgoCD
○ Flux
○ JenkinsX
● Tools are not enough, you also need to change behavior

14. May your Infrastructure Never Drift
gofirefly.io
Thank You!