Keptn is a framework for automating continuous delivery and operations of Kubernetes applications. It uses a GitOps-based approach with event-driven automation to enable unbreakable delivery pipelines and self-healing deployments. Keptn provides autonomous control plane capabilities including automated testing, deployment, evaluation and operations through reusable services. The demo shows how keptn can onboard a service, deploy new versions through the stages, and enable automated remediation through integration with monitoring and runbook tools.

1. Building Autonomous Operations for
Kubernetes with keptn

2. 2Confidential
Agenda
• Why we have to build keptn
• The problems companies are currently facing
• What keptn is and how it works
• Demo – Onboarding a Service (throughout the talk)
• Demo – Runbook Automation & Self-healing

3. 3Confidential
Why we have to build
Because Cloud Native Continuous Delivery is a big challenge.

4. Confidential 4

5. Confidential 5
Collecting more evidence: https://dynatrace.ai/acsurvey
< 1 day < 1 week 1 to 2 weeks 2 to 3 weeks 3 to 4 weeks > 4 weeks
Small (11-100 employees) Medium (101-1000 employees)
Large (1001-5000 employees) Extra large (over 5000 employees)
3%
20%
35%
42%
2 days 12.5 days
95th Percentile Median
Commit Cycle Time: From Dev to Pro
Goal: 1h to Production
1 hour

6. Confidential 6
Median
95th
Percentile

7. Confidential 7
How do we elevate an organization from Median to 95th percentile
• Automated Quality (Shift-Left)
• Goal: Stop Bad Code Changes Early & Automated
• Automated Testing & Quality Gates
• Automated Multi-Stage Deployments (Shift-Right)
• Goal: Increase deployments into stable environments
• Dark, Shadow, Blue/Green Deployments with Auto-Validation
• Automated Operations (Self-Healing)
• Goal: NoOps & Zero-Impact on End Users
• Automated Remediation & (On-Demand) Scaling

8. Confidential 8
Why companies can‘t delivering high
quality faster

9. Confidential 9
What we have seen organizations actually do
• Building Custom Integrations with OpenSource & Commercial tools for
• Deployment Pipelines
• Testing Pipelines
• Auto-Remediation
• Notifications
• Auditing

10. Confidential 10
What we have seen organizations struggle with
Quote: „Pipelines seem to become our new future unmanagable legacy code!“
• Teams stick with existing tools to protect investement
• Containing lots of custom code for tool integration, error handling, logging, ...
• Getting harder to maintain the more tools get integrated
• Pipelines becoming more complex requiring dedicated teams
• Uncoordinated deployments between pipelines resulting in unstable environments

11. Confidential 11
What is and how it works
Enterprise-grade framework for shipping and running cloud native apps

12. Confidential 12
from 10000ft
GitOps-based collaboration
Operator patterns for all logic components
Monitoring and operations as code
Built on and for Kubernetes
Event-driven and serverless
Pluggable tooling
Automated multistage unbreakable delivery pipelines
Self-healing blue/green deployments
Event-driven runbook automation
Design Principles
Core capabilities

13. Confidential 13
keptn – Conceptual Architecture
Autonomous Cloud Control Plane
GitOps
Container
Registry
Continuous
Delivery AIOps
Operations
Automation
Test
Automation
Environment Definition (shipyard file)
ChatOps
Dev Namespace Staging Namespace Production Namespace
CoreServicesPlatform
Data
Provider

14. Confidential 14
Autonomous Cloud Control Plane
event-
broker
authenticator
controlcli
keptn-channel
event-
broker-ext
configuration
deployment
evaluation
test
problem
new-artefact
Services
jenkins-
service
github-
service
service
now-
service
keptn: Phase 3 – Pushing an artifact and deploy to dev

15. Confidential 15
Autonomous Cloud Control Plane
prodstagedev
1: push
2: deploy (dark)
3: test (functional)
4: evaluate (architecture)
6: deploy (shadow)
7: test (performance)
8: evaluate (scalablity)
10: deploy (blue/green)
11: evaluate (business impact)
12: operate (NoOps)
5: promote 9: promote
Dev
keptn – Shipping through Unbreakable Continuous Delivery Pipelines

16. Confidential 16
keptn: Phase 1 – Creating a Project
$ keptn create project sockshop shipyard.yaml
prodstagedev prodstagedev
1
1 Create GitOps Config Repos
2
2 Create k8s Namespaces
shipyard.yaml
stages:
- name: "dev"
deployment_strategy: "dark"
test_strategy: "functional"
- name: "staging"
deployment_strategy: "shadow"
test_strategy: "performance"
- name: "production"
deployment_strategy: "blue_green"

17. Confidential 17
keptn: Phase 2 – Onboarding a Service
$ keptn onboard service --project=sockshop --values=values_carts.yaml
prodstagedev
1
1 Setup Registry Hooks
2
2 Apply Configuration + Istio
2
prodstagedev
2 2
values_carts.yaml
replicaCount: 1
service:
name: carts
type: LoadBalancer
externalPort: 80
container:
name: carts
resources:
limits:
cpu: 100m
memory: 128Mi
...

18. Confidential 18
keptn: Phase 3 – Pushing an artifact and deploy to dev
$ docker push yourregistry/carts:0.2
prodstagedev
1
prodstagedev
1
1 New Artifact: carts:0.2 dev
2
2 Start Deployment: carts:0.2 (dev, dark)
3
3 Deployment Finished: Success or Failure
4
4 Start Tests: Functional, Dev, carts
5
5 Tests Finished: Test Result & State
6
6 Start Deployment Evaluation: Monspec
7
7 Evaluation Finished: Deployment Score
8
"indicators": [ {
"id" : “Prometheus:go_memstats_alloc",
"metricScore": 20
},{
"id": “Dynatrace:Avg_ActionDuration",
"metricScore": 20
} ],
"objective" : { "pass" : 90, “warning” : 75 }
6
8 Send new Artifact Event: carts:0.2 stage

19. Confidential 19
keptn: Phase 4 – Promoting an artifact to staging
$ keptn send event newartifact carts:0.2 stage
prodstagedev
1
prodstagedev
1
1 New Artifact: carts:0.2 stage
2
2 Start Deployment: carts:0.2 (stage, shadow)
3
3 Deployment Finished: Success or Failure
4
4 Start Tests: Performance, Stage, carts
5
5 Tests Finished: Test Result & State
6
6 Start Deployment Evaluation: Monspec
7
7 Evaluation Finished: Deployment Score
8
6
8 Send new Artifact Event: carts:0.2 prod
"indicators": [ {
"id": “Dynatrace:RT_Real",
"metricScore": 20
},{
"id": “Dynatrace:RT_Shadow",
"metricScore": 20
} ],
"objective" : { "pass" : 95 }

20. Confidential 20
keptn: Phase 4 – Promoting an artifact to production: Blue/Green no issues
$ keptn send event newartifact carts:0.2 production
prodstagedev
1
prodstagedev
1
1 New Artifact: carts:0.2 prod
2
2 Start Deployment: carts:0.2 (prod, bluegreen)
3
3 Deployment Finished: Success or Failure
4
4 Start Deployment Evaluation: Monspec
5
5 Evaluation Finished: Deployment Score4
6
6 Blue/Green Switch
"indicators": [ {
"id": “Dynatrace:ConversionRate",
"metricScore": 100
}],
"objective" : { "pass" : 95 }

21. Confidential 21
keptn: Phase 4 – Promoting an artifact to production: Blue/Green with problem
$ keptn send event newartifact carts:0.2 production
prodstagedev
1
prodstagedev
1
1 New Artifact: carts:0.2
2
2 Start Deployment: carts:0.2 (prod, bluegreen)
3
3 Deployment Finished: Success or Failure
4
4 Start Deployment Evaluation: Monspec
5
5 Evaluation Result: Dynatrace Problem Notify
4
6
6 Remediation: Back to Blue
6 Remediation: Trigger ServiceNow Workflow
6
66
6 Remediation: Update Configuration: carts:0.1

22. 22Confidential
Demo – Onboarding a Service

23. Confidential 23

24. Confidential 24
Use case in a nutshell
• Create project sockshop
• Onboard carts service and carts database
• Build a new artefact for the carts service and watch keptn doing the deployment
stages:
- name: "dev"
deployment_strategy: "direct"
test_strategy: "functional"
- name: "staging"
deployment_strategy: "blue_green_service"
test_strategy: "performance"
- name: "production"
deployment_strategy: "blue_green_service“

25. 25Confidential
Demo – Runbook Automation &
Self-Healing

26. Confidential 26

27. Confidential 27
Use case in a nutshell
• Dynatrace for Monitoring & ServiceNow for Runbook Automation
• Both tools a properly configured meaning that Dynatrace can send problem notifications and ServiceNow
can trigger runbooks
• Load generation to simulate production traffic
• Configuration change at runtime
• Problem detection by Dynatrace
• Incident creation & Workflow execution by ServiceNow

28. 28Confidential
Curious …
- how to use keptn?
- how to contribute to keptn?

29. Confidential 29
You want to try out keptn?
• Setup keptn: https://keptn.sh/docs/0.2.0/installation/setup-keptn/
• Instructions to install keptn on a GKE cluster
• Instructions to install keptn on a AWS EKS cluster (coming soon)
•
•
•
•
•

30. Confidential 30
Contribute to the keptn project with your custom service
1. Identify the channel you want to subscribe to:
2. Write your knative service
• A reference service in TypeScript is provided on:
https://github.com/keptn/keptn-service-template
• Use the payload of the CloudEvents to implement the
functionality you want
3. Provide a service description and the channel
subscriptions in a manifest file
4. Deploy your service in the keptn namespace
•
•
•
•
•
•
jenkins-
service
github-
service
custom-
service
slack-
service
spinnake
r-service
neotys-
service

31. Confidential 31
Want to learn more?
• Learn how to use keptn: https://keptn.sh/docs/
• Clone, fork, star, contribute to keptn: https://github.com/keptn/
• Develop your own keptn-services: https://github.com/keptn/keptn-service-template
• Join our community (slack invite, meetings): https://github.com/keptn/community
• Follow us @keptnproject
• Fill out our survey: https://dynatrace.ai/acsurvey

32. Building Autonomous Operations for
Kubernetes with keptn

33. Confidential 33
dynatrace.com

34. Confidential 34
Autonomous Cloud Control Plane
event-
broker
authenticator
controlcli
keptn-channel
event-
broker-ext
deployment
evaluation
test
problem
Services
jenkins-
service
github-
service
service
now-
service
cli
configuration
new-artefact
keptn: Phase 1 – Creating a Project

35. Confidential 35
Autonomous Cloud Control Plane
event-
broker
authenticator
controlcli
keptn-channel
event-
broker-ext
deployment
evaluation
test
problem
Services
jenkins-
service
github-
service
service
now-
service
cli
configuration
new-artefact
keptn: Phase 2 – Onboarding a Service

36. Confidential 36
Autonomous Cloud Control Plane
event-
broker
authenticator
controlcli
keptn-channel
event-
broker-ext
configuration
deployment
evaluation
test
problem
new-artefact
Services
jenkins-
service
github-
service
service
now-
service
keptn: Phase 3 – Pushing an artifact and deploy to dev