Keptn is an open-source project that provides tools to enable continuous delivery and automation for modern applications using Kubernetes. It allows developers to focus on code and DevOps teams to focus on tools rather than building custom pipelines. Keptn provides automated multi-stage delivery pipelines, automated quality gates, self-healing deployments, and enables zero-touch toolchain integration and updates. It also supports automated problem remediation in production for continuous operations. Keptn follows cloud-native design principles and provides a common way for organizations to achieve autonomous delivery and operations.

1. Shipping code like a
@keptnProject
https://keptn.sh
https://github.com/keptn
Rob.Jahn@Dynatrace.com
Technical Partner Manager
& Keptn Advocate

2. 2
Every CIOs vision of their Cloud Native investment

3. 3
The unfortunate reality for most….
1. Complexity
2. Manual operations
3. Lack information
4. People
5. Identifying root cause

4. 4* based on our ACM survey

5. 5
Maturity Questions
Mean Time to Innovation (MTTI)
1. Are you on two week sprints?​
2. Do you also release 25 times a year?​
3. How long does it take you to deploy a new release to production?​
4. Do you measure feature adoption?
Mean Time to Remediate (MTTR)
1. How long does it take to rollback/roll forward a faulty release?​
2. To release any software in your organization, how many manual steps does it
take?​

6. 6
MTTI
Mean Time to Innovation
MTTR
Mean Time to Remediate
4.8 days
4 hours
~ 10min
12.5 days 2 days ~ 1 hour
The reality and evidence supports
it!
https://dynatrace.ai/acsurvey
Only < 5% is „Cloud Native“

7. 7
That is why we are building
Because cloud native delivery and operations is a BIG challenge for enterprises!
Cloud Native

8. 8
.
https://dzone.com/articles/keep-calms-and-devops-c-is-for-culture
Culture first, but change is not easy…

9. 9
Common Pipeline delivery challenges
1. Pipelines are semi-automated
2. Each team building duplicate delivery & integration mechanisms
3. Changing tooling causes a ton pipeline re-work
4. Lack software architecture for DevOps automation
5. Lack auditing support for customized tool integrations
6. Lack of standards
7. Too many technologies to keep up with
8. What else ????

10. 10
So what is this thing?
• Open source initiative to make continuous delivery and operations automation
a commodity for modern cloud-native applications
• Enterprise-grade event-based framework for shipping and operating cloud-
native applications built to run on any Kubernetes cluster
• With Keptn ...
• Developers can focus on code instead of creating YAML files
• DevOps can focus on tools instead of building pipelines
• SREs can enforce processes instead of debugging problems

11. 11
Automated OperationsContinuous Deployment
Core Capabilities -- accelerates building autonomous clouds
Event-drivenrunbook
automation
Productionproblemscan beautomatically
remediatedin real-timebyexecuting
runbooksthatrequirenomanual
intervention.
Self-healing blue/green
deployments
Deploymentsthatfollowthe“Operationsas
Code”paradigmautomaticallyremediate
problemsandgetyourdeploymentpipeline
workingagainin underaminute.
Automatedmulti-stage
unbreakable pipelines
GitOps-enableddeliverypipelineswith
automatedqualitygatessupportautomated
testingandmonitoring-as-a-service.

12. 12
First: solves the Continuous Delivery Problem!
Many approaches today are brittle, error-prone and hard to maintain

13. 13
Quote: “We spend more time in manual communicaton than remediating issues“
Second: has Continuous Operations at its Core!
ENGAGE TRIAGE FIND & ASSEMBLE RESOLVE RESTORE
MANUAL
COMMUNICATION
MANUAL
COMM
Before
After
RESTORERESOLVE
NUMBER
OF ISSUES BEFORE: mostly manual
AFTER: mostly automated

14. 14
Increase Quality &
Level of Automation
Increase Speed &
Reduce Costs
Automated
Testing
Continuous
Performance
Auto Quality
Gates
AUTOMATE
OPERATIONS
AUTOMATE
DEPLOYMENT
AUTOMATE
MONITORING
Feature
Flagging
Adaptive
Scaling
Auto
Roll-Back
Canary
Releases
Blue /
Green
Deploymen
ts
Auto-
Remediation
AUTOMATE
QUALITY
Strategically Used as Pipeline Feature
Building Blocks for ACM/Cloud Natives!

15. 15
Design Principals – follows patterns for modern applications
GitOps-based
collaboration
Operator patterns for all
logic components
Monitoring and
operations ascode
Built onandfor Kubernetes Event-drivenandserverless Pluggabletooling

16. 16
Keptn architecture
Platform
keptn
cli
api eventbroker
REST
API
bridge datastore
gitconfiguration-service
internal-
keptn-
services
distributor
distributor
keptn services
Continuous
Delivery
Container
Registry
Operations
Automation AIOps
GitOps
Test
Automation ChatOps
Data
Provider
dev staging production keptnNamespaces:
distributor
Mini-kube AWS EKS Azure AKS Google GKE Openshift Pivotal PKS

17. 17
Keptn Use Cases - Installation
1. One-Line Installation: on most popular k8s platforms
2. Zero-Touch Toolchain Integration: No custom tool integrations needed
3. Zero-Touch Cloud Native Services: Enables GitOps event-driven CD/CO for your
services

18. 18
Config ChatOps IT Autom
Deploy Test Observe
One-Line Installation: $ keptn install --platform=[aks, gke, openshift]
1. Keptn installer pod
2. Keptn pods and services(such as logging, keptn bridge,event broker)
3. Keptn routes (such asauthenticator, bridge, gate-keeper,helm service)
4. Keptn channels (such as configuration-changed, deployment-finished, evaluation-done)
5. Istio and knative pods

19. 19
Config ChatOps IT Autom
Deploy Test Observe
Zero-Touch Toolchain Integration: $ keptn wear uniform uniform.yaml
uniform.yaml
services:
- name: "dockerhub"
- name: "jenkins-service"
subscribedchannels:
- new-artifact
- deployment-finished
-name: "slack-service”

20. 20
Zero-Touch Toolchain Integration:
Config ChatOps IT Autom
Deploy Test Observe
Pluggable tooling
All tools leveraged by keptn
can be replaced based
on your tool preferences.

21. Keptn Orders Demo -- tool chain ”uniform”
Config ChatOps IT Autom
Deploy Test Observe
• Configuration:GitHub
• Deploy:Helm
• Chat:Slack& MS Teams
• IT Automation:None
• Test:Jmeter
• Observe:PitometerQualityGates

22. Keptn Orders Demo Setup
DockerRegistrywithpre-builtimages
GitHub Org
Staging Production
Update GitHub repositories
- front-end
- catalog-service
- customer-service
- catalog-service
Dev
https://github.com/keptn-orders

23. 23
Keptn Use Cases – Continuous Delivery
1. Re-Think Pipelines: Gone are the days of custom pipeline coding!
2. Automated Multi-Stage Delivery: Risk-Free auto deployment through multi-stage
delivery pipelines
3. Automated Quality Gates: stops bad changes before production using Pitometer
4. Self-Healing Blue/Green Deployments: reverts bad changes before impacting end-
users
5. Zero-Touch Toolchain Updates: Add/Remove/Replace tools without custom coding
6. Mastering Continuous Delivery: Risk-Free Automated Deployments

24. 24
Config ChatOps IT Autom
Deploy Test Observe
Re-Think Pipelines: $ keptn create project keptn-sample shipyard.yaml
S
T
A
G
I
N
G
P
R
O
D
DirectUpdateC D
Blue/GreenUpdateC D

25. 25
Config ChatOps IT Autom
Deploy Test Observe
Zero-Touch Cloud Native Services: $ keptn onboard service myservice [values.yaml]
S
T
A
G
I
N
G
P
R
O
D
DirectUpdateC D
Blue/GreenUpdateC D
PLACEHOLDER
PLACEHOLDER

26. As a developer, I want to safely deploy a new
version by just providing my docker image

27. 27
Config ChatOps IT Autom
Deploy Test Observe
Automated Multi-Stage Delivery: $ keptn new artifact myservice:1.0.0
S
T
A
G
I
N
G
P
R
O
D
ScoreDirect PerformanceUpdate Promote?C D T O
ScoreBlue/GreenUpdate Keep?C D T O
PLACEHOLDER
PLACEHOLDER
1.0.0
1 1 90
/
100
1.0.0
1 1 1 75
/
100
P
R
O
M
O
T
E
K
E
E
P

28. 28
Config ChatOps IT Autom
Deploy Test Observe
Automated Quality Gates: $ keptn new artifact myservice:2.0.0
S
T
A
G
I
N
G
P
R
O
D
ScoreDirect PerformanceUpdate Promote?C D T O
ScoreBlue/GreenUpdate Keep?C D T O
1.0.0
1 1 45
/
100
1.0.0
1 1 1
2.0.0
2 2
A
B
O
R
T

29. Pitometer ???
devices used to measure a ship's speed
relative to the water. They are used on
both surface ships and submarines.
Data from the pitometer log is usually
fed directly into the ship's navigation
system. (Wikipedia)

30. 30
A Quick word on Keptn Quality Gates: Automated Deployment Validation
Metric Source &
Query
Grading Details
& Metric Score
Pitometer Specfile
Total Scoring
Objectives
2GB
Allocated Bytes (from Prometheus)
> 2GB: 0 Points
< 2GB: 20 Points
5%
2% < 2%: 0 Points
< 5%: 10 Points
> 5%: 20 Points
Conversion Rate (Dynatrace)
GraderSource
If value: 3GB
Score: 0
If value: 3.9%
Score: 10
Total Score: 10

31. 31
Automated Quality Gates with Keptn’s Pitometer
#1 Call Pitometer Library
Pass / Fail
Promote?
Quality Gates as Code
•
First extensions by
https://www.dynatrace.com/news/blog/automated-deployment-and-architectural-validation-with-pitometer-and-keptn/

32. 32
Pitometer Client
 Evaluates & Scores results
 Community can build more
 1 to many modules as required
 Queries data for requested
indicators
 Community can build more
Pitometer Module  Core library for Data, sources,
graders and perfspec file parsing
PerfSpec JSON
Grader Module
Graders
Graders
Data Module(s)
Custom Code
 Logic to assemble and configure
modules. Read file and pass to
Pitometer functions
Pitometer JSON Results
Keptn Pitometer: Extensible design using NodeJS Modules
https://github.com/keptn/pitometer

33. 33
Quality Gate Examples and Metrics
Check 1
• Is bad coding
leading to higher
costs?
Check 2
• New dependencies? On
Purpose?
• Services connecting
accurately?
• Number of container
instances needed?
Check 3
• Are we jeopardizing
our SLAs?
• Does load balancing
work?
• Difference between
Canaries?
Check 4
• Did we introduce
new “hidden”
exceptions?
Metrics
 Memory usage
 Bytes sent /
received
 Overall CPU
 CPU per
transaction type
Metrics
 Number of incoming /
outgoing
dependencies
 Number of instances
running on containers
 Metrics
 Response Time
(Percentiles)
 Throughput & Perf
per Instance /
Canary
 Metrics
 Total Exceptions
 Exceptions by
Class & Service

34. 34
Config ChatOps IT Autom
Deploy Test Observe
Self-Healing Blue/Green Deployments: $ keptn new artifact myservice:3.0.0
S
T
A
G
I
N
G
P
R
O
D
ScoreDirect PerformanceUpdate Promote?C D T O
ScoreBlue/GreenUpdate Keep?C D T O
1.0.0
1 1 85
/
100
1.0.0
1 1 1
2.0.0
2 2
3.0.0
3 3
3.0.0
3 3 3
80
/
100
P
R
O
M
O
T
E
R
E
V
E
R
T

35. As a DevOps Engineer, I want switch tools
without writing custom pipeline code

36. 36
Config ChatOps IT Autom
Deploy Test Observe
Zero-Touch Toolchain Updates: $ keptn update uniform <+neo,+end2end,+spinnaker>
S
T
A
G
I
N
G
P
R
O
D
ScoreDirect PerformanceUpdate Promote?C D T O
ScoreBlue/GreenUpdate Keep?C D O
1.0.0
1 1
1.0.0
1 1 1
2.0.0
2 2
3.0.0
3 3
End2EndT

37. 37
Config ChatOps IT Autom
Deploy Test Observe
Mastering Continuous Delivery: $ keptn new artifact myservice:4.0.0
S
T
A
G
I
N
G
P
R
O
D
ScoreDirect PerformanceUpdate Promote?C D T O
ScoreBlue/GreenUpdate Keep?C D O
1.0.0
1 1
1.0.0
1 1 1
2.0.0
2 2
3.0.0
3 3
4 4 4
End2EndT
4.0.0
4.0.0
4 4 95
/
100
90
/
100
K
E
E
P
P
R
O
M
O
T
E

38. 38
Keptn Use Cases – Continuous Operations
1. Self-Healing Production: Automated Problem Remediation (Scale-Up, Scale-
Down, ...)
2. Self-Healing/Continuous BizDevOps: Automated Business Operations
Optimization, Turn on/off feature flags based on conversion rates ...
3. Auto-Protect Production & Business: Stop DDOS attacks, Redirect Bot Traffic ...
4. Chaos-Driven Operation Readiness: Chaos Engineering to validate your
production self-healing

39. As an SRE, I want production to auto-
remediate problems detected by monitioring

40. 40
4 4
Self-Healing Production: $ keptn new problem <Services, Root Cause>
P
R
O
D
Evaluate
Decide
Act
Notify
4 Escalate
4 4
#1 Evaluate Problem #3 Execute Action #4 Notify on Success #5 Escalate on Failure#2 Decide on Remediation

41. 41
• https://github.com/keptn/community
• Twitter - @keptnproject
• Slack - keptn.slack.com
• Biweekly community meetings
• Zoom https://dynatrace.zoom.us/j/578828410
• Notes https://tinyurl.com/keptn-meeting-notes
• Blogs– http://medium.com/keptn
• Web – http://keptn.sh
• Email- keptn@dynatrace.com
Jointhe community
Rob.Jahn@Dynatrace.com
Technical PartnerManager
& KeptnAdvocate

42. 42

43. 43
Keptn Orders Application
• Order processing application
• Web UI with 3 java spring boot microservices
with embedded databases
• Components pre-compiled, built as a Docker
image, and staged on dockerhub.com
• All source code can be found at
https://github.com/keptn-orders

44. 44
Keptn-Orders GitHub Repo Structure
App Repos
front-end
customer-
service
order-service
catalog-
service
/jmeter/
basiccheck.jmx
front-end_load.jmx
/perfspec/
perfspec.json
Other files
* Source Code
* CI Build pipeline files
* Dockerfile
* Build scripts
Same structure as
front-end
Same structure as
front-end
Same structure as
front-end
orders-
project
Dev Branch
/Helm-Charts/
chart values files
/templates/
istio virtual gateway YAML
service YAML
deployment YAML
environment gateway YAML
Staging Branch
Same structure and files as Dev Branch
Production Branch
Same structure and files as Dev Branch
Folders &Files Keptn Repo Folders &Files
https://github.com/jahn-keptn-orders-aks2

45. 45
VM Worker Pool
NAME SPACES
Kubernetes Demo Cluster Setup with Keptn & Deployed Keptn-Orders Application
Node 1
Dynatrace
Keptn
Staging
Production front-end
front-end
customer-
service
order-service
catalog-
service
bridge
OneAgent
Operator
OneAgent
(node 1)
front-end
customer-
service
order-service
catalog-
service
gatekeeper
Jmeter-
service
helm-service …
Dev front-end
customer-
service
order-service
catalog-
service
Node X