The document discusses the use of OpenTelemetry for improving GitOps workflows in Kubernetes, addressing challenges like unmeasurable change impact, chaotic pipelines, and observability tool sprawl. It highlights how Keptn can provide app-aware change traceability, enforce delivery checks natively in Kubernetes, and unify observability data access. The goal is to enhance the deployment process through event-driven delivery tasks and standardized observability metrics.

1. OpenTelemetry for GitOps:
Tracing Deployments from Git
Git Commit to Production on
K8s
Andreas Grabner
CNCF Ambassador, DevRel @ CNCF Keptn
Global DevRelLead @ Dynatrace
Keptn: https://lifecycle.keptn.sh

2. We are at a Dev Conference: Anybody knows my first programming language?

3. We are doing
LIVE Demos
using only
Open-Source
tooling 
If you stay you get to see YAML, a Terminal, graphs, traces and much more 
https://github.com/keptn-sandbox/klt-on-k3s-with-argocd

4. Let me get a quick pulse on those topics…
Hands-Up if you are using or intend to use …
DORA Metrics
1 2 3 5
4 6

5. 0: What is OpenTelemetry
1: Unmeasurable Change Impact
• “Its hard to troubleshoot failed GitOps deployments on K8s!”
• DEMO: DORA & Deployment Traces natively from K8s Pod Scheduler
2: Pipelines Gone Wild
• “Its hard adding tests, security, and SLO checks to deployments!”
• DEMO: Deployment Checks Natively in K8s Pod Scheduler
3: Observablity Tool Sprawl
• “Its hard connecting observability data to all GitOps tools!”
• DEMO: Unified Observability Data Access within K8s
Agenda: What is OpenTelemetry! And which observability challenges we can solve!

6. • A set of standardized, vendor-agnostic SDKs, APIs, and tools
• Ingests, transforms, and sends data to an Observability back-end
• Makes systems observable through Traces, Metrics, and Logs
• More signals in the works: real user monitoring, profiling …
What is OpenTelemetry and its architecture!
API
SDK
EXPORTER
OTLP
EXPORTER
COLLECTOR
Application
OTLP
Observability
Back-end
Collector
(optional)

7. OpenTelemetry is here to stay …
“By 2025, 70% of new
cloud-native applications
will adopt OpenTelemetry
for observability”,
Gartner Predicts 2022

8. OpenTelemetry’s first use case was Distributed Traces: Traces consist of Spans

9. Distributed Traces can then be analyzed in the backend observability tool / platform
Overview of all traces
Individual Trace
Single Span

10. To learn more about OpenTelemetry: https://opentelemetry.io
https://isitobservable.io/

11. 0: What is OpenTelemetry
1: Unmeasurable Change Impact
• “Its hard to troubleshoot failed GitOps deployments on K8s!”
• DEMO: DORA & Deployment Traces natively from K8s Pod Scheduler
2: Pipelines Gone Wild
• “Its hard adding tests, security, and SLO checks to deployments!”
• DEMO: Deployment Checks Natively in K8s Pod Scheduler
3: Observablity Tool Sprawl
• “Its hard connecting observability data to all GitOps tools!”
• DEMO: Unified Observability Data Access within K8s
Back to the agenda: Observability problems we can solve!

12. Various tools make changes to apps & infra
Platform Teams must build custom tooling to enable
app-aware Observability and traceability
Your teams use any app or infra delivery tool of choice
Keptn emits app-aware deployment lifecycle traces
(OTel) and app-aware DORA metrics (Prometheus)
Challenge #1: Unmeasurable Change Impact Solution #1: App-Aware Change Traceability
Declare Apps as CRDs
Keptn observes deployments app-aware
Business App1
Business App 2
Shared Services Shared Infra
Lifecycle Operator App2
App1 Svc Infra
But K8s lacks the concept of an app
Only knows who deployed change
“Its hard to troubleshoot failed deployments on K8s!” “Keptn: Telemetry for your end-2-end delivery process”

13. My-Application:2.0 **
Frontend-Svc:2.0
Backend-Svc:1.5
Storage-Svc:1.0
Post
Pre
Post
Pre
Post
Pre
Timespan & Result for each single deployment
Pre-App-Deployment
Post-App-Deployment
Timespan Time & Result for whole app deployment
* K8s Operators can leverage K8s webhooks and extend K8s scheduler for pre- and post-deployment hooks
** K8s doesn’t yet have a standard application concept but Delivery SIG is working on it
Observe: Metrics (DORA) & Traces
Some K8s technical details …

14. Demo #1: Deployment Tracing and DORA Metrics

15. Test Security Validate
Your app teams use various delivery tools
DevOps need to code all checks in every tool!
Unavoidable shortcuts increase quality & security risk!
Your app teams use delivery tool of choice
DevOps define pre- and post-deployment tasks once
Checks are always enforced K8s natively!
Challenge #2: Pipelines gone wild! Solution #2: Delivery checks native in K8s
Lifecycle Operator
Deploy straight to K8s
Dev Production Dev Production
Lifecycle Operator
Keptn prevents K8s
from deploying bad changes
No shortcuts possible
“Its hard adding tests, security, and SLO checks to deployments!” “Keptn: Event-driven delivery tasks natively in k8s”

16. My-Application:2.0 **
Frontend-Svc:2.0
Backend-Svc:1.5
Storage-Svc:1.0
Post
Pre
Post
Pre
Post
Pre
Timespan & Result for each single deployment
Pre-App-Deployment
Post-App-Deployment
Timespan Time & Result for whole app deployment
Tasks: Dependency, Env Health,
Certificates, Approval, ...
Evaluations: SLOs, Error Budgets ...
Timespan & result of
each task / evaluation
Tasks: Tests, Security Scans, Cleanups,
Promote ...
Evaluations: SLOs, User Experience,
Adoption ...
Timespan & result of
each task / evaluation
* K8s Operators can leverage K8s webhooks and extend K8s scheduler for pre- and post-deployment hooks
** K8s doesn’t yet have a standard application concept but Delivery SIG is working on it
Observe: Metrics (DORA) & Traces
Orchestrate: Pre-Deploy Orchestrate: Post-Deploy
Some K8s technical details continued …

17. Demo #2: Deployment Checks: Dependencies & SLOs
KLT 0.8 now also supports
custom containers

18. Your app teams use various observability tools
SREs need multiple tool-specific metric configs for
Auto-scaling, release validation, SLOs, alerting …
Metrics Server
Your app teams use observability tool of choice
SREs have a single standard-based configuration for
Auto-scaling, release validation, SLOs, alerting …
Keptn queries metrics from any provider define as CRDs …
… exposes them via K8s Metric API & Prometheus
Challenge #3: Observability Tool Sprawl! Solution #2: One Metric Standard for All!
“Its hard connecting observability data to all GitOps tools!” “Keptn: Unified way to access your observability data”

19. Demo #2: A Single Metric Standard for all K8s tools

20. “How do you connect your observability
data with all your GitOps tools?”
“How do you add tests, security, and SLO
checks to all your k8s deployments?”
“How do you measure success or
troubleshoot failed deployments?”
“Keptn: Unified way to access your
observability data”
“Keptn: Telemetry for your end-2-end
delivery process”
“Keptn: Event-driven delivery tasks
natively in k8s”
To wrap it up: 3 Solutions for 3 Observability challenges!

21. To learn more or try those things out

22. Thank you!
Questions?
Andreas Grabner
CNCF Ambassador, DevRel @ CNCF Keptn
Global DevRelLead @ Dynatrace
Learn more about Keptn: https://lifecycle.keptn.sh