The document outlines a presentation by Sebastian Poxhofer on the state of observability in DevOps and security, focusing on OpenTelemetry as a solution for monitoring complex microservices architectures. It discusses key concepts such as instrumentation, data querying, and the significance of metadata standards to avoid vendor lock-in. The presentation also covers the components of OpenTelemetry, its support across multiple programming languages, and the current status of its implementation.

1. Vienna DevOps & Security Meetup
From front- to backend with OpenTelemetry

2. $ whoami - Sebastian
Sebastian Poxhofer
Software Engineer @ Whizus
Github: @secustor
Open Source Enthusiast & Contributor

3. Agenda
- Context
- Concepts
- Lookout
- Demo
- QA

4. Why should I care?

5. What’s out there? / The state of monitoring
- Microservices lead to a more complex debugging process
- OSS
- Observability pillars are separated
- Proprietary products
- Heavy vendor lock in, especially if SDKs are used
- Public clouds
- Mostly support only their own products
- Limited imports available
- Product changes require system redesigns
- Metadata is not standardized

6. What is observability?
- We need to answer questions about our systems.
What characteristics did the queries that timed out at 500ms share in common? Service
versions? Browser plugins?
- Instrumentation produces data.
- Querying data answers our questions.

7. Origins

8. One protocol to rule them all
The pillars of observability (signals):
- Traces ( GA )
- Metrics ( BETA )
- Logs ( Draft/Experimental )

9. The container for monitoring?

10. But why OpenTelemetry and not Y?
- Semantic conventions for metadata
- Open standard
- Supported by :
- + 300 companies
- + 2,000 contributors
- + 100,000 contributions in the last year
- CNCF ( incubating )
- FAANG ( MAANA? 🤔) + M

11. CONCEPTS

12. Components
- Open Telemetry Protocol ( OTLP )
- Instrumentation libraries
- Auto instrumentation
- Manual instrumentation
- Collector

13. Context propagation
- HTTP/2
- GRPC
- Messaging
- Depends on instrumentation
- Kafka
- RabbitMQ
- ...

14. Instrumentations
Supported Languages:
- Golang
- Java
- JavaScript/TypeScript
- .NET
- Swift
- C++
- Python
- ...
Distributions:
- Core
- Contrib
- Instrumentation

15. The collector pipeline

16. Collector components
Receiver:
- OTLP
- AWS ( xRay, Container
Insights )
- GCP ( Sub & Span)
- Jaeger/Zipkin
- Prometheus
- Filelog
- Kafka
- …
Processor:
- Batch
- MemoryLimiter
- GroupByAttribute
- GroupByTrace
- Filter
- SpanMetrics
- K8Attributes
- ...
Exporter:
- OTLP
- AWS ( xRay, CloudWatch, …)
- GCP
- AzureMonitor
- Jaeger/Zipkin
- Kafka
- Loki
- Splunk
- …

17. Gateway vs Agent deployment
-
Source: OpenTelemetry documentation

18. What’s next?

19. Status
- Like Kubernetes in 2015/16
- Basics are there, advanced stuff needs some work
- Most SDKs GA
- Auto instrumentation available for Java and JS
- Opentelemetry Operator in development
- Java agent injection possible

20. Lookout
- Agent Management Protocol
- eBPF tracing
- .NET auto instrumentation ( ❤ Datadog )

21. Demo

22. Applications

23. Applications

24. Trace flow

25. Metrics flow

26. Log flow

27. Links
- Instrumentations: https://opentelemetry.io/registry/
- Collector
- Receivers:
https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver
- Exporters:
https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter
- Demo: https://github.com/secustor/opentelemetry-meetup

28. We are hiring!
Go to https://whizus.com

29. Q&A