A POC of distributed tracing I did years ago with Jaeger

1. Distributed tracing: State of affairs
Puneeth Nanjundaswamy,
Sr. Infrastructure Engineer

2. Microservices vs Monoliths

3. Microservices: Observability

4. Microservices at Uber

5. Ops hell

6. Observability in microservices
➢ Logging
➢ Application metrics
➢ Distributed tracing

7. Distributed tracing

8. Distributed tracing
➢ Aims to stitch metrics and logs together
➢ Traces a request through the system through all the
services
➢ Corresponding logs messages
➢ Corresponding metrics

9. Distributed tracing: advantages
➢ Distributed context propagation
➢ Distributed transaction monitoring
➢ Root cause analysis
➢ Service dependency analysis
➢ Performance / latency optimization

10. What is a trace?
Source: https:/
/www.jaegertracing.io/

11. What is a span?
Source: https:/
/opentracing.io/

12. What is a trace?

13. Distributed tracing

14. What is a span?

15. Tracer
➢ Start a span
➢ Inject span context
➢ Extract span context
➢ Set span tags
➢ Log structured data
➢ Set baggage item
➢ Finish a span

16. Auto instrumentation

17. Opentracing

18. Opentracing
The Opentracing API project is working towards creating more
standardized APIs and instrumentation for distributed tracing.

19. Opentracing compatible tracers
➢ CNCF Jaeger
➢ Appdash
➢ LightStep
➢ Instana
➢ Apache SkyWalking
➢ inspectIT
➢ stagemonitor
➢ Datadog

20. Jaeger tracer

21. Jaeger tracer: architecture
Source: https:/
/www.jaegertracing.io/

22. The good: Jaeger
➢ Clear opensource winner
➢ Adopted by CNCF
➢ Super fast and very negligible resource footprint for agent and
collector.
➢ Actively developed
➢ Backed by Uber

23. The bad: Jaeger
Not all features supported in all the clients

24. The ugly: Opentracing auto instrumentation
➢ No well maintained auto-instrumentation available for Node.js
➢ Opentracing-auto fork at Comtravo
➢ Python instrumentation libraries
➢ Node.js instrumentation libraries

25. Sampling can be controlled via Terraform
container_definition = <<DEFINITION
.
.
.
],
"environment" : [
{ "name" : "ENABLE_TRACING", "value" : "true" },
{ "name" : "CT_TRACER_SAMPLER_PARAM", "value" : "${var.production_env ? "0.01" : "0.1"}" },
{ "name" : "NODE_ENV", "value" : "production" },
.
.
.
DEFINITION

26. Opencensus

27. Opencensus
➢ Google’s “standard”
➢ Open source version of their census library
➢ Tracing and instrumentation both packaged and shipped
➢ Available features depend on the programming language
➢ Actively maintained
➢ Zpages
➢ Works with Jaeger storage and UI components

28. DEMO

29. FIN