Tracing and telemetry are popular topics right now, but the development is so quick that it also confuses: – Starting with OpenTracing, then W3C Trace-Context, and now OpenTelemetry there are plenty of standards, but what do or don’t they cover? – How do the Cloud Native Computing Foundation (CNCF) and its projects like Jaeger play into that. – Where is OpenTelemetry headed, and how can projects tie into it? This talk gives an overview of standards, projects, and how they all tie together.

1. On the Bleeding Edge of
Philipp Krenn @xeraa

2. Developer
@xeraa

3. Logs
Used by everyone?
Chronological records of events
@xeraa

6. Logs
!
Events
Structure & context
@xeraa

7. {
"@timestamp": "2019-11-20T16:53:01.746Z",
"log.level": "INFO",
"message": "[philipp] failed to log in with password [***]",
"service.name": "gs-securing-web",
"process.thread.name": "http-nio-8080-exec-8",
"log.logger": "hello.AuthenticationEventListener",
"labels.event.category": "LOGIN_FAILURE",
"labels.user.name": "philipp",
"labels.source.ip": "0:0:0:0:0:0:0:1",
"labels.url.full": "/login"
}
@xeraa

8. Metrics
Used by most?
Periodic measurements of
numeric KPIs
@xeraa

11. Uptime
Used by many?
Synthetic / (pro)active
monitoring
@xeraa

14. Traces
Used by some?
Context for requests
@xeraa

17. Traces
Application Performance
Monitoring
Distributed Tracing
@xeraa

18. ...the monitoring and
management of performance
and availability of software
applications.
— Wikipedia
@xeraa

19. Simpler
Times
https:/
/www.kartar.net/2019/07/intro-to-distributed-tracing/
@xeraa

20. Better
Times for
Vendors
https:/
/www.kartar.net/2019/07/intro-to-distributed-tracing/
@xeraa

21. Goals
Transaction context
Reconstruct flow
Query & visualize transactions
@xeraa

22. Terminology
Trace: record of a request, DAG
Span: named & timed single
operation, optionally nested
@xeraa

23. Terminology
Span Context: IDing a request
Context Propagation: Attaching
IDs
@xeraa

24. Problem
uber-trace-id:
118c6c15301b9b3b3:56e66177e6e55a91:
18c6c15301b9b3b3:1
elastic-apm-traceparent: 00-
f109f092a7d869fb4615784bacefcfd7-
5bf936f4fcde3af0-01

25. OpenTracing (OT)
Vendor-neutral APIs for tracing
@xeraa

26. Trace
@xeraa

27. SpanContext
SpanContext:
- trace_id:"abc123"
- span_id:"xyz789"
- Baggage Items:
- special_id:"vsid1738"
@xeraa

28. API / Headers Only
No wire protocol
(in- or out-band)
@xeraa

29. W3C Trace Context
Pass trace context information
across systems
@xeraa

30. Trace Context Propagation
traceparent:
00- // Version
0af7651916cd43dd8448eb211c80319c- // Trace ID
b7ad6b7169203331- // Parent ID
01 // Sampling flags
CorrelationContext / baggage supported
@xeraa

31. OpenTelemetry
OSS & CNCF
Traces, metrics, logs
@xeraa

32. OTel
Unify instrumentation & pillars
Supersede OpenTracing +
OpenCensus
@xeraa

33. OTel
Vendor neutral but wide support
Google, Splunk, Lightstep, Honeycomb,
Elastic,...
@xeraa

34. Status
Tracing: stable
Metrics: feature-freeze / experimental
Logs: draft
https:/
/opentelemetry.io/status/
@xeraa

35. 3 Ways to OTel
Integration
Vendors can implement 1+
@xeraa

39. Elastic's
OpenTelemetry
Integration
@xeraa

41. Demo
https:/
/github.com/open-telemetry/
opentelemetry-collector-contrib
https:/
/github.com/open-telemetry/
opentelemetry-java-instrumentation
https:/
/github.com/cyrille-leclerc/
my-shopping-cart/tree/open-telemetry
@xeraa

42. Agents
Live in / attach to app process
public static void main
premain
agentmain
https:/
/www.javaadvent.com/2019/12/a-beginners-guide-to-java-agents.html
@xeraa

43. Java Agent Setup
export OTEL_RESOURCE_ATTRIBUTES=service.name=frontend,
service.version=1.0
deployment.environment=production
export OTEL_EXPORTER_OTLP_ENDPOINT=https://apm_server_url:8200
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <secret_token>"
java -javaagent:/path/to/opentelemetry-javaagent-all.jar
-classpath lib/*:classes/
com.mycompany.checkout.CheckoutServiceServer
@xeraa

45. Manual Instrumentation
Tracer tracer =
OpenTelemetry.getGlobalTracer("instrumentation-library-name", "1.0.0");
Span span = tracer.spanBuilder("my span").startSpan();
try (Scope scope = span.makeCurrent()) {
// your use case
...
} catch (Throwable t) {
span.setStatus(StatusCode.ERROR, "Change it to your error message");
} finally {
span.end(); // closing the scope does not end the span; has to be done manually
}
@xeraa

46. Options?
Features: Elastic APM
Central config management
Error reports
Profiling capabilities
@xeraa

47. Options?
Standardization: OpenTracing
APIs + Elastic agent
@xeraa

48. Options?
Migration from Jaeger: Jaeger
protocol in Elastic's APM server
@xeraa

49. Options?
OpenTelemetry as the new
standard
Metrics & logs less mature
@xeraa

50. Conclusion
@xeraa

51. Distributed Tracing
Why & how
@xeraa

53. Available
Today & free
Cloud & on-prem
Traces + logs, metrics, uptime
@xeraa

54. On the Bleeding Edge of
Philipp Krenn @xeraa