What is Distributed Tracing, and Why Should you care About it?
•
0 likes•118 views
Distributed tracing allows services in a distributed system to propagate contextual information about a request as it travels across services. It works by assigning a unique ID to each transaction and propagating that ID across services as the transaction executes. This allows the path of a request to be tracked across the distributed system. The benefits of distributed tracing include reduced mean time to resolution for issues, finding bugs between releases, and improved team collaboration. Some challenges with distributed tracing are picking an open source solution, balancing black box vs white box instrumentation, and handling high volumes of traces.
Recommended
Recommended
More Related Content
Similar to What is Distributed Tracing, and Why Should you care About it?
Similar to What is Distributed Tracing, and Why Should you care About it? (20)
More from All Things Open
More from All Things Open (20)
Recently uploaded
Recently uploaded (20)
What is Distributed Tracing, and Why Should you care About it?
- 1. What is Distributed Tracing, and Why Should You Care About it? Ricardo Ferreira Principal Developer Advocate Elastic @riferrei
- 2. @riferrei Criminal Investigation TV Shows and MOvies
- 3. @riferrei Detective TV shows and movies • Collect evidence • Searching data • Interrogation • Creating timelines • Forensic science • Motive probing • Human Psychology • Build strong case @riferrei
- 4. @riferrei Agenda • What is distributed tracing? • How distributed tracing works? • Benefits of distributed tracing • Challenges: it ain't all flowers @riferrei
- 5. • Principal Developer 🥑 at Elastic • Community, Developer Relations • Before Elastic ➡ Confluent, Oracle, and Red HAT (JBoss) • Distributed Systems, databases, observability, streaming systems • https://riferrei.com Who am I? @riferrei
- 7. @riferrei for starters: it is nothing new! type Log struct { request_path string request_size int64 status int32 latency_ms float64 } logEntry := Log{ "/customers/find", 840, 200, 35 } fmt.Printf("%+v", logEntry) Each log statement has its own “schema”
- 8. @riferrei Follow the thread troubleshooting API Customer Database Thread 1 API Customer Database Thread 2 Log{"/api/find", 840, 200, 35} Log{"/customer/find", 235, 200, 30} Log{"/db/find", 450, 200, 5} Log{"/api/find", 840, 200, 45} Log{"/customer/find", 235, 200, 42} Log{"/db/find", 450, 200, 3}
- 9. @riferrei Stitching the threads manually logEntry := CustomLog{ requestPath(), requestSize(), requestStatus(), latencyMS(), cust.customerID(), db.databaseTenantID() } type CustomLog struct { request_path string request_size int64 status int32 latency_ms float64 customer_id int64 database_id int64 }
- 10. @riferrei begin of chaos: distributed computing Thread 1 Host 1 Thread 2 Thread 1 Host 2 Thread 2
- 11. @riferrei Using Virtualization Host 1 VM 1 VM 2 API Customer Database Thread 1 API Customer Database Thread 2 API Customer Database Thread 1 API Customer Database Thread 2
- 12. @riferrei Using containerization Host 1 VM 1 VM 2 Container 1 Container 2 API Customer Database Thread 1 API Customer Database Thread 2 API Customer Database Thread 1 API Customer Database Thread 2
- 13. @riferrei Is this a Joke to you? @riferrei Ops: “Let’s now break down the services into functions”
- 14. @riferrei Tracing automates system-wide stitching Service A Service B Service C Service D Transaction Transaction data is collected and becomes searchable ready
- 15. @riferrei Detective TV shows and movies • Collect evidence • Searching data • Interrogation • Creating timelines • Forensic science • Motive probing • Human Psychology • Build strong case @riferrei
- 17. @riferrei @riferrei It is all about setting the context “Watcha talkin’ about willis?”
- 18. @riferrei Tracing, spans, and context propagation Service A (Child Span) Service B (Child Span) Service C (Child Span) Service D (Child Span) Transaction (Root Span) Trace ID: 12345 Trace ID: 12345 Trace ID: 12345 Trace ID: 12345 Trace ID: 12345 ⬅ This is the context! Time: 55ms Time: 30ms Time: 15ms Time: 5ms Time: 5ms
- 19. @riferrei Capture, process, store, repeat Data Store Tracing Pipeline Service A (Child Span) Time: 30ms Service B (Child Span) Time: 15ms Service C (Child Span) Time: 5ms Service D (Child Span) Time: 5ms
- 20. @riferrei One heck of a data store 💡 Logs Metrics Traces Data Store
- 21. @riferrei Detective TV shows and movies • Collect evidence • Searching data • Interrogation • Creating timelines • Forensic science • Motive probing • Human Psychology • Build strong case @riferrei
- 23. @riferrei Reduced Mean time to resolution (MTTR) Suspect Drill Down Solve • Watching metric values • Caught up with alerts • Bringing people onboard • Understand topologies • Isolate the anomalies • Collect contextual data • Read logs and events • Create code patches • Create a new release 10% 60% 30%
- 24. @riferrei Finding Bugs between releases V1.12 V1.13
- 26. @riferrei Detective TV shows and movies • Collect evidence • Searching data • Interrogation • Creating timelines • Forensic science • Motive probing • Human Psychology • Build strong case @riferrei
- 27. @riferrei Challenges: because it ain't all flowers
- 28. @riferrei Picking open-source is not always Easy Agent for my Programming language? Data Store Scalability? Frameworks versus Libraries? Does my Architecture Fit?
- 29. @riferrei Black-Box White-Box • Code is not changed • Handled by the runtime • Minimal execution visibility • Require code changes • Handled by the application • Full Execution visibility Black-Box versus White-Box tracing
- 30. @riferrei Client-side tracing is on its early days
- 31. @riferrei Sampling everything comes at a price Tracing Pipeline ? ?
- 33. @riferrei Useful Resources ü Join #openTelemetry on https://slack.cncf.io ü https://github.com/riferrei/otel-with-java ü Book ➡ distributed tracing in practice ✅ @riferrei