Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
container-solutions.com | @containersoluti | info@container-solutions.com
Distributed Tracing
with ZipKin &
Kubernetes
Max...
container-solutions.com | @containersoluti | info@container-solutions.com
Microservices...
In short, the microservice arch...
www.container-solutions.com | info@container-solutions.com
The “Socks Shop”...
microservices-demo.github.io
www.container-solutions.com | info@container-solutions.com
The “Socks Shop”...
www.container-solutions.com | info@container-solutions.com
Let’s install it in the cluster...
Distributed Tracing | container-solutions.com
Microservice benefits
● Modeling after business domains
● Independent deploy...
Distributed Tracing | container-solutions.com
Microservice costs
● Distribution
● Eventual Consistency
● Operational compl...
Distributed Tracing | container-solutions.com
Microservice requirements
● Rapid provisioning
● Monitoring
● Rapid deployme...
Distributed Tracing | container-solutions.com
Microservice architectures
● Monolithic to microservice architecture
● Apps ...
Distributed Tracing | container-solutions.com
Why distributed tracing?
Example: Google search query
● Multiple index looku...
Distributed Tracing | container-solutions.com
Why distributed tracing?
“Per-process logging and metric monitoring have the...
Distributed Tracing | container-solutions.com
Fundamental requirements to make it work
● Ubiquitous deployment
● Continuou...
Distributed Tracing | container-solutions.com
Requirements to make is useful
● Low overhead
● Application-level transparen...
Distributed Tracing | container-solutions.com
A distributed trace...
“A tracing infrastructure for distributed
services ne...
Distributed Tracing | container-solutions.com
Data aggregation
Message record:
Record = Message identifier + timestamped e...
Distributed Tracing | container-solutions.com
● Trace as a tree of nested calls
● Trace trees and spans
Trace data model
www.container-solutions.com | info@container-solutions.com
Span
Logged event in a typical span
● Span name
● Span start ti...
www.container-solutions.com | info@container-solutions.com
OpenTracing & ZipKin
Common libraries for several programming l...
www.container-solutions.com | info@container-solutions.com
Let’s install that also in the cluster...
www.container-solutions.com | info@container-solutions.com
Supported languages
➔ Javascript
➔ Python
➔ Java
➔ Scala
➔ Ruby...
www.container-solutions.com | info@container-solutions.com
Supported frameworks
➔ Express (nodejs - http)
➔ Jersey, RestEa...
www.container-solutions.com | info@container-solutions.com
Opentracing Example (Go)
explicitely instrumenting a SQL query ...
www.container-solutions.com | info@container-solutions.com
Annotations
● Arbitrary text
● Key/Value pairs
➔Can be used for...
www.container-solutions.com | info@container-solutions.com
Architecture (ZipKin with Scribe + Cassandra)
www.container-solutions.com | info@container-solutions.com
Performance
Low overhead is the key!
Sampling is the solution!
...
www.container-solutions.com | info@container-solutions.com
Sampling
➔ 2-stage sampling:
a. Client: Don’t send every trace ...
www.container-solutions.com | info@container-solutions.com
But what about...
● Proprietary services?
● Ancient/Legacy Serv...
www.container-solutions.com | info@container-solutions.com
But what about...
● Proprietary services?
● Ancient/Legacy Serv...
www.container-solutions.com | info@container-solutions.com
Linkerd overview
● Intelligent, adaptive load-balancing
● Globa...
www.container-solutions.com | info@container-solutions.com
Some of the answered questions...
...with a distributed tracing...
www.container-solutions.com | info@container-solutions.com
A word of caution about distributed tracing
● Documentation is ...
www.container-solutions.com | info@container-solutions.com
A word of caution about distributed tracing
● Documentation is ...
www.container-solutions.com | info@container-solutions.com
Demo time...
www.container-solutions.com | info@container-solutions.com
www.container-solutions.com | info@container-solutions.com
Questions? Want to learn more?
● Come to our 2 day tinyurl.com/...
Upcoming SlideShare
Loading in …5
×

Distributed Tracing with OpenTracing, ZipKin and Kubernetes

8,453 views

Published on

Presented at the Microservice Meetup Zürich - October 20, 2016.

Published in: Technology
  • Dating direct: ❶❶❶ http://bit.ly/2u6xbL5 ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Follow the link, new dating source: ❶❶❶ http://bit.ly/2u6xbL5 ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/qm3rdqy } ......................................................................................................................... Download Full EPUB Ebook here { https://tinyurl.com/qm3rdqy } ......................................................................................................................... Download Full doc Ebook here { https://tinyurl.com/qm3rdqy } ......................................................................................................................... Download PDF EBOOK here { https://tinyurl.com/qm3rdqy } ......................................................................................................................... Download EPUB Ebook here { https://tinyurl.com/qm3rdqy } ......................................................................................................................... Download doc Ebook here { https://tinyurl.com/qm3rdqy } ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/qm3rdqy } ......................................................................................................................... Download Full EPUB Ebook here { https://tinyurl.com/qm3rdqy } ......................................................................................................................... Download Full doc Ebook here { https://tinyurl.com/qm3rdqy } ......................................................................................................................... Download PDF EBOOK here { https://tinyurl.com/qm3rdqy } ......................................................................................................................... Download EPUB Ebook here { https://tinyurl.com/qm3rdqy } ......................................................................................................................... Download doc Ebook here { https://tinyurl.com/qm3rdqy } ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Distributed Tracing with OpenTracing, ZipKin and Kubernetes

  1. 1. container-solutions.com | @containersoluti | info@container-solutions.com Distributed Tracing with ZipKin & Kubernetes Maximilian Schöfmann @schoefmann Container Solutions AG @containersoluti
  2. 2. container-solutions.com | @containersoluti | info@container-solutions.com Microservices... In short, the microservice architectural style is an approach to develop a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies. -- James Lewis and Martin Fowler
  3. 3. www.container-solutions.com | info@container-solutions.com The “Socks Shop”... microservices-demo.github.io
  4. 4. www.container-solutions.com | info@container-solutions.com The “Socks Shop”...
  5. 5. www.container-solutions.com | info@container-solutions.com Let’s install it in the cluster...
  6. 6. Distributed Tracing | container-solutions.com Microservice benefits ● Modeling after business domains ● Independent deployment ● Technology diversity
  7. 7. Distributed Tracing | container-solutions.com Microservice costs ● Distribution ● Eventual Consistency ● Operational complexity
  8. 8. Distributed Tracing | container-solutions.com Microservice requirements ● Rapid provisioning ● Monitoring ● Rapid deployment ● Autonomous teams
  9. 9. Distributed Tracing | container-solutions.com Microservice architectures ● Monolithic to microservice architecture ● Apps as a collection of distributed services ● Tools becoming necessary to gather metrics
  10. 10. Distributed Tracing | container-solutions.com Why distributed tracing? Example: Google search query ● Multiple index lookups ● Selecting Ads ● Check spelling ● Personalise results ● Filter DMCA takedowns ● Include relevant images... ● ...and videos ● ...and news ● ...
  11. 11. Distributed Tracing | container-solutions.com Why distributed tracing? “Per-process logging and metric monitoring have their place, but neither can reconstruct the elaborate journeys that transactions take as they propagate across a distributed system. Distributed traces are these journeys.” -- Chris Aniszczyk, Cloud Native Computing Foundation
  12. 12. Distributed Tracing | container-solutions.com Fundamental requirements to make it work ● Ubiquitous deployment ● Continuous monitoring See also: “Dapper, a Large-Scale Distributed Systems Tracing Infrastructure” http://research.google.com/pubs/pub36356.html (2010)
  13. 13. Distributed Tracing | container-solutions.com Requirements to make is useful ● Low overhead ● Application-level transparency ● Scalability ● (Timely) data availability
  14. 14. Distributed Tracing | container-solutions.com A distributed trace... “A tracing infrastructure for distributed services needs to record information about all the work done in a system, on behalf of a given initiator”
  15. 15. Distributed Tracing | container-solutions.com Data aggregation Message record: Record = Message identifier + timestamped event Data aggregation classes: ● Black box ● Annotation-based
  16. 16. Distributed Tracing | container-solutions.com ● Trace as a tree of nested calls ● Trace trees and spans Trace data model
  17. 17. www.container-solutions.com | info@container-solutions.com Span Logged event in a typical span ● Span name ● Span start time ● Span end time ● Trace id ● Span id ● Span parent id ● Any timing information recorded by the instrumentation library (RPC, HTTP) ● Additional custom labels (“foo”)
  18. 18. www.container-solutions.com | info@container-solutions.com OpenTracing & ZipKin Common libraries for several programming languages ➔ Libraries attach a trace context to the thread local storage ➔ RPC friendly (specially when using gRPC) ➔ The data is language-independent opentracing.io zipkin.io
  19. 19. www.container-solutions.com | info@container-solutions.com Let’s install that also in the cluster...
  20. 20. www.container-solutions.com | info@container-solutions.com Supported languages ➔ Javascript ➔ Python ➔ Java ➔ Scala ➔ Ruby ➔ C# ➔ Golang
  21. 21. www.container-solutions.com | info@container-solutions.com Supported frameworks ➔ Express (nodejs - http) ➔ Jersey, RestEasy, JAXRS2, Apache HttpClient, Mysql (Java, HTTP, gRPC) ➔ HDFS, HBASE ➔ Spring, Spring Cloud ➔ Apache Cassandra ➔ Finagle ➔ Rack ➔ Golang Context ➔ GoKit ➔ Akka, Spray, Play ➔ Dropwizard ➔ Roll your own
  22. 22. www.container-solutions.com | info@container-solutions.com Opentracing Example (Go) explicitely instrumenting a SQL query in a service written in Go
  23. 23. www.container-solutions.com | info@container-solutions.com Annotations ● Arbitrary text ● Key/Value pairs ➔Can be used for common vocabulary, e.g. “http.status_code”, “peer.service”, “sampling.priority”
  24. 24. www.container-solutions.com | info@container-solutions.com Architecture (ZipKin with Scribe + Cassandra)
  25. 25. www.container-solutions.com | info@container-solutions.com Performance Low overhead is the key! Sampling is the solution! … at least partially...
  26. 26. www.container-solutions.com | info@container-solutions.com Sampling ➔ 2-stage sampling: a. Client: Don’t send every trace instrumented ● limits client-side CPU and bandwidth overhead ● adjustable per service, hard to change in one go b. Server: Don’t persist every trace received ● limits server-side IO and data volume overhead ● adjustable centrally with simple config change ➔ Adaptive sampling to trade off overhead against missing relevant traces
  27. 27. www.container-solutions.com | info@container-solutions.com But what about... ● Proprietary services? ● Ancient/Legacy Services? ● 3rd-Party services outside your control?
  28. 28. www.container-solutions.com | info@container-solutions.com But what about... ● Proprietary services? ● Ancient/Legacy Services? ● 3rd-Party services outside your control? Proxying!
  29. 29. www.container-solutions.com | info@container-solutions.com Linkerd overview ● Intelligent, adaptive load-balancing ● Global, fine-grained instrumentation ● Application-centric naming ● Powerful traffic routing mechanisms
  30. 30. www.container-solutions.com | info@container-solutions.com Some of the answered questions... ...with a distributed tracing system are: ● Which parts of my system are slow? ● Which call pattern can be optimized with parallelization? ● Which calls are redundant? ● Which routes are affected by this failing part? ● Under which circumstances is it failing? ● How often is it failing? ● Detect queries issued to read and write masters, instead of read only replicas
  31. 31. www.container-solutions.com | info@container-solutions.com A word of caution about distributed tracing ● Documentation is still rather poor ● Yet another moving part ● Can accumulate huge amounts of data ● Metrics need to be interpreted ● Commercial APM solutions might be an easier route for your use case...
  32. 32. www.container-solutions.com | info@container-solutions.com A word of caution about distributed tracing ● Documentation is still rather poor ● Yet another moving part ● Can accumulate huge amounts of data ● Metrics need to be interpreted ● Commercial APM solutions might be an easier route for your use case...
  33. 33. www.container-solutions.com | info@container-solutions.com Demo time...
  34. 34. www.container-solutions.com | info@container-solutions.com
  35. 35. www.container-solutions.com | info@container-solutions.com Questions? Want to learn more? ● Come to our 2 day tinyurl.com/microservice-workshop (November 8. + 9. or at your company on request) ● Follow us on Twitter: @containersoluti ● Read more on our blog: container-solutions.com/blog ● Or just get in touch: info@container-solutions.com

×