SlideShare a Scribd company logo
1 of 66
Z I P K I N
A distributed tracing framework
Why Zipkin?
Google



0.5 sec slower



20% traffic drop


                  @skr | @thisisfranklin   3
Google



0.5 sec slower    1.5 sec faster



20% traffic drop   CTR up 12%


                               @skr | @thisisfranklin   3
Performance matters
Front end




            @skr | @thisisfranklin   5
Front end   Back end




                   @skr | @thisisfranklin   5
@skr | @thisisfranklin   6
@skr | @thisisfranklin   6
@skr | @thisisfranklin   7
@skr | @thisisfranklin   8
Picture from http://www.flickr.com/photos/jpellgen
@skr | @thisisfranklin   9
• Collects traces from production requests
• Low overhead
• Minimum of extra work for developers




                                             @skr | @thisisfranklin   9
Finagle
“Finagle is an asynchronous network stack for the JVM that you can use to build
asynchronous Remote Procedure Call (RPC) clients and servers in Java, Scala, or
any JVM-hosted language.”


                         github.com/twitter/finagle
What to capture?
@skr | @thisisfranklin   12
Finagle Http service




                       @skr | @thisisfranklin   12
Finagle Http service




                       @skr | @thisisfranklin   12
Finagle Http service



Finagle Thrift Service




                                          @skr | @thisisfranklin   12
Finagle Http service



Finagle Thrift Service




                                          @skr | @thisisfranklin   12
Finagle Http service



Finagle Thrift Service




                                          @skr | @thisisfranklin   12
Finagle Http service



Finagle Thrift Service                Service




                                                @skr | @thisisfranklin   12
Finagle Http service



Finagle Thrift Service                Service




                                                @skr | @thisisfranklin   12
Finagle Http service



Finagle Thrift Service                Service




                                                @skr | @thisisfranklin   12
Zipkin terminology
‣   Annotation: string data associated with a particular timestamp, service, and
    host


                                        Time



         time: 2012-01-21 22:37:01
         value: “something happened”
         server: 135.34.53.2
         service: “timelineservice”




                                                                           @skr | @thisisfranklin   13
‣   Span: represents one specific method call; made up of a set of annotations.
    Has a name and an id.
                                        Time

         T:0ms Client Send



                                        Span




                                                                           @skr | @thisisfranklin   14
‣   Span: represents one specific method call; made up of a set of annotations.
    Has a name and an id.
                                        Time

         T:0ms Client Send



                                        Span


             T:10ms Server Receive




                                                                           @skr | @thisisfranklin   14
‣   Span: represents one specific method call; made up of a set of annotations.
    Has a name and an id.
                                        Time

         T:0ms Client Send



                                        Span


             T:10ms Server Receive                        T:90ms Server Send




                                                                               @skr | @thisisfranklin   14
‣   Span: represents one specific method call; made up of a set of annotations.
    Has a name and an id.
                                        Time

         T:0ms Client Send                               T:100ms Client Receive



                                        Span


             T:10ms Server Receive                        T:90ms Server Send




                                                                                  @skr | @thisisfranklin   14
‣   Span: represents one specific method call; made up of a set of annotations.
    Has a name and an id.
                                               Time

         T:0ms Client Send                                            T:100ms Client Receive



                                               Span

                                     T:20ms Read 30 kbytes from file
             T:10ms Server Receive                                     T:90ms Server Send




                                                                                               @skr | @thisisfranklin   14
‣   Span: represents one specific method call; made up of a set of annotations.
    Has a name and an id.
                                               Time

         T:0ms Client Send                                            T:100ms Client Receive



                                               Span

                                     T:20ms Read 30 kbytes from file
             T:10ms Server Receive                                     T:90ms Server Send



‣   Trace: a set of spans all associated with the same request


                                                                                               @skr | @thisisfranklin   14
Finagle
 http
service




          @skr | @thisisfranklin   15
• Generate a random i64 trace id
                                   Finagle
                                    http
                                   service




                                             @skr | @thisisfranklin   15
• Generate a random i64 trace id
• Decide if we should sample the
trace or not                       Finagle
                                    http
                                   service




                                             @skr | @thisisfranklin   15
• Generate a random i64 trace id
• Decide if we should sample the
trace or not                       Finagle
                                    http
                                   service




                                   Finagle
                                    thrift
                                   service



                                             @skr | @thisisfranklin   15
• Generate a random i64 trace id
• Decide if we should sample the
trace or not                       Finagle
                                    http
                                   service

• Generate new span id


                                   Finagle
                                    thrift
                                   service



                                             @skr | @thisisfranklin   15
• Generate a random i64 trace id
• Decide if we should sample the
trace or not                       Finagle
                                    http
                                   service

• Generate new span id
• Pass trace header

                                   Finagle
                                    thrift
                                   service



                                             @skr | @thisisfranklin   15
• Generate a random i64 trace id
• Decide if we should sample the
trace or not                       Finagle
                                    http     struct RequestHeader {
                                   service     i64 trace_id,
                                               i64 span_id,
• Generate new span id                         optional i64 parent_span_id,
• Pass trace header
                                               optional bool sampled
                                             }
                                   Finagle
                                    thrift
                                   service



                                                                 @skr | @thisisfranklin   15
• Generate a random i64 trace id
• Decide if we should sample the
trace or not                            Finagle
                                         http     struct RequestHeader {
                                        service     i64 trace_id,
                                                    i64 span_id,
• Generate new span id                              optional i64 parent_span_id,
• Pass trace header
                                                    optional bool sampled
                                                  }
                                        Finagle
• Thrift service adopts trace id from    thrift
header if it exists                     service



                                                                      @skr | @thisisfranklin   15
Finagle
 http
service




Finagle
 thrift
service




          @skr | @thisisfranklin   15
Finagle
           http
          service




Finagle   Finagle
 thrift    thrift
service   service




                    @skr | @thisisfranklin   15
Finagle
           http
          service




Finagle   Finagle
 thrift    thrift
service   service



                    Finagle
                     thrift
                    service

                              @skr | @thisisfranklin   15
Finagle
            http
          service
          S




Finagle   Finagle
 thrift    thrift
S
service   service
          S



                    Finagle
                     thrift
                    S
                    service

                              @skr | @thisisfranklin   15
Finagle
            http
          service
          S

                               Zipkin
                              collector


Finagle   Finagle
 thrift    thrift
S
service   service
          S
                              Cassandra


                    Finagle
                     thrift
                    S
                    service

                                          @skr | @thisisfranklin   15
Finagle
            http
          service
          S

                               Zipkin
                              collector


Finagle   Finagle
 thrift    thrift
S
service   service
          S                               Zipkin        Zipkin
                              Cassandra
                                          Query           UI

                    Finagle
                     thrift
                    S
                    service

                                                   @skr | @thisisfranklin   15
Finagle ♥ Zipkin
Finagle Thrift server

       ServerBuilder()
        .bindTo(address)
        .codec(ThriftServerFramedCodec())
        .name("servicename")
        .build(someService)




                                            @skr | @thisisfranklin   17
Finagle Thrift server

       ServerBuilder()
        .bindTo(address)
        .codec(ThriftServerFramedCodec())
        .name("servicename")
        .build(someService)




                                            @skr | @thisisfranklin   17
Finagle Thrift server

       ServerBuilder()
        .bindTo(address)
        .codec(ThriftServerFramedCodec())
        .name("servicename")
        .build(someService)




                                            @skr | @thisisfranklin   17
Finagle Thrift server

       ServerBuilder()
        .bindTo(address)
        .codec(ThriftServerFramedCodec())
        .name("servicename")
        .build(someService)




                                            @skr | @thisisfranklin   17
ServerBuilder()
 .bindTo(address)
 .codec(ThriftServerFramedCodec())
 .name("servicename")
 .tracerFactory(ZipkinTracer())
 .build(someService)




                                     @skr | @thisisfranklin   18
ClientBuilder()
 .cluster(hosts)
 .codec(ThriftClientFramedCodec())
 .name("clientname")
 .tracerFactory(ZipkinTracer())
 .build(someService)




                                     @skr | @thisisfranklin   19
Trace.record("doing stuff")




                              @skr | @thisisfranklin   20
Trace.record("doing stuff")




                              @skr | @thisisfranklin   20
Trace.record("doing stuff")



   time: 2012-01-21 22:37:01
   value: “doing stuff”
   server: 135.34.53.2
   service: “timelineservice”




                                @skr | @thisisfranklin   20
Trace.recordBinary("key", data)




                                  @skr | @thisisfranklin   21
Key           Value


Trace.recordBinary("key", data)   responsecode       500


                                  cache:somekey      Hit


                                    sql.query     select *...



                                                      @skr | @thisisfranklin   21
Platform    Protocol   Client   Server
 Finagle     Thrift     Yes      Yes
 Finagle     HTTP       Yes      Yes
 Finagle    Memcache    Yes      No
 Finagle     Redis      Yes      No
 Cassie      Thrift     Yes      No
Querulous    JDBC       Yes      No
  Ruby       Thrift     Yes      Yes
                                       @skr | @thisisfranklin   22
Zipkin UI
@skr | @thisisfranklin   24
@skr | @thisisfranklin   24
@skr | @thisisfranklin   24
What did we find?
github.com/twitter/zipkin
     @zipkinproject
                            @skr | @thisisfranklin   27

More Related Content

Viewers also liked

Distributed tracing - get a grasp on your production
Distributed tracing - get a grasp on your productionDistributed tracing - get a grasp on your production
Distributed tracing - get a grasp on your productionnklmish
 
Implementing microservices tracing with spring cloud and zipkin (spring one)
Implementing microservices tracing with spring cloud and zipkin (spring one)Implementing microservices tracing with spring cloud and zipkin (spring one)
Implementing microservices tracing with spring cloud and zipkin (spring one)Reshmi Krishna
 
Advances in Cassandra Tracing with Zipkin (Michael Semb Wever, The Last Pickl...
Advances in Cassandra Tracing with Zipkin (Michael Semb Wever, The Last Pickl...Advances in Cassandra Tracing with Zipkin (Michael Semb Wever, The Last Pickl...
Advances in Cassandra Tracing with Zipkin (Michael Semb Wever, The Last Pickl...DataStax
 
Apache Htrace overview (20160520)
Apache Htrace overview (20160520)Apache Htrace overview (20160520)
Apache Htrace overview (20160520)Steve Min
 
Openzipkin conf: Zipkin at Yelp
Openzipkin conf: Zipkin at YelpOpenzipkin conf: Zipkin at Yelp
Openzipkin conf: Zipkin at YelpPrateek Agarwal
 
Guidelines For The Animation Of J P I C
Guidelines For The Animation Of  J P I CGuidelines For The Animation Of  J P I C
Guidelines For The Animation Of J P I CchitoA
 
Securing Your Deployment Pipeline With Docker
Securing Your Deployment Pipeline With DockerSecuring Your Deployment Pipeline With Docker
Securing Your Deployment Pipeline With DockerContainer Solutions
 
Troubleshooting RabbitMQ and services that use it
Troubleshooting RabbitMQ and services that use itTroubleshooting RabbitMQ and services that use it
Troubleshooting RabbitMQ and services that use itMichael Klishin
 
Make 2016 your year of SMACK talk
Make 2016 your year of SMACK talkMake 2016 your year of SMACK talk
Make 2016 your year of SMACK talkDataStax Academy
 
IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)
IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)
IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)Red Hat Developers
 
Microservices architecture examples
Microservices architecture examplesMicroservices architecture examples
Microservices architecture examplesChanny Yun
 
Tracing Microservices with Zipkin
Tracing Microservices with ZipkinTracing Microservices with Zipkin
Tracing Microservices with Zipkintakezoe
 
Monitoring Microservices
Monitoring MicroservicesMonitoring Microservices
Monitoring MicroservicesWeaveworks
 
Technical Seminar PPT
Technical Seminar PPTTechnical Seminar PPT
Technical Seminar PPTKshitiz_Vj
 
Dockercon State of the Art in Microservices
Dockercon State of the Art in MicroservicesDockercon State of the Art in Microservices
Dockercon State of the Art in MicroservicesAdrian Cockcroft
 
Advantages and Disadvantages of Technology
Advantages and Disadvantages of TechnologyAdvantages and Disadvantages of Technology
Advantages and Disadvantages of TechnologyPave Maris Cortez
 
What is tackled in the Java EE Security API (Java EE 8)
What is tackled in the Java EE Security API (Java EE 8)What is tackled in the Java EE Security API (Java EE 8)
What is tackled in the Java EE Security API (Java EE 8)Rudy De Busscher
 

Viewers also liked (19)

Distributed tracing - get a grasp on your production
Distributed tracing - get a grasp on your productionDistributed tracing - get a grasp on your production
Distributed tracing - get a grasp on your production
 
Implementing microservices tracing with spring cloud and zipkin (spring one)
Implementing microservices tracing with spring cloud and zipkin (spring one)Implementing microservices tracing with spring cloud and zipkin (spring one)
Implementing microservices tracing with spring cloud and zipkin (spring one)
 
Advances in Cassandra Tracing with Zipkin (Michael Semb Wever, The Last Pickl...
Advances in Cassandra Tracing with Zipkin (Michael Semb Wever, The Last Pickl...Advances in Cassandra Tracing with Zipkin (Michael Semb Wever, The Last Pickl...
Advances in Cassandra Tracing with Zipkin (Michael Semb Wever, The Last Pickl...
 
Apache Htrace overview (20160520)
Apache Htrace overview (20160520)Apache Htrace overview (20160520)
Apache Htrace overview (20160520)
 
SRECon Coherent Performance
SRECon Coherent PerformanceSRECon Coherent Performance
SRECon Coherent Performance
 
Openzipkin conf: Zipkin at Yelp
Openzipkin conf: Zipkin at YelpOpenzipkin conf: Zipkin at Yelp
Openzipkin conf: Zipkin at Yelp
 
Guidelines For The Animation Of J P I C
Guidelines For The Animation Of  J P I CGuidelines For The Animation Of  J P I C
Guidelines For The Animation Of J P I C
 
Securing Your Deployment Pipeline With Docker
Securing Your Deployment Pipeline With DockerSecuring Your Deployment Pipeline With Docker
Securing Your Deployment Pipeline With Docker
 
Troubleshooting RabbitMQ and services that use it
Troubleshooting RabbitMQ and services that use itTroubleshooting RabbitMQ and services that use it
Troubleshooting RabbitMQ and services that use it
 
Make 2016 your year of SMACK talk
Make 2016 your year of SMACK talkMake 2016 your year of SMACK talk
Make 2016 your year of SMACK talk
 
IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)
IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)
IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)
 
Microservices architecture examples
Microservices architecture examplesMicroservices architecture examples
Microservices architecture examples
 
Tracing Microservices with Zipkin
Tracing Microservices with ZipkinTracing Microservices with Zipkin
Tracing Microservices with Zipkin
 
Monitoring Microservices
Monitoring MicroservicesMonitoring Microservices
Monitoring Microservices
 
Technical Seminar PPT
Technical Seminar PPTTechnical Seminar PPT
Technical Seminar PPT
 
Dockercon State of the Art in Microservices
Dockercon State of the Art in MicroservicesDockercon State of the Art in Microservices
Dockercon State of the Art in Microservices
 
Advantages and Disadvantages of Technology
Advantages and Disadvantages of TechnologyAdvantages and Disadvantages of Technology
Advantages and Disadvantages of Technology
 
What is tackled in the Java EE Security API (Java EE 8)
What is tackled in the Java EE Security API (Java EE 8)What is tackled in the Java EE Security API (Java EE 8)
What is tackled in the Java EE Security API (Java EE 8)
 
Customer Service Basics
Customer Service BasicsCustomer Service Basics
Customer Service Basics
 

Recently uploaded

Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 

Recently uploaded (20)

Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 

Zipkin - Strangeloop

  • 1. Z I P K I N A distributed tracing framework
  • 3. Google 0.5 sec slower 20% traffic drop @skr | @thisisfranklin 3
  • 4. Google 0.5 sec slower 1.5 sec faster 20% traffic drop CTR up 12% @skr | @thisisfranklin 3
  • 6. Front end @skr | @thisisfranklin 5
  • 7. Front end Back end @skr | @thisisfranklin 5
  • 11. @skr | @thisisfranklin 8 Picture from http://www.flickr.com/photos/jpellgen
  • 13. • Collects traces from production requests • Low overhead • Minimum of extra work for developers @skr | @thisisfranklin 9
  • 14. Finagle “Finagle is an asynchronous network stack for the JVM that you can use to build asynchronous Remote Procedure Call (RPC) clients and servers in Java, Scala, or any JVM-hosted language.” github.com/twitter/finagle
  • 17. Finagle Http service @skr | @thisisfranklin 12
  • 18. Finagle Http service @skr | @thisisfranklin 12
  • 19. Finagle Http service Finagle Thrift Service @skr | @thisisfranklin 12
  • 20. Finagle Http service Finagle Thrift Service @skr | @thisisfranklin 12
  • 21. Finagle Http service Finagle Thrift Service @skr | @thisisfranklin 12
  • 22. Finagle Http service Finagle Thrift Service Service @skr | @thisisfranklin 12
  • 23. Finagle Http service Finagle Thrift Service Service @skr | @thisisfranklin 12
  • 24. Finagle Http service Finagle Thrift Service Service @skr | @thisisfranklin 12
  • 25. Zipkin terminology ‣ Annotation: string data associated with a particular timestamp, service, and host Time time: 2012-01-21 22:37:01 value: “something happened” server: 135.34.53.2 service: “timelineservice” @skr | @thisisfranklin 13
  • 26. Span: represents one specific method call; made up of a set of annotations. Has a name and an id. Time T:0ms Client Send Span @skr | @thisisfranklin 14
  • 27. Span: represents one specific method call; made up of a set of annotations. Has a name and an id. Time T:0ms Client Send Span T:10ms Server Receive @skr | @thisisfranklin 14
  • 28. Span: represents one specific method call; made up of a set of annotations. Has a name and an id. Time T:0ms Client Send Span T:10ms Server Receive T:90ms Server Send @skr | @thisisfranklin 14
  • 29. Span: represents one specific method call; made up of a set of annotations. Has a name and an id. Time T:0ms Client Send T:100ms Client Receive Span T:10ms Server Receive T:90ms Server Send @skr | @thisisfranklin 14
  • 30. Span: represents one specific method call; made up of a set of annotations. Has a name and an id. Time T:0ms Client Send T:100ms Client Receive Span T:20ms Read 30 kbytes from file T:10ms Server Receive T:90ms Server Send @skr | @thisisfranklin 14
  • 31. Span: represents one specific method call; made up of a set of annotations. Has a name and an id. Time T:0ms Client Send T:100ms Client Receive Span T:20ms Read 30 kbytes from file T:10ms Server Receive T:90ms Server Send ‣ Trace: a set of spans all associated with the same request @skr | @thisisfranklin 14
  • 32. Finagle http service @skr | @thisisfranklin 15
  • 33. • Generate a random i64 trace id Finagle http service @skr | @thisisfranklin 15
  • 34. • Generate a random i64 trace id • Decide if we should sample the trace or not Finagle http service @skr | @thisisfranklin 15
  • 35. • Generate a random i64 trace id • Decide if we should sample the trace or not Finagle http service Finagle thrift service @skr | @thisisfranklin 15
  • 36. • Generate a random i64 trace id • Decide if we should sample the trace or not Finagle http service • Generate new span id Finagle thrift service @skr | @thisisfranklin 15
  • 37. • Generate a random i64 trace id • Decide if we should sample the trace or not Finagle http service • Generate new span id • Pass trace header Finagle thrift service @skr | @thisisfranklin 15
  • 38. • Generate a random i64 trace id • Decide if we should sample the trace or not Finagle http struct RequestHeader { service i64 trace_id, i64 span_id, • Generate new span id optional i64 parent_span_id, • Pass trace header optional bool sampled } Finagle thrift service @skr | @thisisfranklin 15
  • 39. • Generate a random i64 trace id • Decide if we should sample the trace or not Finagle http struct RequestHeader { service i64 trace_id, i64 span_id, • Generate new span id optional i64 parent_span_id, • Pass trace header optional bool sampled } Finagle • Thrift service adopts trace id from thrift header if it exists service @skr | @thisisfranklin 15
  • 40. Finagle http service Finagle thrift service @skr | @thisisfranklin 15
  • 41. Finagle http service Finagle Finagle thrift thrift service service @skr | @thisisfranklin 15
  • 42. Finagle http service Finagle Finagle thrift thrift service service Finagle thrift service @skr | @thisisfranklin 15
  • 43. Finagle http service S Finagle Finagle thrift thrift S service service S Finagle thrift S service @skr | @thisisfranklin 15
  • 44. Finagle http service S Zipkin collector Finagle Finagle thrift thrift S service service S Cassandra Finagle thrift S service @skr | @thisisfranklin 15
  • 45. Finagle http service S Zipkin collector Finagle Finagle thrift thrift S service service S Zipkin Zipkin Cassandra Query UI Finagle thrift S service @skr | @thisisfranklin 15
  • 47. Finagle Thrift server ServerBuilder() .bindTo(address) .codec(ThriftServerFramedCodec()) .name("servicename") .build(someService) @skr | @thisisfranklin 17
  • 48. Finagle Thrift server ServerBuilder() .bindTo(address) .codec(ThriftServerFramedCodec()) .name("servicename") .build(someService) @skr | @thisisfranklin 17
  • 49. Finagle Thrift server ServerBuilder() .bindTo(address) .codec(ThriftServerFramedCodec()) .name("servicename") .build(someService) @skr | @thisisfranklin 17
  • 50. Finagle Thrift server ServerBuilder() .bindTo(address) .codec(ThriftServerFramedCodec()) .name("servicename") .build(someService) @skr | @thisisfranklin 17
  • 51. ServerBuilder() .bindTo(address) .codec(ThriftServerFramedCodec()) .name("servicename") .tracerFactory(ZipkinTracer()) .build(someService) @skr | @thisisfranklin 18
  • 52. ClientBuilder() .cluster(hosts) .codec(ThriftClientFramedCodec()) .name("clientname") .tracerFactory(ZipkinTracer()) .build(someService) @skr | @thisisfranklin 19
  • 53. Trace.record("doing stuff") @skr | @thisisfranklin 20
  • 54. Trace.record("doing stuff") @skr | @thisisfranklin 20
  • 55. Trace.record("doing stuff") time: 2012-01-21 22:37:01 value: “doing stuff” server: 135.34.53.2 service: “timelineservice” @skr | @thisisfranklin 20
  • 56. Trace.recordBinary("key", data) @skr | @thisisfranklin 21
  • 57. Key Value Trace.recordBinary("key", data) responsecode 500 cache:somekey Hit sql.query select *... @skr | @thisisfranklin 21
  • 58. Platform Protocol Client Server Finagle Thrift Yes Yes Finagle HTTP Yes Yes Finagle Memcache Yes No Finagle Redis Yes No Cassie Thrift Yes No Querulous JDBC Yes No Ruby Thrift Yes Yes @skr | @thisisfranklin 22
  • 63.
  • 64.
  • 65. What did we find?
  • 66. github.com/twitter/zipkin @zipkinproject @skr | @thisisfranklin 27

Editor's Notes

  1. \n
  2. Before we get into what Zipkin is: why we created it\n
  3. Shit is slow, you lose users and money\n
  4. \n
  5. Simplify wildly, there are two parts to web performance\nFront end. The order assets are loaded, minifying and other tricsk\nBack end. How quickly can we generate and push out the HTML/JSON/whatever\n
  6. For the front end we have these nice development tools. Shows us how the assets loaded, what it was waiting for and so on.\nWe want a fancy tool like this for the backend.\nPicked a page where server side was unusually bad, normally it’s mostly FE\n
  7. For the back end we only had these graphs to look at per service.\nNothing that ties them together\n
  8. Capture information about how services in a datacentre is working together to respond to a request. \n
  9. Read this paper from Google called Dapper\n
  10. Read this paper from Google called Dapper\n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  28. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  29. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  30. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  31. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  32. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  33. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  34. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  35. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  36. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  37. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  38. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  39. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  40. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  41. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  42. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  43. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  44. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  45. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  46. Mention that Cassandra and Scribe can both be replaced. Zookeeper coordination?\n
  47. \n
  48. Mention dependency on finagle-zipkin\n
  49. Mention dependency on finagle-zipkin\n
  50. Mention dependency on finagle-zipkin\n
  51. Mention dependency on finagle-zipkin\n
  52. \n
  53. \n
  54. \n
  55. \n
  56. \n
  57. \n
  58. \n
  59. \n
  60. \n
  61. Trace view\nServices on the left. Time scale on top. Least impactful parts of trace collapsed automatically.\nMention bootstrap and dj\n
  62. \n
  63. It’s all open source, check it out now.\n