SlideShare a Scribd company logo
1 of 32
Download to read offline
DEBUGGING
MICROSERVICES
Idit Levine
solo.io
About
meIdit Levine
Founder and CEO of solo.io
@Idit_Levine
@ilevine
Debugging microservices
applications is hard
The problem:
The problem
A monolithic application
consists of a single process
An attached debugger allows
viewing the complete state of
the application during runtime
A microservices application
consists of potentially hundreds
of processes
Is it possible to get a complete
view of the state of a such
application?!
The problem
The problem
OpenTracing
Solution I
OpenTracing
A
B
E
D
C
Edge service
Unique ID → {context}
{context}
{context}
{context}
{context}
1. assign a unique identifier to each
request at the edge service
2. store it in a context object, along with
other metadata
3. propagate the context across process
boundaries (in-band)
4. baggage is arbitrary K/V
5. capture timing, events, tags and collect
them out of band (async)
6.re-assemble the call tree from the storage
for the UI
OpenTracing
A
B
E
D
C
Edge service
Unique ID → {context}
{context}
{context}
{context}
{context}
A
B
E
C
D
TRACE
SPANS
OpenTracing Architecture
spans - basic unit of timing and causality. can be tagged with key/value pairs. logs - structured
data recorded on a span.
span context - serializable format for linking spans across network boundaries. carries baggage,
such as a request and client IDs.
tracers - anything that plugs into the OpenTracing API to record information. zipKin, jaeger &
lightstep. but also metrics (Prometheus) and logging.
OpenTracing
OpenTracing is a consistent, expressive, vendor-neutral APIs
for popular platforms, OpenTracing makes it easy for
developers to add (or switch) tracing implementations with an
O(1) configuration change.
OpenTracing Demo
OpenTracing uses
logging - easy to output to any logging tool, even from OSS components.
metrics/alerting - measure based on tags, span timing, log data.
context propagation - use baggage to carry request and user ID’s, etc.
critical path analysis - drill down into request latency in very high fidelity.
system topology analysis - identify bottlenecks due to shared resources.
OpenTracing limitations
openTracing does not provide run-time debugging
openTracing requires wrapping and changing the code
no holistic view of the application state – can only see what was printed
the process (repeatedly modify the application and test) is expansive
Impossible to change variable values in runtime
logging and printing results in performances overhead
Squash
Solution II
Squash brings the power of modern popular debuggers to developers of microservices apps that run on
container orchestration platforms.
Squash bridges between the orchestration platform (without changing it) and IDE.
With Squash, you can:
• Live debugging cross multi microservices
• Debug container in a pod
• Debug a service
• Set breakpoints
• Step through the code
• View and modify values of variables
• and more ...
Squash Demo
Squash Architecture
Squash server: holds the information about the breakpoints for
each application, orchestrates and controls the squash clients.
Squash server deploys and runs on Kubernetes
Squash client: deploys as daemon set on Kubernetes node. The
client wraps as docker container, and also contains the binary of
the debuggers.
Squash User Interface: squash uses IDEs as its user interface.
After installing the Squash extension, Squash commands are
available in the IDE command palette.
What vegetable scares all the
bugs? Squash!”
one of my 8-year old daughter's
favorite riddles
Squash Architecture: vs code
extension
vs code extension ➜ kubectl to present the
user pod/container/debugger options
vs code extension ➜ Squash server with
debug config (pod/container/debugger
/breakpoint) ➜ waits for debug session
vs code extension ➜ connects to the debug
server & transfers control to the native debug
extension.
Squash Architecture: Squash
Server
vs code extension ➜ Squash server
Squash server ➜ relevant Squash client with debug config
(pod/container/debugger /breakpoint)
Squash server ➜ waits for debug session
Squash Architecture: Squash Client
Squash server ➜ Squash client
Squash client ➜ container runtime interface (to obtain the
container host pid)
Squash client ➜ runs the debugger, attaches it to the process in the
container, and sets the application breakpoints
Squash client ➜ return debug session.
Squash high level
Architecture
Platforms IDEs Debuggers
Squash high level
Architecture
Platforms IDEs Debuggers
We are looking for community help to add support for more debuggers, platforms and IDEs.
Check out at github:
https://github.com/solo-io/squash
Squash: open source
project
Service mesh
Solution III
Service
MeshService mesh data plane:
Touches every packet/request in the system.
Responsible for service discovery, health
checking, routing, load balancing,
authentication/authorization, and observability.
Service mesh control plane:
Provides policy and configuration for all the
running data planes in the mesh. Does not touch
any packets/requests in the system. The control
plane turns all of the data planes into a
distributed system.
Envoy – data
planeOut of process architecture: developers to focus on business logic
Modern C++11 code base: Fast and productive.
L3/L4 filter architecture: Can be used for things other than HTTP (TCP proxy at its core)
HTTP L7 filter architecture: Make it easy to plug in different functionality.
HTTP/2 first! (Including gRPC and a nifty gRPC HTTP/1.1 bridge).
Service discovery and active health checking.
Advanced load balancing: Retry, timeouts, circuit breaking, rate limiting, shadowing, etc.
Best in class observability: stats, logging, and tracing.
Edge proxy: routing and TLS.
The network should be
transparent to applications.
When network and
application problems do
occur it should be easy to
determine the source of
the problem.
Istio – control
planePilot: responsible for the lifecycle of
Envoy instances deployed across the Istio
service mesh. Pilot exposes APIs for
service discovery, dynamic updates to
load balancing pools and routing
tables.
Mixer: provides Precondition Checking
(authentication, ACL checks and more),
Quota Management and Telemetry
Reporting.
Istio-Auth enhance the security of
microservices and their communication
without requiring service code changes.
Service mesh,
OpenTracing, and
Squash
Towards an integrated solution
The whole
solutionStep 1:
vs code extension ➜ Squash server creating a debug
config (service & image) and wait for the debug session to
connect.
Step 2:
envoy gets a curl request with squash header
Step 3:
➜ envoy ask Squash server to debug itself and wait for the
debug session.
1
2
3
4
5
The whole
solutionStep 4:
Squash server ➜ Squash client
Squash client ➜ container runtime interface (to obtain the
container host pid)
Squash client ➜ runs the debugger, attaches it to the process in
the container, and sets the application breakpoints
Squash client ➜ return debug session.
Step 5:
vs code extension ➜ connects to the debug
server & transfers control to the native debug
extension.
envoy resume traffic to the app
1
2
3
4
5
Service Mesh Demo

More Related Content

What's hot

NGINX ADC: Basics and Best Practices – EMEA
NGINX ADC: Basics and Best Practices – EMEANGINX ADC: Basics and Best Practices – EMEA
NGINX ADC: Basics and Best Practices – EMEANGINX, Inc.
 
Show Me Kafka Tools That Will Increase My Productivity! (Stephane Maarek, Dat...
Show Me Kafka Tools That Will Increase My Productivity! (Stephane Maarek, Dat...Show Me Kafka Tools That Will Increase My Productivity! (Stephane Maarek, Dat...
Show Me Kafka Tools That Will Increase My Productivity! (Stephane Maarek, Dat...confluent
 
[오픈소스컨설팅]쿠버네티스를 활용한 개발환경 구축
[오픈소스컨설팅]쿠버네티스를 활용한 개발환경 구축[오픈소스컨설팅]쿠버네티스를 활용한 개발환경 구축
[오픈소스컨설팅]쿠버네티스를 활용한 개발환경 구축Ji-Woong Choi
 
Prometheus - basics
Prometheus - basicsPrometheus - basics
Prometheus - basicsJuraj Hantak
 
Reactive Card Magic: Understanding Spring WebFlux and Project Reactor
Reactive Card Magic: Understanding Spring WebFlux and Project ReactorReactive Card Magic: Understanding Spring WebFlux and Project Reactor
Reactive Card Magic: Understanding Spring WebFlux and Project ReactorVMware Tanzu
 
Microservices Meetup San Francisco - August 2017 Talk on NATS
Microservices Meetup San Francisco - August 2017 Talk on NATSMicroservices Meetup San Francisco - August 2017 Talk on NATS
Microservices Meetup San Francisco - August 2017 Talk on NATSNATS
 
Multi cluster management with rancher
Multi cluster management with rancherMulti cluster management with rancher
Multi cluster management with rancherKyohei Mizumoto
 
[오픈소스컨설팅]Scouter 설치 및 사용가이드(JBoss)
[오픈소스컨설팅]Scouter 설치 및 사용가이드(JBoss)[오픈소스컨설팅]Scouter 설치 및 사용가이드(JBoss)
[오픈소스컨설팅]Scouter 설치 및 사용가이드(JBoss)Ji-Woong Choi
 
Introduction to Chef
Introduction to ChefIntroduction to Chef
Introduction to ChefKnoldus Inc.
 
Introduction to React Native
Introduction to React NativeIntroduction to React Native
Introduction to React NativeSambhu Lakshmanan
 
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...Simplilearn
 
Monitoring Kubernetes with Prometheus
Monitoring Kubernetes with PrometheusMonitoring Kubernetes with Prometheus
Monitoring Kubernetes with PrometheusGrafana Labs
 
Firebase - A real-time server
Firebase - A real-time serverFirebase - A real-time server
Firebase - A real-time serverAneeq Anwar
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker IntroductionPeng Xiao
 
Ownership System in Rust
Ownership System in RustOwnership System in Rust
Ownership System in RustChih-Hsuan Kuo
 
Spring 3.1 and MVC Testing Support
Spring 3.1 and MVC Testing SupportSpring 3.1 and MVC Testing Support
Spring 3.1 and MVC Testing SupportSam Brannen
 
NGINX Installation and Tuning
NGINX Installation and TuningNGINX Installation and Tuning
NGINX Installation and TuningNGINX, Inc.
 
Coroutines in Kotlin
Coroutines in KotlinCoroutines in Kotlin
Coroutines in KotlinAlexey Soshin
 

What's hot (20)

NGINX ADC: Basics and Best Practices – EMEA
NGINX ADC: Basics and Best Practices – EMEANGINX ADC: Basics and Best Practices – EMEA
NGINX ADC: Basics and Best Practices – EMEA
 
Show Me Kafka Tools That Will Increase My Productivity! (Stephane Maarek, Dat...
Show Me Kafka Tools That Will Increase My Productivity! (Stephane Maarek, Dat...Show Me Kafka Tools That Will Increase My Productivity! (Stephane Maarek, Dat...
Show Me Kafka Tools That Will Increase My Productivity! (Stephane Maarek, Dat...
 
[오픈소스컨설팅]쿠버네티스를 활용한 개발환경 구축
[오픈소스컨설팅]쿠버네티스를 활용한 개발환경 구축[오픈소스컨설팅]쿠버네티스를 활용한 개발환경 구축
[오픈소스컨설팅]쿠버네티스를 활용한 개발환경 구축
 
Prometheus - basics
Prometheus - basicsPrometheus - basics
Prometheus - basics
 
Reactive Card Magic: Understanding Spring WebFlux and Project Reactor
Reactive Card Magic: Understanding Spring WebFlux and Project ReactorReactive Card Magic: Understanding Spring WebFlux and Project Reactor
Reactive Card Magic: Understanding Spring WebFlux and Project Reactor
 
Microservices Meetup San Francisco - August 2017 Talk on NATS
Microservices Meetup San Francisco - August 2017 Talk on NATSMicroservices Meetup San Francisco - August 2017 Talk on NATS
Microservices Meetup San Francisco - August 2017 Talk on NATS
 
Multi cluster management with rancher
Multi cluster management with rancherMulti cluster management with rancher
Multi cluster management with rancher
 
[오픈소스컨설팅]Scouter 설치 및 사용가이드(JBoss)
[오픈소스컨설팅]Scouter 설치 및 사용가이드(JBoss)[오픈소스컨설팅]Scouter 설치 및 사용가이드(JBoss)
[오픈소스컨설팅]Scouter 설치 및 사용가이드(JBoss)
 
Introduction to Chef
Introduction to ChefIntroduction to Chef
Introduction to Chef
 
Introduction to React Native
Introduction to React NativeIntroduction to React Native
Introduction to React Native
 
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
 
Kubernetes Basics
Kubernetes BasicsKubernetes Basics
Kubernetes Basics
 
Monitoring Kubernetes with Prometheus
Monitoring Kubernetes with PrometheusMonitoring Kubernetes with Prometheus
Monitoring Kubernetes with Prometheus
 
Firebase - A real-time server
Firebase - A real-time serverFirebase - A real-time server
Firebase - A real-time server
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
 
Kubernetes
KubernetesKubernetes
Kubernetes
 
Ownership System in Rust
Ownership System in RustOwnership System in Rust
Ownership System in Rust
 
Spring 3.1 and MVC Testing Support
Spring 3.1 and MVC Testing SupportSpring 3.1 and MVC Testing Support
Spring 3.1 and MVC Testing Support
 
NGINX Installation and Tuning
NGINX Installation and TuningNGINX Installation and Tuning
NGINX Installation and Tuning
 
Coroutines in Kotlin
Coroutines in KotlinCoroutines in Kotlin
Coroutines in Kotlin
 

Similar to Debugging Microservices - QCON 2017

Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)QAware GmbH
 
Tungsten Fabric Overview
Tungsten Fabric OverviewTungsten Fabric Overview
Tungsten Fabric OverviewMichelle Holley
 
Spring boot microservice metrics monitoring
Spring boot   microservice metrics monitoringSpring boot   microservice metrics monitoring
Spring boot microservice metrics monitoringOracle Korea
 
Spring Boot - Microservice Metrics Monitoring
Spring Boot - Microservice Metrics MonitoringSpring Boot - Microservice Metrics Monitoring
Spring Boot - Microservice Metrics MonitoringDonghuKIM2
 
Scale and Load Testing of Micro-Service
Scale and Load Testing of Micro-ServiceScale and Load Testing of Micro-Service
Scale and Load Testing of Micro-ServiceIRJET Journal
 
Managing microservices with Istio Service Mesh
Managing microservices with Istio Service MeshManaging microservices with Istio Service Mesh
Managing microservices with Istio Service MeshRafik HARABI
 
Here Be Dragons: Security Maps of the Container New World
Here Be Dragons: Security Maps of the Container New WorldHere Be Dragons: Security Maps of the Container New World
Here Be Dragons: Security Maps of the Container New WorldC4Media
 
Red Hat and kubernetes: awesome stuff coming your way
Red Hat and kubernetes:  awesome stuff coming your wayRed Hat and kubernetes:  awesome stuff coming your way
Red Hat and kubernetes: awesome stuff coming your wayJohannes Brännström
 
Dynatrace - Red Hat workshop : Monolith to Microservices
Dynatrace - Red Hat workshop : Monolith to MicroservicesDynatrace - Red Hat workshop : Monolith to Microservices
Dynatrace - Red Hat workshop : Monolith to MicroservicesSteve Caron
 
Application cloudification with liberty and urban code deploy - UCD
Application cloudification with liberty and urban code deploy - UCDApplication cloudification with liberty and urban code deploy - UCD
Application cloudification with liberty and urban code deploy - UCDDavide Veronese
 
Vijay Oscon
Vijay OsconVijay Oscon
Vijay Osconvijayrvr
 
Build cloud native solution using open source
Build cloud native solution using open source Build cloud native solution using open source
Build cloud native solution using open source Nitesh Jadhav
 
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-orsCharacterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-orsLee Calcote
 
Putting Microservices on a Diet: with Istio!
Putting Microservices on a Diet: with Istio!Putting Microservices on a Diet: with Istio!
Putting Microservices on a Diet: with Istio!QAware GmbH
 
Docker Application to Scientific Computing
Docker Application to Scientific ComputingDocker Application to Scientific Computing
Docker Application to Scientific ComputingPeter Bryzgalov
 
OpenShift Meetup - Tokyo - Service Mesh and Serverless Overview
OpenShift Meetup - Tokyo - Service Mesh and Serverless OverviewOpenShift Meetup - Tokyo - Service Mesh and Serverless Overview
OpenShift Meetup - Tokyo - Service Mesh and Serverless OverviewMaría Angélica Bracho
 
Cerberus : Framework for Manual and Automated Testing (Web Application)
Cerberus : Framework for Manual and Automated Testing (Web Application)Cerberus : Framework for Manual and Automated Testing (Web Application)
Cerberus : Framework for Manual and Automated Testing (Web Application)CIVEL Benoit
 
Cerberus_Presentation1
Cerberus_Presentation1Cerberus_Presentation1
Cerberus_Presentation1CIVEL Benoit
 

Similar to Debugging Microservices - QCON 2017 (20)

KrakenD API Gateway
KrakenD API GatewayKrakenD API Gateway
KrakenD API Gateway
 
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
 
Tungsten Fabric Overview
Tungsten Fabric OverviewTungsten Fabric Overview
Tungsten Fabric Overview
 
Spring boot microservice metrics monitoring
Spring boot   microservice metrics monitoringSpring boot   microservice metrics monitoring
Spring boot microservice metrics monitoring
 
Spring Boot - Microservice Metrics Monitoring
Spring Boot - Microservice Metrics MonitoringSpring Boot - Microservice Metrics Monitoring
Spring Boot - Microservice Metrics Monitoring
 
Scale and Load Testing of Micro-Service
Scale and Load Testing of Micro-ServiceScale and Load Testing of Micro-Service
Scale and Load Testing of Micro-Service
 
Managing microservices with Istio Service Mesh
Managing microservices with Istio Service MeshManaging microservices with Istio Service Mesh
Managing microservices with Istio Service Mesh
 
Here Be Dragons: Security Maps of the Container New World
Here Be Dragons: Security Maps of the Container New WorldHere Be Dragons: Security Maps of the Container New World
Here Be Dragons: Security Maps of the Container New World
 
Red Hat and kubernetes: awesome stuff coming your way
Red Hat and kubernetes:  awesome stuff coming your wayRed Hat and kubernetes:  awesome stuff coming your way
Red Hat and kubernetes: awesome stuff coming your way
 
Dynatrace - Red Hat workshop : Monolith to Microservices
Dynatrace - Red Hat workshop : Monolith to MicroservicesDynatrace - Red Hat workshop : Monolith to Microservices
Dynatrace - Red Hat workshop : Monolith to Microservices
 
Application cloudification with liberty and urban code deploy - UCD
Application cloudification with liberty and urban code deploy - UCDApplication cloudification with liberty and urban code deploy - UCD
Application cloudification with liberty and urban code deploy - UCD
 
Vijay Oscon
Vijay OsconVijay Oscon
Vijay Oscon
 
Build cloud native solution using open source
Build cloud native solution using open source Build cloud native solution using open source
Build cloud native solution using open source
 
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-orsCharacterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors
 
Microservices with Spring
Microservices with SpringMicroservices with Spring
Microservices with Spring
 
Putting Microservices on a Diet: with Istio!
Putting Microservices on a Diet: with Istio!Putting Microservices on a Diet: with Istio!
Putting Microservices on a Diet: with Istio!
 
Docker Application to Scientific Computing
Docker Application to Scientific ComputingDocker Application to Scientific Computing
Docker Application to Scientific Computing
 
OpenShift Meetup - Tokyo - Service Mesh and Serverless Overview
OpenShift Meetup - Tokyo - Service Mesh and Serverless OverviewOpenShift Meetup - Tokyo - Service Mesh and Serverless Overview
OpenShift Meetup - Tokyo - Service Mesh and Serverless Overview
 
Cerberus : Framework for Manual and Automated Testing (Web Application)
Cerberus : Framework for Manual and Automated Testing (Web Application)Cerberus : Framework for Manual and Automated Testing (Web Application)
Cerberus : Framework for Manual and Automated Testing (Web Application)
 
Cerberus_Presentation1
Cerberus_Presentation1Cerberus_Presentation1
Cerberus_Presentation1
 

Recently uploaded

[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdfSteve Caron
 
Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...
Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...
Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...Milind Agarwal
 
Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...
Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...
Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...Piyovi
 
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...OnePlan Solutions
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolsosttopstonverter
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesVictoriaMetrics
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...Bert Jan Schrijver
 
What are the core components of Azure Data Engineer courses.docx
What are the core components of Azure Data Engineer courses.docxWhat are the core components of Azure Data Engineer courses.docx
What are the core components of Azure Data Engineer courses.docxkzayra69
 
Preparing BitVisor for Supporting Multiple Architectures
Preparing BitVisor for Supporting Multiple ArchitecturesPreparing BitVisor for Supporting Multiple Architectures
Preparing BitVisor for Supporting Multiple ArchitecturesAke Koomsin
 
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxUnderstanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxSasikiranMarri
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdfAndrey Devyatkin
 
Tech Tuesday Slides - Getting Started with the Portfolio Module.
Tech Tuesday Slides - Getting Started with the Portfolio Module.Tech Tuesday Slides - Getting Started with the Portfolio Module.
Tech Tuesday Slides - Getting Started with the Portfolio Module.OnePlan Solutions
 
Revolutionize Your Video Editing with InVideo.io: A Comprehensive Review
Revolutionize Your Video Editing with InVideo.io: A Comprehensive ReviewRevolutionize Your Video Editing with InVideo.io: A Comprehensive Review
Revolutionize Your Video Editing with InVideo.io: A Comprehensive Reviewjw364beach
 
Mastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxMastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxAS Design & AST.
 
oracle 23c new features for developer and dba
oracle 23c new features for developer and dbaoracle 23c new features for developer and dba
oracle 23c new features for developer and dbaRemote DBA Services
 
Key Steps in Agile Software Delivery Roadmap
Key Steps in Agile Software Delivery RoadmapKey Steps in Agile Software Delivery Roadmap
Key Steps in Agile Software Delivery RoadmapIshara Amarasekera
 
ETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBU
ETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBUETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBU
ETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBUsamruddhijedgule2004
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jNeo4j
 
Keeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldKeeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldRoberto Pérez Alcolea
 
full course of software engineering mid term.pdf
full course of software engineering mid term.pdffull course of software engineering mid term.pdf
full course of software engineering mid term.pdfAbdul salam
 

Recently uploaded (20)

[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
 
Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...
Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...
Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...
 
Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...
Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...
Explore the Three Main Types of Logistics - Inbound Logistics, Outbound Logis...
 
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration tools
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
 
What are the core components of Azure Data Engineer courses.docx
What are the core components of Azure Data Engineer courses.docxWhat are the core components of Azure Data Engineer courses.docx
What are the core components of Azure Data Engineer courses.docx
 
Preparing BitVisor for Supporting Multiple Architectures
Preparing BitVisor for Supporting Multiple ArchitecturesPreparing BitVisor for Supporting Multiple Architectures
Preparing BitVisor for Supporting Multiple Architectures
 
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxUnderstanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
 
Tech Tuesday Slides - Getting Started with the Portfolio Module.
Tech Tuesday Slides - Getting Started with the Portfolio Module.Tech Tuesday Slides - Getting Started with the Portfolio Module.
Tech Tuesday Slides - Getting Started with the Portfolio Module.
 
Revolutionize Your Video Editing with InVideo.io: A Comprehensive Review
Revolutionize Your Video Editing with InVideo.io: A Comprehensive ReviewRevolutionize Your Video Editing with InVideo.io: A Comprehensive Review
Revolutionize Your Video Editing with InVideo.io: A Comprehensive Review
 
Mastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxMastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptx
 
oracle 23c new features for developer and dba
oracle 23c new features for developer and dbaoracle 23c new features for developer and dba
oracle 23c new features for developer and dba
 
Key Steps in Agile Software Delivery Roadmap
Key Steps in Agile Software Delivery RoadmapKey Steps in Agile Software Delivery Roadmap
Key Steps in Agile Software Delivery Roadmap
 
ETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBU
ETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBUETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBU
ETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBU
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
 
Keeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldKeeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository world
 
full course of software engineering mid term.pdf
full course of software engineering mid term.pdffull course of software engineering mid term.pdf
full course of software engineering mid term.pdf
 

Debugging Microservices - QCON 2017

  • 2. About meIdit Levine Founder and CEO of solo.io @Idit_Levine @ilevine
  • 4. The problem A monolithic application consists of a single process An attached debugger allows viewing the complete state of the application during runtime A microservices application consists of potentially hundreds of processes Is it possible to get a complete view of the state of a such application?!
  • 8. OpenTracing A B E D C Edge service Unique ID → {context} {context} {context} {context} {context} 1. assign a unique identifier to each request at the edge service 2. store it in a context object, along with other metadata 3. propagate the context across process boundaries (in-band) 4. baggage is arbitrary K/V 5. capture timing, events, tags and collect them out of band (async) 6.re-assemble the call tree from the storage for the UI
  • 9. OpenTracing A B E D C Edge service Unique ID → {context} {context} {context} {context} {context} A B E C D TRACE SPANS
  • 10. OpenTracing Architecture spans - basic unit of timing and causality. can be tagged with key/value pairs. logs - structured data recorded on a span. span context - serializable format for linking spans across network boundaries. carries baggage, such as a request and client IDs. tracers - anything that plugs into the OpenTracing API to record information. zipKin, jaeger & lightstep. but also metrics (Prometheus) and logging.
  • 11. OpenTracing OpenTracing is a consistent, expressive, vendor-neutral APIs for popular platforms, OpenTracing makes it easy for developers to add (or switch) tracing implementations with an O(1) configuration change.
  • 13. OpenTracing uses logging - easy to output to any logging tool, even from OSS components. metrics/alerting - measure based on tags, span timing, log data. context propagation - use baggage to carry request and user ID’s, etc. critical path analysis - drill down into request latency in very high fidelity. system topology analysis - identify bottlenecks due to shared resources.
  • 14. OpenTracing limitations openTracing does not provide run-time debugging openTracing requires wrapping and changing the code no holistic view of the application state – can only see what was printed the process (repeatedly modify the application and test) is expansive Impossible to change variable values in runtime logging and printing results in performances overhead
  • 16. Squash brings the power of modern popular debuggers to developers of microservices apps that run on container orchestration platforms. Squash bridges between the orchestration platform (without changing it) and IDE. With Squash, you can: • Live debugging cross multi microservices • Debug container in a pod • Debug a service • Set breakpoints • Step through the code • View and modify values of variables • and more ...
  • 18. Squash Architecture Squash server: holds the information about the breakpoints for each application, orchestrates and controls the squash clients. Squash server deploys and runs on Kubernetes Squash client: deploys as daemon set on Kubernetes node. The client wraps as docker container, and also contains the binary of the debuggers. Squash User Interface: squash uses IDEs as its user interface. After installing the Squash extension, Squash commands are available in the IDE command palette. What vegetable scares all the bugs? Squash!” one of my 8-year old daughter's favorite riddles
  • 19. Squash Architecture: vs code extension vs code extension ➜ kubectl to present the user pod/container/debugger options vs code extension ➜ Squash server with debug config (pod/container/debugger /breakpoint) ➜ waits for debug session vs code extension ➜ connects to the debug server & transfers control to the native debug extension.
  • 20. Squash Architecture: Squash Server vs code extension ➜ Squash server Squash server ➜ relevant Squash client with debug config (pod/container/debugger /breakpoint) Squash server ➜ waits for debug session
  • 21. Squash Architecture: Squash Client Squash server ➜ Squash client Squash client ➜ container runtime interface (to obtain the container host pid) Squash client ➜ runs the debugger, attaches it to the process in the container, and sets the application breakpoints Squash client ➜ return debug session.
  • 24. We are looking for community help to add support for more debuggers, platforms and IDEs. Check out at github: https://github.com/solo-io/squash Squash: open source project
  • 26. Service MeshService mesh data plane: Touches every packet/request in the system. Responsible for service discovery, health checking, routing, load balancing, authentication/authorization, and observability. Service mesh control plane: Provides policy and configuration for all the running data planes in the mesh. Does not touch any packets/requests in the system. The control plane turns all of the data planes into a distributed system.
  • 27. Envoy – data planeOut of process architecture: developers to focus on business logic Modern C++11 code base: Fast and productive. L3/L4 filter architecture: Can be used for things other than HTTP (TCP proxy at its core) HTTP L7 filter architecture: Make it easy to plug in different functionality. HTTP/2 first! (Including gRPC and a nifty gRPC HTTP/1.1 bridge). Service discovery and active health checking. Advanced load balancing: Retry, timeouts, circuit breaking, rate limiting, shadowing, etc. Best in class observability: stats, logging, and tracing. Edge proxy: routing and TLS. The network should be transparent to applications. When network and application problems do occur it should be easy to determine the source of the problem.
  • 28. Istio – control planePilot: responsible for the lifecycle of Envoy instances deployed across the Istio service mesh. Pilot exposes APIs for service discovery, dynamic updates to load balancing pools and routing tables. Mixer: provides Precondition Checking (authentication, ACL checks and more), Quota Management and Telemetry Reporting. Istio-Auth enhance the security of microservices and their communication without requiring service code changes.
  • 30. The whole solutionStep 1: vs code extension ➜ Squash server creating a debug config (service & image) and wait for the debug session to connect. Step 2: envoy gets a curl request with squash header Step 3: ➜ envoy ask Squash server to debug itself and wait for the debug session. 1 2 3 4 5
  • 31. The whole solutionStep 4: Squash server ➜ Squash client Squash client ➜ container runtime interface (to obtain the container host pid) Squash client ➜ runs the debugger, attaches it to the process in the container, and sets the application breakpoints Squash client ➜ return debug session. Step 5: vs code extension ➜ connects to the debug server & transfers control to the native debug extension. envoy resume traffic to the app 1 2 3 4 5