SlideShare a Scribd company logo
GO Performance Tooling
Adil Hafeez
@adilhafeez
- Performance bites hard when app hits scale
- Even though GO is garbage collected language, allocated objects have
implications on latency and gc time
- When measuring latency,look at percentiles not just averages
- Percentiles show you tail latencies - this also helps you to understand what perf is those
unlucky customers seeing (bottom 1% or 5%)
Importance of Performance Measurement
What is profiling
- Pause application and capture thread stack multiple times per-second
- Usually takes around 100 stack dumps per second
- For java developers it is similar to running jstack (or yourkit) couple of times in a second
- With profiling data we can do lot of things like,
- Shows what functions are used
- Can build call graphs
- Find out what functions are at the top of the stack (taking CPU time)
Viewing CPU/Memory profile snapshot
- Every node is a function call
- A vertex from X to Y indicates a call from X -> Y
- For example in the profile snapshot below
- FindLoops took 4.41s of CPU time
- Remaining 30.28s were spent on outgoing function call
- “web” command opens up web view of profiler
GO Profiling tools - pprof
- What we can profile?
- Standalone application (start with profiling enabled through commandline arg)
- A live process
- Using net.http.pprof, go can capture profile of a live process
- Many others features available thorugh web interface like viewing passed in
command line arguments, memory profile, cpu profile etc.
- What you can do with profiling data
- View methods that are taking more time, or allocation more objects
- Annotate code with cpu/memory profile data
- Slice and dice into different parts of the program for better understanding of cpu time
- GO allows you to do cpu and memory profile (and blocking)
- CPU allows you to look at what functions are taking cpu time
- Memory profile lets you see memory allocation per function
GO Profiling tools - benchmarking
- Run test X number of times and reports the average time
- Prints allocations per call
- Run with multiple CPU (GOMAXPROCs)
- Execute following command to get memory allocation along with runtime,
- $ go test -bench=. -benchmem
pprof commands (top)
- Top command lets you view functions that are taking up most CPU time
pprof commands (weblist)
- weblist annotates source code with profiler data (and assembly) in a
webview
- Drill down and expand each source line to see assembly instructions (pretty powerful)
Live demo
- Type of garbage collectors
- STW - Stop the World
- Concurrent
- With 1.5, go started to have concurrent gc
- This means less time spent in STW phase (~ 10ms)
- Latencies improve overall
- More details here
- GO 1.6 does little better
- GC pauses is even lower
- See here for details
Garbage Collector
- Simplicity - core principle
- GOGC - the only parameter that you’d ever to tune
- Defaults to 100%, which really means that your heap size after garbage collection will be
kept at
- gctrace (GODEBUG=gctrace=1 commandline)
- Go program will start writing detailed information about GC on stdout
- Helpful in debuggin whether GC is the cause of latency or the not
- GOMAXPROC
- This sets the maximum number of processes GO process can use
- As of go1.5, there isnt any need to set this as runtime figures it out automatically
GO Performance Tuning (GC)
Summary
- Be judicious when allocating new objects
- See if you can use simpler data structures (e.g. slice instead of map)
- Reuse objects if possible (connection pooling, objects cache etc)
- Measure latencies in percentiles
- Enable web pprof in our application
- Doesnt cost much, and lets you take traces of live process
- For all users of 1.3, upgrade to 1.6
- Concurrent GC
- Less time spent in GC, more time for the app (mutator)
- Play with GOGC and gcdebug parameter
Links and further reading
- GO 1.5 gc release notes - https://blog.golang.org/go15gc
- Concurrent vs stop the world gc - https://talks.golang.org/2015/go-gc.pdf
- Testing and benchmarking - https://golang.org/pkg/testing/
- Running pprof from http - https://golang.org/pkg/net/http/pprof/
- Profiling GO Programs - https://blog.golang.org/profiling-go-programs
- http://www.stuartcheshire.org/rants/Latency.html

More Related Content

What's hot

Let’s Fix Logging Once and for All
Let’s Fix Logging Once and for AllLet’s Fix Logging Once and for All
Let’s Fix Logging Once and for All
ScyllaDB
 
Tips on High Performance Server Programming
Tips on High Performance Server ProgrammingTips on High Performance Server Programming
Tips on High Performance Server Programming
Joshua Zhu
 
Basics of Logical Replication,Streaming replication vs Logical Replication ,U...
Basics of Logical Replication,Streaming replication vs Logical Replication ,U...Basics of Logical Replication,Streaming replication vs Logical Replication ,U...
Basics of Logical Replication,Streaming replication vs Logical Replication ,U...
Rajni Baliyan
 
plProxy, pgBouncer, pgBalancer
plProxy, pgBouncer, pgBalancerplProxy, pgBouncer, pgBalancer
plProxy, pgBouncer, pgBalancer
elliando dias
 
JavaScript code academy - introduction
JavaScript code academy - introductionJavaScript code academy - introduction
JavaScript code academy - introduction
Jaroslav Kubíček
 
DB Latency Using DRAM + PMem in App Direct & Memory Modes
DB Latency Using DRAM + PMem in App Direct & Memory ModesDB Latency Using DRAM + PMem in App Direct & Memory Modes
DB Latency Using DRAM + PMem in App Direct & Memory Modes
ScyllaDB
 
GulpJs - An Introduction
GulpJs - An IntroductionGulpJs - An Introduction
GulpJs - An Introduction
Knoldus Inc.
 
Islamabad PUG - 7th meetup - performance tuning
Islamabad PUG - 7th meetup - performance tuningIslamabad PUG - 7th meetup - performance tuning
Islamabad PUG - 7th meetup - performance tuning
Umair Shahid
 
Salt Stack - Subhankar Sengupta
Salt Stack - Subhankar SenguptaSalt Stack - Subhankar Sengupta
Salt Stack - Subhankar Sengupta
DevOpsBangalore
 
Shootout at the AWS Corral
Shootout at the AWS CorralShootout at the AWS Corral
Shootout at the AWS Corral
PostgreSQL Experts, Inc.
 
Linux talk | scheduled tasks
Linux talk | scheduled tasksLinux talk | scheduled tasks
Linux talk | scheduled tasks
YashwantVarma1
 
OSNoise Tracer: Who Is Stealing My CPU Time?
OSNoise Tracer: Who Is Stealing My CPU Time?OSNoise Tracer: Who Is Stealing My CPU Time?
OSNoise Tracer: Who Is Stealing My CPU Time?
ScyllaDB
 
Linux fundamental - Chap 15 Job Scheduling
Linux fundamental - Chap 15 Job SchedulingLinux fundamental - Chap 15 Job Scheduling
Linux fundamental - Chap 15 Job Scheduling
Kenny (netman)
 
Scalability strategies for cloud based system architecture
Scalability strategies for cloud based system architectureScalability strategies for cloud based system architecture
Scalability strategies for cloud based system architecture
SangJin Kang
 
Full Stack Load Testing
Full Stack Load Testing Full Stack Load Testing
Full Stack Load Testing
Terral R Jordan
 
Gulp
GulpGulp
Gulp
ShuYu You
 
Rails Application Optimization Techniques & Tools
Rails Application Optimization Techniques & ToolsRails Application Optimization Techniques & Tools
Rails Application Optimization Techniques & Tools
guest05c09d
 
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 InstanceExtreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
ScyllaDB
 
Scaling Apache Pulsar to 10 Petabytes/Day
Scaling Apache Pulsar to 10 Petabytes/DayScaling Apache Pulsar to 10 Petabytes/Day
Scaling Apache Pulsar to 10 Petabytes/Day
ScyllaDB
 
Bareon functional testing ci
Bareon functional testing   ciBareon functional testing   ci
Bareon functional testing ci
Max Lobur
 

What's hot (20)

Let’s Fix Logging Once and for All
Let’s Fix Logging Once and for AllLet’s Fix Logging Once and for All
Let’s Fix Logging Once and for All
 
Tips on High Performance Server Programming
Tips on High Performance Server ProgrammingTips on High Performance Server Programming
Tips on High Performance Server Programming
 
Basics of Logical Replication,Streaming replication vs Logical Replication ,U...
Basics of Logical Replication,Streaming replication vs Logical Replication ,U...Basics of Logical Replication,Streaming replication vs Logical Replication ,U...
Basics of Logical Replication,Streaming replication vs Logical Replication ,U...
 
plProxy, pgBouncer, pgBalancer
plProxy, pgBouncer, pgBalancerplProxy, pgBouncer, pgBalancer
plProxy, pgBouncer, pgBalancer
 
JavaScript code academy - introduction
JavaScript code academy - introductionJavaScript code academy - introduction
JavaScript code academy - introduction
 
DB Latency Using DRAM + PMem in App Direct & Memory Modes
DB Latency Using DRAM + PMem in App Direct & Memory ModesDB Latency Using DRAM + PMem in App Direct & Memory Modes
DB Latency Using DRAM + PMem in App Direct & Memory Modes
 
GulpJs - An Introduction
GulpJs - An IntroductionGulpJs - An Introduction
GulpJs - An Introduction
 
Islamabad PUG - 7th meetup - performance tuning
Islamabad PUG - 7th meetup - performance tuningIslamabad PUG - 7th meetup - performance tuning
Islamabad PUG - 7th meetup - performance tuning
 
Salt Stack - Subhankar Sengupta
Salt Stack - Subhankar SenguptaSalt Stack - Subhankar Sengupta
Salt Stack - Subhankar Sengupta
 
Shootout at the AWS Corral
Shootout at the AWS CorralShootout at the AWS Corral
Shootout at the AWS Corral
 
Linux talk | scheduled tasks
Linux talk | scheduled tasksLinux talk | scheduled tasks
Linux talk | scheduled tasks
 
OSNoise Tracer: Who Is Stealing My CPU Time?
OSNoise Tracer: Who Is Stealing My CPU Time?OSNoise Tracer: Who Is Stealing My CPU Time?
OSNoise Tracer: Who Is Stealing My CPU Time?
 
Linux fundamental - Chap 15 Job Scheduling
Linux fundamental - Chap 15 Job SchedulingLinux fundamental - Chap 15 Job Scheduling
Linux fundamental - Chap 15 Job Scheduling
 
Scalability strategies for cloud based system architecture
Scalability strategies for cloud based system architectureScalability strategies for cloud based system architecture
Scalability strategies for cloud based system architecture
 
Full Stack Load Testing
Full Stack Load Testing Full Stack Load Testing
Full Stack Load Testing
 
Gulp
GulpGulp
Gulp
 
Rails Application Optimization Techniques & Tools
Rails Application Optimization Techniques & ToolsRails Application Optimization Techniques & Tools
Rails Application Optimization Techniques & Tools
 
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 InstanceExtreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
Extreme HTTP Performance Tuning: 1.2M API req/s on a 4 vCPU EC2 Instance
 
Scaling Apache Pulsar to 10 Petabytes/Day
Scaling Apache Pulsar to 10 Petabytes/DayScaling Apache Pulsar to 10 Petabytes/Day
Scaling Apache Pulsar to 10 Petabytes/Day
 
Bareon functional testing ci
Bareon functional testing   ciBareon functional testing   ci
Bareon functional testing ci
 

Viewers also liked

EChr - CSR & HR - Presentation Dr/ Nicholas Andreou
EChr - CSR & HR - Presentation Dr/ Nicholas AndreouEChr - CSR & HR - Presentation Dr/ Nicholas Andreou
EChr - CSR & HR - Presentation Dr/ Nicholas Andreou
EChr
 
бодовнA листa интерних активности стручног усавршавања
бодовнA листa интерних активности стручног усавршавањабодовнA листa интерних активности стручног усавршавања
бодовнA листa интерних активности стручног усавршавањаmkaseric
 
Whatchamacallit: Controlled Vocabularies for Technical Writers (Write the Doc...
Whatchamacallit: Controlled Vocabularies for Technical Writers (Write the Doc...Whatchamacallit: Controlled Vocabularies for Technical Writers (Write the Doc...
Whatchamacallit: Controlled Vocabularies for Technical Writers (Write the Doc...
Emilie Boillat
 
15:16museums and cultural pro
15:16museums and cultural pro15:16museums and cultural pro
15:16museums and cultural pro
Melba Martinez. PhD
 
WiFifor.me - увеличение лояльности ваших покупателей и гостей
WiFifor.me - увеличение лояльности ваших покупателей и гостейWiFifor.me - увеличение лояльности ваших покупателей и гостей
WiFifor.me - увеличение лояльности ваших покупателей и гостей
Soloten
 
Whatchamacallit: Controlled Vocabularies for Technical Writers (soap! 2015)
Whatchamacallit: Controlled Vocabularies for Technical Writers (soap! 2015)Whatchamacallit: Controlled Vocabularies for Technical Writers (soap! 2015)
Whatchamacallit: Controlled Vocabularies for Technical Writers (soap! 2015)
Emilie Boillat
 
Biographies
BiographiesBiographies
Biographies
ab8179
 
SoloTours
SoloToursSoloTours
SoloToursSoloten
 
Bahaging ginampanan ng simbahan sa pagpapalaganap ng kristaynismo at sa pamam...
Bahaging ginampanan ng simbahan sa pagpapalaganap ng kristaynismo at sa pamam...Bahaging ginampanan ng simbahan sa pagpapalaganap ng kristaynismo at sa pamam...
Bahaging ginampanan ng simbahan sa pagpapalaganap ng kristaynismo at sa pamam...
Ran Apagar
 
Power point lesson_5_prep_g
Power point lesson_5_prep_gPower point lesson_5_prep_g
Power point lesson_5_prep_g
bluepool
 
Steven ten Have. Integrated Coastal Zone (Change) Management
Steven ten Have. Integrated Coastal Zone (Change) ManagementSteven ten Have. Integrated Coastal Zone (Change) Management
Steven ten Have. Integrated Coastal Zone (Change) Management
SUSCOD
 
Biographies
BiographiesBiographies
Biographies
ab8179
 
Мобильное рабочее место руководителя
Мобильное рабочее место руководителяМобильное рабочее место руководителя
Мобильное рабочее место руководителя
Soloten
 
Re-desenhando um novo Paradigma para a ECOnomia - Filipe Moreira Alves
Re-desenhando um novo Paradigma para a ECOnomia - Filipe Moreira AlvesRe-desenhando um novo Paradigma para a ECOnomia - Filipe Moreira Alves
Re-desenhando um novo Paradigma para a ECOnomia - Filipe Moreira Alves
PROUTugal
 
Solo Docs
Solo DocsSolo Docs
Solo Docs
Soloten
 
CSR & HR - Presentation Pr. Chizu Nakajima
CSR & HR - Presentation Pr. Chizu NakajimaCSR & HR - Presentation Pr. Chizu Nakajima
CSR & HR - Presentation Pr. Chizu Nakajima
EChr
 
Solowords
SolowordsSolowords
Solowords
Soloten
 
Контент-менеджмент: руководство к действию.
Контент-менеджмент: руководство к действию.Контент-менеджмент: руководство к действию.
Контент-менеджмент: руководство к действию.
Soloten
 
Readme
ReadmeReadme
Readme
Naji Krayem
 
The North Carolina Sales Institute
The North Carolina Sales InstituteThe North Carolina Sales Institute
The North Carolina Sales Institute
John L. Chapman
 

Viewers also liked (20)

EChr - CSR & HR - Presentation Dr/ Nicholas Andreou
EChr - CSR & HR - Presentation Dr/ Nicholas AndreouEChr - CSR & HR - Presentation Dr/ Nicholas Andreou
EChr - CSR & HR - Presentation Dr/ Nicholas Andreou
 
бодовнA листa интерних активности стручног усавршавања
бодовнA листa интерних активности стручног усавршавањабодовнA листa интерних активности стручног усавршавања
бодовнA листa интерних активности стручног усавршавања
 
Whatchamacallit: Controlled Vocabularies for Technical Writers (Write the Doc...
Whatchamacallit: Controlled Vocabularies for Technical Writers (Write the Doc...Whatchamacallit: Controlled Vocabularies for Technical Writers (Write the Doc...
Whatchamacallit: Controlled Vocabularies for Technical Writers (Write the Doc...
 
15:16museums and cultural pro
15:16museums and cultural pro15:16museums and cultural pro
15:16museums and cultural pro
 
WiFifor.me - увеличение лояльности ваших покупателей и гостей
WiFifor.me - увеличение лояльности ваших покупателей и гостейWiFifor.me - увеличение лояльности ваших покупателей и гостей
WiFifor.me - увеличение лояльности ваших покупателей и гостей
 
Whatchamacallit: Controlled Vocabularies for Technical Writers (soap! 2015)
Whatchamacallit: Controlled Vocabularies for Technical Writers (soap! 2015)Whatchamacallit: Controlled Vocabularies for Technical Writers (soap! 2015)
Whatchamacallit: Controlled Vocabularies for Technical Writers (soap! 2015)
 
Biographies
BiographiesBiographies
Biographies
 
SoloTours
SoloToursSoloTours
SoloTours
 
Bahaging ginampanan ng simbahan sa pagpapalaganap ng kristaynismo at sa pamam...
Bahaging ginampanan ng simbahan sa pagpapalaganap ng kristaynismo at sa pamam...Bahaging ginampanan ng simbahan sa pagpapalaganap ng kristaynismo at sa pamam...
Bahaging ginampanan ng simbahan sa pagpapalaganap ng kristaynismo at sa pamam...
 
Power point lesson_5_prep_g
Power point lesson_5_prep_gPower point lesson_5_prep_g
Power point lesson_5_prep_g
 
Steven ten Have. Integrated Coastal Zone (Change) Management
Steven ten Have. Integrated Coastal Zone (Change) ManagementSteven ten Have. Integrated Coastal Zone (Change) Management
Steven ten Have. Integrated Coastal Zone (Change) Management
 
Biographies
BiographiesBiographies
Biographies
 
Мобильное рабочее место руководителя
Мобильное рабочее место руководителяМобильное рабочее место руководителя
Мобильное рабочее место руководителя
 
Re-desenhando um novo Paradigma para a ECOnomia - Filipe Moreira Alves
Re-desenhando um novo Paradigma para a ECOnomia - Filipe Moreira AlvesRe-desenhando um novo Paradigma para a ECOnomia - Filipe Moreira Alves
Re-desenhando um novo Paradigma para a ECOnomia - Filipe Moreira Alves
 
Solo Docs
Solo DocsSolo Docs
Solo Docs
 
CSR & HR - Presentation Pr. Chizu Nakajima
CSR & HR - Presentation Pr. Chizu NakajimaCSR & HR - Presentation Pr. Chizu Nakajima
CSR & HR - Presentation Pr. Chizu Nakajima
 
Solowords
SolowordsSolowords
Solowords
 
Контент-менеджмент: руководство к действию.
Контент-менеджмент: руководство к действию.Контент-менеджмент: руководство к действию.
Контент-менеджмент: руководство к действию.
 
Readme
ReadmeReadme
Readme
 
The North Carolina Sales Institute
The North Carolina Sales InstituteThe North Carolina Sales Institute
The North Carolina Sales Institute
 

Similar to Go performance tooling

Profiling PHP with Xdebug / Webgrind
Profiling PHP with Xdebug / WebgrindProfiling PHP with Xdebug / Webgrind
Profiling PHP with Xdebug / Webgrind
Sam Keen
 
How to pinpoint and fix sources of performance problems in your SAP BusinessO...
How to pinpoint and fix sources of performance problems in your SAP BusinessO...How to pinpoint and fix sources of performance problems in your SAP BusinessO...
How to pinpoint and fix sources of performance problems in your SAP BusinessO...
Xoomworks Business Intelligence
 
Performance Tuning Oracle Weblogic Server 12c
Performance Tuning Oracle Weblogic Server 12cPerformance Tuning Oracle Weblogic Server 12c
Performance Tuning Oracle Weblogic Server 12c
Ajith Narayanan
 
Odoo command line interface
Odoo command line interfaceOdoo command line interface
Odoo command line interface
Jalal Zahid
 
PuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into OperationsPuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into Operations
grim_radical
 
Understanding Spark Tuning: Strata New York
Understanding Spark Tuning: Strata New YorkUnderstanding Spark Tuning: Strata New York
Understanding Spark Tuning: Strata New York
Rachel Warren
 
Spark Autotuning Talk - Strata New York
Spark Autotuning Talk - Strata New YorkSpark Autotuning Talk - Strata New York
Spark Autotuning Talk - Strata New York
Holden Karau
 
Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018
Holden Karau
 
Scaling python webapps from 0 to 50 million users - A top-down approach
Scaling python webapps from 0 to 50 million users - A top-down approachScaling python webapps from 0 to 50 million users - A top-down approach
Scaling python webapps from 0 to 50 million users - A top-down approach
Jinal Jhaveri
 
Hadoop cluster performance profiler
Hadoop cluster performance profilerHadoop cluster performance profiler
Hadoop cluster performance profiler
Ihor Bobak
 
Azure Functions @ global azure day 2017
Azure Functions  @ global azure day 2017Azure Functions  @ global azure day 2017
Azure Functions @ global azure day 2017
Sean Feldman
 
Cómo se diseña una base de datos que pueda ingerir más de cuatro millones de ...
Cómo se diseña una base de datos que pueda ingerir más de cuatro millones de ...Cómo se diseña una base de datos que pueda ingerir más de cuatro millones de ...
Cómo se diseña una base de datos que pueda ingerir más de cuatro millones de ...
javier ramirez
 
Autosar Basics hand book_v1
Autosar Basics  hand book_v1Autosar Basics  hand book_v1
Autosar Basics hand book_v1
Keroles karam khalil
 
Java Performance and Profiling
Java Performance and ProfilingJava Performance and Profiling
Java Performance and Profiling
WSO2
 
Performance Tuning Cheat Sheet for MongoDB
Performance Tuning Cheat Sheet for MongoDBPerformance Tuning Cheat Sheet for MongoDB
Performance Tuning Cheat Sheet for MongoDB
Severalnines
 
Guide to alfresco monitoring
Guide to alfresco monitoringGuide to alfresco monitoring
Guide to alfresco monitoring
Miguel Rodriguez
 
airflowpresentation1-180717183432.pptx
airflowpresentation1-180717183432.pptxairflowpresentation1-180717183432.pptx
airflowpresentation1-180717183432.pptx
VIJAYAPRABAP
 
Lipstick On Pig
Lipstick On Pig Lipstick On Pig
Lipstick On Pig
bigdatagurus_meetup
 
Netflix - Pig with Lipstick by Jeff Magnusson
Netflix - Pig with Lipstick by Jeff Magnusson Netflix - Pig with Lipstick by Jeff Magnusson
Netflix - Pig with Lipstick by Jeff Magnusson
Hakka Labs
 
Putting Lipstick on Apache Pig at Netflix
Putting Lipstick on Apache Pig at NetflixPutting Lipstick on Apache Pig at Netflix
Putting Lipstick on Apache Pig at Netflix
Jeff Magnusson
 

Similar to Go performance tooling (20)

Profiling PHP with Xdebug / Webgrind
Profiling PHP with Xdebug / WebgrindProfiling PHP with Xdebug / Webgrind
Profiling PHP with Xdebug / Webgrind
 
How to pinpoint and fix sources of performance problems in your SAP BusinessO...
How to pinpoint and fix sources of performance problems in your SAP BusinessO...How to pinpoint and fix sources of performance problems in your SAP BusinessO...
How to pinpoint and fix sources of performance problems in your SAP BusinessO...
 
Performance Tuning Oracle Weblogic Server 12c
Performance Tuning Oracle Weblogic Server 12cPerformance Tuning Oracle Weblogic Server 12c
Performance Tuning Oracle Weblogic Server 12c
 
Odoo command line interface
Odoo command line interfaceOdoo command line interface
Odoo command line interface
 
PuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into OperationsPuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into Operations
 
Understanding Spark Tuning: Strata New York
Understanding Spark Tuning: Strata New YorkUnderstanding Spark Tuning: Strata New York
Understanding Spark Tuning: Strata New York
 
Spark Autotuning Talk - Strata New York
Spark Autotuning Talk - Strata New YorkSpark Autotuning Talk - Strata New York
Spark Autotuning Talk - Strata New York
 
Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018
 
Scaling python webapps from 0 to 50 million users - A top-down approach
Scaling python webapps from 0 to 50 million users - A top-down approachScaling python webapps from 0 to 50 million users - A top-down approach
Scaling python webapps from 0 to 50 million users - A top-down approach
 
Hadoop cluster performance profiler
Hadoop cluster performance profilerHadoop cluster performance profiler
Hadoop cluster performance profiler
 
Azure Functions @ global azure day 2017
Azure Functions  @ global azure day 2017Azure Functions  @ global azure day 2017
Azure Functions @ global azure day 2017
 
Cómo se diseña una base de datos que pueda ingerir más de cuatro millones de ...
Cómo se diseña una base de datos que pueda ingerir más de cuatro millones de ...Cómo se diseña una base de datos que pueda ingerir más de cuatro millones de ...
Cómo se diseña una base de datos que pueda ingerir más de cuatro millones de ...
 
Autosar Basics hand book_v1
Autosar Basics  hand book_v1Autosar Basics  hand book_v1
Autosar Basics hand book_v1
 
Java Performance and Profiling
Java Performance and ProfilingJava Performance and Profiling
Java Performance and Profiling
 
Performance Tuning Cheat Sheet for MongoDB
Performance Tuning Cheat Sheet for MongoDBPerformance Tuning Cheat Sheet for MongoDB
Performance Tuning Cheat Sheet for MongoDB
 
Guide to alfresco monitoring
Guide to alfresco monitoringGuide to alfresco monitoring
Guide to alfresco monitoring
 
airflowpresentation1-180717183432.pptx
airflowpresentation1-180717183432.pptxairflowpresentation1-180717183432.pptx
airflowpresentation1-180717183432.pptx
 
Lipstick On Pig
Lipstick On Pig Lipstick On Pig
Lipstick On Pig
 
Netflix - Pig with Lipstick by Jeff Magnusson
Netflix - Pig with Lipstick by Jeff Magnusson Netflix - Pig with Lipstick by Jeff Magnusson
Netflix - Pig with Lipstick by Jeff Magnusson
 
Putting Lipstick on Apache Pig at Netflix
Putting Lipstick on Apache Pig at NetflixPutting Lipstick on Apache Pig at Netflix
Putting Lipstick on Apache Pig at Netflix
 

Recently uploaded

Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial ApplicationsLarge Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
Rohit Gautam
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Zilliz
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIEnchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Vladimir Iglovikov, Ph.D.
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
SOFTTECHHUB
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Zilliz
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 

Recently uploaded (20)

Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial ApplicationsLarge Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIEnchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 

Go performance tooling

  • 1. GO Performance Tooling Adil Hafeez @adilhafeez
  • 2. - Performance bites hard when app hits scale - Even though GO is garbage collected language, allocated objects have implications on latency and gc time - When measuring latency,look at percentiles not just averages - Percentiles show you tail latencies - this also helps you to understand what perf is those unlucky customers seeing (bottom 1% or 5%) Importance of Performance Measurement
  • 3. What is profiling - Pause application and capture thread stack multiple times per-second - Usually takes around 100 stack dumps per second - For java developers it is similar to running jstack (or yourkit) couple of times in a second - With profiling data we can do lot of things like, - Shows what functions are used - Can build call graphs - Find out what functions are at the top of the stack (taking CPU time)
  • 4. Viewing CPU/Memory profile snapshot - Every node is a function call - A vertex from X to Y indicates a call from X -> Y - For example in the profile snapshot below - FindLoops took 4.41s of CPU time - Remaining 30.28s were spent on outgoing function call - “web” command opens up web view of profiler
  • 5. GO Profiling tools - pprof - What we can profile? - Standalone application (start with profiling enabled through commandline arg) - A live process - Using net.http.pprof, go can capture profile of a live process - Many others features available thorugh web interface like viewing passed in command line arguments, memory profile, cpu profile etc. - What you can do with profiling data - View methods that are taking more time, or allocation more objects - Annotate code with cpu/memory profile data - Slice and dice into different parts of the program for better understanding of cpu time - GO allows you to do cpu and memory profile (and blocking) - CPU allows you to look at what functions are taking cpu time - Memory profile lets you see memory allocation per function
  • 6. GO Profiling tools - benchmarking - Run test X number of times and reports the average time - Prints allocations per call - Run with multiple CPU (GOMAXPROCs) - Execute following command to get memory allocation along with runtime, - $ go test -bench=. -benchmem
  • 7. pprof commands (top) - Top command lets you view functions that are taking up most CPU time
  • 8. pprof commands (weblist) - weblist annotates source code with profiler data (and assembly) in a webview - Drill down and expand each source line to see assembly instructions (pretty powerful)
  • 10. - Type of garbage collectors - STW - Stop the World - Concurrent - With 1.5, go started to have concurrent gc - This means less time spent in STW phase (~ 10ms) - Latencies improve overall - More details here - GO 1.6 does little better - GC pauses is even lower - See here for details Garbage Collector
  • 11. - Simplicity - core principle - GOGC - the only parameter that you’d ever to tune - Defaults to 100%, which really means that your heap size after garbage collection will be kept at - gctrace (GODEBUG=gctrace=1 commandline) - Go program will start writing detailed information about GC on stdout - Helpful in debuggin whether GC is the cause of latency or the not - GOMAXPROC - This sets the maximum number of processes GO process can use - As of go1.5, there isnt any need to set this as runtime figures it out automatically GO Performance Tuning (GC)
  • 12. Summary - Be judicious when allocating new objects - See if you can use simpler data structures (e.g. slice instead of map) - Reuse objects if possible (connection pooling, objects cache etc) - Measure latencies in percentiles - Enable web pprof in our application - Doesnt cost much, and lets you take traces of live process - For all users of 1.3, upgrade to 1.6 - Concurrent GC - Less time spent in GC, more time for the app (mutator) - Play with GOGC and gcdebug parameter
  • 13. Links and further reading - GO 1.5 gc release notes - https://blog.golang.org/go15gc - Concurrent vs stop the world gc - https://talks.golang.org/2015/go-gc.pdf - Testing and benchmarking - https://golang.org/pkg/testing/ - Running pprof from http - https://golang.org/pkg/net/http/pprof/ - Profiling GO Programs - https://blog.golang.org/profiling-go-programs - http://www.stuartcheshire.org/rants/Latency.html