SlideShare a Scribd company logo
1 of 31
Download to read offline
Node.js Community
Benchmarking Efforts
Michael Dawson, IBM
About Michael Dawson
Loves the web and building software (with Node.js!)
Senior Software Developer @ IBM
IBM Runtime Technologies Node.js Technical Lead
Node.js collaborator and CTC member
Active in LTS, build, benchmarking , api
and post-mortem working groups
Contact me:
michael_dawson@ca.ibm.com
Twitter: @mhdawson1
https://www.linkedin.com/in/michael-dawson-6051282
4
About The Benchmarking Team
Love benchmarks and making things go faster.
5
Introduction to benchmarking
Node benchmarking workgroup
Real-life example
Agenda
Performance testing
Functional testing
6
Introduction
=
Fundamental run-to-run variance
– False positives
– Collecting enough samples to be sure of the result
– Documenting expected variance
Consistent environment
– Known starting machine state
– Machine isolation
– Interleave comparison runs
Jumping to conclusions
7
Introduction – Key Challenges
Change one thing and one thing only between runs
–Application code
–NPM module
–Runtime
–Machine
8
Introduction – Limit Entropy
9
Micro-Benchmarks
–Ex: Buffer.new()
Whole System
–Ex: AcmeAir - http://github.com/acmeair/acmeair-nodejs
Introduction – Approaches
I’ve found a regression in
…. now what ?
10
Introduction - Regressions
11
Introduction - Regressions
Are you sure ?
– Revalidate environment,
– Expected variance
If so, what changed ?
– Your application
– Node.js
– Your environment
Compare between good/bad cases
– Tools
– Binary search
12
Introduction – Sources of Regressions
Node.js
 lib/*.js – buffer, cluster, etc
V8
OpenSSL
libuv
NPM Module
13
Introduction – Tools
Javascript profiler
 V8 profiler
 Appmetrics - https://www.npmjs.com/package/appmetrics
 --trace-opt --trace-deopt
Native profiler (ex perf, tprof, oprofile)
14
Benchmark Workgroup
 Key goals
– Avoid Regressions
– Track and Evangelize gains between
releases
 How ?
 Define Use Cases
 Identify/Build Benchmarks
 Run/Capture results
https://github.com/nodejs/benchmarking
15
Benchmark Workgroup – Define Use Cases
https://github.com/nodejs/benchmarking/blob/master/docs/use_cases.md
16
Benchmark Workgroup – Runtime Attributes
https://github.com/nodejs/benchmarking/blob/master/docs/runtime_attributes.md
17
Benchmark Workgroup – Build/Identify Benchmarks
https://github.com/nodejs/benchmarking/blob/master/docs/case_coverage.md
18
Benchmark Workgroup – Build/Identify Benchmarks
https://github.com/nodejs/benchmarking/blob/master/docs/case_coverage.md
19
Benchmark Workgroup – Run/Capture Results
https://ci.nodejs.org/view/Node.js%20benchmark/
20
Benchmark Workgroup – Run/Capture Results
https://benchmarking.nodejs.org/
21
Benchmark Workgroup – Current Work
Expand benchmarks
Job -> run core benchmarks
 https://github.com/nodejs/benchmarking/pull/58
22
Example – The Code
var harness = require('../../common/harness.js');
var ARRAY = [1, 2, 23829, 4, 5, 7, 12312321, 2131, 434832, 43792, 23421, 65345, 132210,
77777, 322131, 1, 2, 23829, 4, 5, 7, 12312321, 2131, 434832, 43792, 23421, 65345, 132210,
77777, 322131, 1, 2, 23829, 4, 5, 7, 12312321, 2131, 434832, 43792, 23421, 65345, 132210,
77777, 322131, 1, 2, 23829, 4, 5, 7, 12312321, 2131, 434832, 43792, 23421, 65345, 132210,
77777, 322131];
var ITERATIONS = 300000;
var result;
function test() {
for(var i=0;i<ITERATIONS;i++) {
result = new Buffer(ARRAY);
}
}
harness.run_test(test);
23
Example – The Problem
Node 4.3.2:
./node benchmark.js
total time:5.079s / iterations:54 / ops/sec:10.63 / average time:0.09s /
variance:0.89%
total time:5.076s / iterations:54 / ops/sec:10.64 / average time:0.09s /
variance:0.75%
Node 4.4.0:
./node benchmark.js
total time:5.131s / iterations:31 / ops/sec:6.04 / average time:0.17s /
variance: 2.32%
total time:5.106s / iterations:31 / ops/sec:6.07 / average time:0.16s /
variance: 0.28%
= ~ 40% regression
24
Example – V8 Profiler
 Part of Node.js binary
 Turn on with
– --prof
 Test-tick-process to post-process
– ./node --prof-process isolate-0x2818130-v8.log
 Other helper modules like
– https://www.npmjs.com/package/v8-profiler
25
Example – V8 Profiler
< 5585 23.7% 23.9% LazyCompile: *fromObject buffer.js:121:20
< 1308 5.6% 5.6% LazyCompile: *subarray native typedarray.js:165:28
< 1263 5.4% 5.4% LazyCompile: *Uint8ArrayConstructByArrayBuffer native
typedarray.js:35:42
< 964 4.1% 4.1% Builtin: JSConstructStubGeneric
< 854 3.6% 3.7% Stub: InstanceofStub
< 677 2.9% 2.9% LazyCompile: *test benchmark.js:7:14
< 669 2.8% 2.9% LazyCompile: *Uint8Array native typedarray.js:122:31
---
> 15227 47.1% 47.3% LazyCompile: *fromObject buffer.js:121:20
> 1240 3.8% 3.9% LazyCompile: *subarray native typedarray.js:165:28
> 1166 3.6% 3.6% LazyCompile: *Uint8ArrayConstructByArrayBuffer native
typedarray.js:35:42
> 967 3.0% 3.0% Builtin: JSConstructStubGeneric
> 802 2.5% 2.5% Stub: InstanceofStub
> 780 2.4% 2.4% LazyCompile: *test benchmark.js:7:14
> 654 2.0% 2.0% LazyCompile: *Uint8Array native typedarray.js:122:31
26
Example – Perf
perf record -i -g -e cycles:u -- ./node --perf-basic-prof benchmark.js
perf report
diff perf_good.out perf_bad.out
327c302
< 91.52% 23.43% node perf-16993.map [.]
LazyCompile:*fromObject buffer.js:121
---
> 93.25% 46.56% node perf-16934.map [.]
LazyCompile:*fromObject buffer.js:121
331c306
--trace-opt --trace-deopt
[marking 0x39570dd44951 <JS Function fromObject (SharedFunctionInfo 0x39570dd12f91)> for
recompilation, reason: not much type info but very hot, ICs with typeinfo: 14/64 (21%), generic ICs: 0/64
(0%)]
[compiling method 0x39570dd44951 <JS Function fromObject (SharedFunctionInfo
0x39570dd12f91)> using Crankshaft]
[optimizing 0x39570dd44951 <JS Function fromObject (SharedFunctionInfo 0x39570dd12f91)> - took
0.315, 1.339, 0.511 ms]
[completed optimizing 0x39570dd44951 <JS Function fromObject (SharedFunctionInfo
0x39570dd12f91)>]
[deoptimizing (DEOPT eager): begin 0x39570dd44951 <JS Function fromObject (SharedFunctionInfo
0x39570dd12f91)>
[deoptimizing (eager): end 0x39570dd44951 <JS Function fromObject (SharedFunctionInfo
0x39570dd12f91)>
28
Example – The Narrowing
Binary Chop
– Compare changes between good & bad
– git rev-list ^good bad^
– Potentially time consuming
– git bisect is quite useful
29
Example – The Result
• Issue in v8 optimiser.
• Will be fixed there once TurboFan
becomes default
• Until then…
• github.com/nodejs/node/pull/5819
Get Involved !
 http://www.github.com/nodejs/benchmarking
We need to expand coverage
Need help in tracking perf and investigating issues
28
Copyrights and Trademarks
© IBM Corporation 2016. All Rights Reserved
IBM, the IBM logo, ibm.com are trademarks or registered
trademarks of International Business Machines Corp.,
registered in many jurisdictions worldwide. Other product and
service names might be trademarks of IBM or other companies.
A current list of IBM trademarks is available on the Web at
“Copyright and trademark information” at
www.ibm.com/legal/copytrade.shtml
Node.js is an official trademark of Joyent. IBM SDK for Node.js is not formally related to
or endorsed by the official Joyent Node.js open source or commercial project.
Java, JavaScript and all Java-based trademarks and logos are trademarks or registered
trademarks of Oracle and/or its affiliates.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or
both.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft
Corporation in the United States, other countries, or both.
npm is a trademark of npm, Inc.

More Related Content

Viewers also liked

Demystify big data data science
Demystify big data  data scienceDemystify big data  data science
Demystify big data data scienceMahesh Kumar CV
 
Cloud foundry as driver of hana’s evolution
Cloud foundry as driver of hana’s evolutionCloud foundry as driver of hana’s evolution
Cloud foundry as driver of hana’s evolutionJan Penninkhof
 
Introduction to python for Beginners
Introduction to python for Beginners Introduction to python for Beginners
Introduction to python for Beginners Sujith Kumar
 
Linux Introduction (Commands)
Linux Introduction (Commands)Linux Introduction (Commands)
Linux Introduction (Commands)anandvaidya
 
Node Foundation Membership Overview 20160907
Node Foundation Membership Overview 20160907Node Foundation Membership Overview 20160907
Node Foundation Membership Overview 20160907NodejsFoundation
 
The Enterprise Case for Node.js
The Enterprise Case for Node.jsThe Enterprise Case for Node.js
The Enterprise Case for Node.jsNodejsFoundation
 
Introduction to Python
Introduction to PythonIntroduction to Python
Introduction to PythonNowell Strite
 

Viewers also liked (8)

Sap Teched 2015 Recap
Sap Teched 2015 RecapSap Teched 2015 Recap
Sap Teched 2015 Recap
 
Demystify big data data science
Demystify big data  data scienceDemystify big data  data science
Demystify big data data science
 
Cloud foundry as driver of hana’s evolution
Cloud foundry as driver of hana’s evolutionCloud foundry as driver of hana’s evolution
Cloud foundry as driver of hana’s evolution
 
Introduction to python for Beginners
Introduction to python for Beginners Introduction to python for Beginners
Introduction to python for Beginners
 
Linux Introduction (Commands)
Linux Introduction (Commands)Linux Introduction (Commands)
Linux Introduction (Commands)
 
Node Foundation Membership Overview 20160907
Node Foundation Membership Overview 20160907Node Foundation Membership Overview 20160907
Node Foundation Membership Overview 20160907
 
The Enterprise Case for Node.js
The Enterprise Case for Node.jsThe Enterprise Case for Node.js
The Enterprise Case for Node.js
 
Introduction to Python
Introduction to PythonIntroduction to Python
Introduction to Python
 

Similar to Node.js Community Benchmarking WG update

YOW2020 Linux Systems Performance
YOW2020 Linux Systems PerformanceYOW2020 Linux Systems Performance
YOW2020 Linux Systems PerformanceBrendan Gregg
 
Рахманов Александр "Что полезного в разборе дампов для .NET-разработчиков?"
Рахманов Александр "Что полезного в разборе дампов для .NET-разработчиков?"Рахманов Александр "Что полезного в разборе дампов для .NET-разработчиков?"
Рахманов Александр "Что полезного в разборе дампов для .NET-разработчиков?"Yulia Tsisyk
 
Machine learning key to your formulation challenges
Machine learning key to your formulation challengesMachine learning key to your formulation challenges
Machine learning key to your formulation challengesMarc Borowczak
 
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...DataWorks Summit
 
Comment j'ai mis ma suite de tests au régime en 5 minutes par jour
Comment j'ai mis ma suite de tests au régime en 5 minutes par jourComment j'ai mis ma suite de tests au régime en 5 minutes par jour
Comment j'ai mis ma suite de tests au régime en 5 minutes par jourCARA_Lyon
 
Performance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsPerformance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsSerge Smetana
 
Let's make this test suite run faster! SoftShake 2010
Let's make this test suite run faster! SoftShake 2010Let's make this test suite run faster! SoftShake 2010
Let's make this test suite run faster! SoftShake 2010David Gageot
 
Performance Test Driven Development with Oracle Coherence
Performance Test Driven Development with Oracle CoherencePerformance Test Driven Development with Oracle Coherence
Performance Test Driven Development with Oracle Coherencearagozin
 
Applying profilers to my sql (fosdem 2017)
Applying profilers to my sql (fosdem 2017)Applying profilers to my sql (fosdem 2017)
Applying profilers to my sql (fosdem 2017)Valeriy Kravchuk
 
Finding Bugs, Fixing Bugs, Preventing Bugs - Exploiting Automated Tests to In...
Finding Bugs, Fixing Bugs, Preventing Bugs - Exploiting Automated Tests to In...Finding Bugs, Fixing Bugs, Preventing Bugs - Exploiting Automated Tests to In...
Finding Bugs, Fixing Bugs, Preventing Bugs - Exploiting Automated Tests to In...University of Antwerp
 
MongoDB World 2019: Becoming an Ops Manager Backup Superhero!
MongoDB World 2019: Becoming an Ops Manager Backup Superhero!MongoDB World 2019: Becoming an Ops Manager Backup Superhero!
MongoDB World 2019: Becoming an Ops Manager Backup Superhero!MongoDB
 
GlobalLogic Test Automation Online TechTalk “Test Driven Development as a Per...
GlobalLogic Test Automation Online TechTalk “Test Driven Development as a Per...GlobalLogic Test Automation Online TechTalk “Test Driven Development as a Per...
GlobalLogic Test Automation Online TechTalk “Test Driven Development as a Per...GlobalLogic Ukraine
 
Test driven development in .Net - 2010 + Eclipse
Test driven development in .Net - 2010 + EclipseTest driven development in .Net - 2010 + Eclipse
Test driven development in .Net - 2010 + EclipseUTC Fire & Security
 
Testing ASP.NET - Progressive.NET
Testing ASP.NET - Progressive.NETTesting ASP.NET - Progressive.NET
Testing ASP.NET - Progressive.NETBen Hall
 
Web sphere application server performance tuning workshop
Web sphere application server performance tuning workshopWeb sphere application server performance tuning workshop
Web sphere application server performance tuning workshopRohit Kelapure
 
Windows Azure Acid Test
Windows Azure Acid TestWindows Azure Acid Test
Windows Azure Acid Testexpanz
 

Similar to Node.js Community Benchmarking WG update (20)

Java Micro-Benchmarking
Java Micro-BenchmarkingJava Micro-Benchmarking
Java Micro-Benchmarking
 
YOW2020 Linux Systems Performance
YOW2020 Linux Systems PerformanceYOW2020 Linux Systems Performance
YOW2020 Linux Systems Performance
 
Рахманов Александр "Что полезного в разборе дампов для .NET-разработчиков?"
Рахманов Александр "Что полезного в разборе дампов для .NET-разработчиков?"Рахманов Александр "Что полезного в разборе дампов для .NET-разработчиков?"
Рахманов Александр "Что полезного в разборе дампов для .NET-разработчиков?"
 
Effective Benchmarks
Effective BenchmarksEffective Benchmarks
Effective Benchmarks
 
Machine learning key to your formulation challenges
Machine learning key to your formulation challengesMachine learning key to your formulation challenges
Machine learning key to your formulation challenges
 
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
Dr Elephant: LinkedIn's Self-Service System for Detecting and Treating Hadoop...
 
Comment j'ai mis ma suite de tests au régime en 5 minutes par jour
Comment j'ai mis ma suite de tests au régime en 5 minutes par jourComment j'ai mis ma suite de tests au régime en 5 minutes par jour
Comment j'ai mis ma suite de tests au régime en 5 minutes par jour
 
Performance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsPerformance Optimization of Rails Applications
Performance Optimization of Rails Applications
 
Let's make this test suite run faster! SoftShake 2010
Let's make this test suite run faster! SoftShake 2010Let's make this test suite run faster! SoftShake 2010
Let's make this test suite run faster! SoftShake 2010
 
Performance Test Driven Development with Oracle Coherence
Performance Test Driven Development with Oracle CoherencePerformance Test Driven Development with Oracle Coherence
Performance Test Driven Development with Oracle Coherence
 
Applying profilers to my sql (fosdem 2017)
Applying profilers to my sql (fosdem 2017)Applying profilers to my sql (fosdem 2017)
Applying profilers to my sql (fosdem 2017)
 
Finding Bugs, Fixing Bugs, Preventing Bugs - Exploiting Automated Tests to In...
Finding Bugs, Fixing Bugs, Preventing Bugs - Exploiting Automated Tests to In...Finding Bugs, Fixing Bugs, Preventing Bugs - Exploiting Automated Tests to In...
Finding Bugs, Fixing Bugs, Preventing Bugs - Exploiting Automated Tests to In...
 
JavaMicroBenchmarkpptm
JavaMicroBenchmarkpptmJavaMicroBenchmarkpptm
JavaMicroBenchmarkpptm
 
MongoDB World 2019: Becoming an Ops Manager Backup Superhero!
MongoDB World 2019: Becoming an Ops Manager Backup Superhero!MongoDB World 2019: Becoming an Ops Manager Backup Superhero!
MongoDB World 2019: Becoming an Ops Manager Backup Superhero!
 
GlobalLogic Test Automation Online TechTalk “Test Driven Development as a Per...
GlobalLogic Test Automation Online TechTalk “Test Driven Development as a Per...GlobalLogic Test Automation Online TechTalk “Test Driven Development as a Per...
GlobalLogic Test Automation Online TechTalk “Test Driven Development as a Per...
 
Test driven development in .Net - 2010 + Eclipse
Test driven development in .Net - 2010 + EclipseTest driven development in .Net - 2010 + Eclipse
Test driven development in .Net - 2010 + Eclipse
 
Testing ASP.NET - Progressive.NET
Testing ASP.NET - Progressive.NETTesting ASP.NET - Progressive.NET
Testing ASP.NET - Progressive.NET
 
Web sphere application server performance tuning workshop
Web sphere application server performance tuning workshopWeb sphere application server performance tuning workshop
Web sphere application server performance tuning workshop
 
Performance_Programming
Performance_ProgrammingPerformance_Programming
Performance_Programming
 
Windows Azure Acid Test
Windows Azure Acid TestWindows Azure Acid Test
Windows Azure Acid Test
 

More from Michael Dawson

Index 2018 talk to your code
Index 2018   talk to your codeIndex 2018   talk to your code
Index 2018 talk to your codeMichael Dawson
 
Index 2018 node.js what's next
Index 2018   node.js what's nextIndex 2018   node.js what's next
Index 2018 node.js what's nextMichael Dawson
 
N api - node interactive 2017
N api - node interactive 2017N api - node interactive 2017
N api - node interactive 2017Michael Dawson
 
N api-node summit-2017-final
N api-node summit-2017-finalN api-node summit-2017-final
N api-node summit-2017-finalMichael Dawson
 
Accelerate your digital transformation
Accelerate your digital transformationAccelerate your digital transformation
Accelerate your digital transformationMichael Dawson
 
A294 fips support in node
A294  fips support in nodeA294  fips support in node
A294 fips support in nodeMichael Dawson
 
A295 nodejs-knowledge-accelerator
A295   nodejs-knowledge-acceleratorA295   nodejs-knowledge-accelerator
A295 nodejs-knowledge-acceleratorMichael Dawson
 
A301 ctu madrid2016-monitoring
A301 ctu madrid2016-monitoringA301 ctu madrid2016-monitoring
A301 ctu madrid2016-monitoringMichael Dawson
 
Post mortem talk - Node Interactive EU
Post mortem talk - Node Interactive EUPost mortem talk - Node Interactive EU
Post mortem talk - Node Interactive EUMichael Dawson
 
Update from-build-workgroup
Update from-build-workgroupUpdate from-build-workgroup
Update from-build-workgroupMichael Dawson
 
Micro app-framework - NodeLive Boston
Micro app-framework - NodeLive BostonMicro app-framework - NodeLive Boston
Micro app-framework - NodeLive BostonMichael Dawson
 
Node liveboston welcome
Node liveboston welcomeNode liveboston welcome
Node liveboston welcomeMichael Dawson
 
Node home automation with Node.js and MQTT
Node home automation with Node.js and MQTTNode home automation with Node.js and MQTT
Node home automation with Node.js and MQTTMichael Dawson
 

More from Michael Dawson (18)

Index 2018 talk to your code
Index 2018   talk to your codeIndex 2018   talk to your code
Index 2018 talk to your code
 
Index 2018 node.js what's next
Index 2018   node.js what's nextIndex 2018   node.js what's next
Index 2018 node.js what's next
 
N api - node interactive 2017
N api - node interactive 2017N api - node interactive 2017
N api - node interactive 2017
 
N api-node summit-2017-final
N api-node summit-2017-finalN api-node summit-2017-final
N api-node summit-2017-final
 
Accelerate your digital transformation
Accelerate your digital transformationAccelerate your digital transformation
Accelerate your digital transformation
 
Ask us anything v9
Ask us anything v9Ask us anything v9
Ask us anything v9
 
Cascon intro
Cascon introCascon intro
Cascon intro
 
A294 fips support in node
A294  fips support in nodeA294  fips support in node
A294 fips support in node
 
A295 nodejs-knowledge-accelerator
A295   nodejs-knowledge-acceleratorA295   nodejs-knowledge-accelerator
A295 nodejs-knowledge-accelerator
 
A301 ctu madrid2016-monitoring
A301 ctu madrid2016-monitoringA301 ctu madrid2016-monitoring
A301 ctu madrid2016-monitoring
 
Post mortem talk - Node Interactive EU
Post mortem talk - Node Interactive EUPost mortem talk - Node Interactive EU
Post mortem talk - Node Interactive EU
 
Update from-build-workgroup
Update from-build-workgroupUpdate from-build-workgroup
Update from-build-workgroup
 
Node fips
Node fipsNode fips
Node fips
 
Micro app-framework - NodeLive Boston
Micro app-framework - NodeLive BostonMicro app-framework - NodeLive Boston
Micro app-framework - NodeLive Boston
 
Node liveboston welcome
Node liveboston welcomeNode liveboston welcome
Node liveboston welcome
 
Micro app-framework
Micro app-frameworkMicro app-framework
Micro app-framework
 
Node home automation with Node.js and MQTT
Node home automation with Node.js and MQTTNode home automation with Node.js and MQTT
Node home automation with Node.js and MQTT
 
Java one 2015 - v1
Java one   2015 - v1Java one   2015 - v1
Java one 2015 - v1
 

Recently uploaded

Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMKumar Satyam
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data SciencePaolo Missier
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...panagenda
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxFIDO Alliance
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdfMuhammad Subhan
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024Lorenzo Miniero
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingScyllaDB
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityVictorSzoltysek
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxFIDO Alliance
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTopCSSGallery
 
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...SOFTTECHHUB
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...FIDO Alliance
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontologyjohnbeverley2021
 
Navigating the Large Language Model choices_Ravi Daparthi
Navigating the Large Language Model choices_Ravi DaparthiNavigating the Large Language Model choices_Ravi Daparthi
Navigating the Large Language Model choices_Ravi DaparthiRaviKumarDaparthi
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxjbellis
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Paige Cruz
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAnitaRaj43
 

Recently uploaded (20)

Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptx
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
Navigating the Large Language Model choices_Ravi Daparthi
Navigating the Large Language Model choices_Ravi DaparthiNavigating the Large Language Model choices_Ravi Daparthi
Navigating the Large Language Model choices_Ravi Daparthi
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptx
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 

Node.js Community Benchmarking WG update

  • 1.
  • 3. About Michael Dawson Loves the web and building software (with Node.js!) Senior Software Developer @ IBM IBM Runtime Technologies Node.js Technical Lead Node.js collaborator and CTC member Active in LTS, build, benchmarking , api and post-mortem working groups Contact me: michael_dawson@ca.ibm.com Twitter: @mhdawson1 https://www.linkedin.com/in/michael-dawson-6051282
  • 4. 4 About The Benchmarking Team Love benchmarks and making things go faster.
  • 5. 5 Introduction to benchmarking Node benchmarking workgroup Real-life example Agenda
  • 7. Fundamental run-to-run variance – False positives – Collecting enough samples to be sure of the result – Documenting expected variance Consistent environment – Known starting machine state – Machine isolation – Interleave comparison runs Jumping to conclusions 7 Introduction – Key Challenges
  • 8. Change one thing and one thing only between runs –Application code –NPM module –Runtime –Machine 8 Introduction – Limit Entropy
  • 9. 9 Micro-Benchmarks –Ex: Buffer.new() Whole System –Ex: AcmeAir - http://github.com/acmeair/acmeair-nodejs Introduction – Approaches
  • 10. I’ve found a regression in …. now what ? 10 Introduction - Regressions
  • 11. 11 Introduction - Regressions Are you sure ? – Revalidate environment, – Expected variance If so, what changed ? – Your application – Node.js – Your environment Compare between good/bad cases – Tools – Binary search
  • 12. 12 Introduction – Sources of Regressions Node.js  lib/*.js – buffer, cluster, etc V8 OpenSSL libuv NPM Module
  • 13. 13 Introduction – Tools Javascript profiler  V8 profiler  Appmetrics - https://www.npmjs.com/package/appmetrics  --trace-opt --trace-deopt Native profiler (ex perf, tprof, oprofile)
  • 14. 14 Benchmark Workgroup  Key goals – Avoid Regressions – Track and Evangelize gains between releases  How ?  Define Use Cases  Identify/Build Benchmarks  Run/Capture results https://github.com/nodejs/benchmarking
  • 15. 15 Benchmark Workgroup – Define Use Cases https://github.com/nodejs/benchmarking/blob/master/docs/use_cases.md
  • 16. 16 Benchmark Workgroup – Runtime Attributes https://github.com/nodejs/benchmarking/blob/master/docs/runtime_attributes.md
  • 17. 17 Benchmark Workgroup – Build/Identify Benchmarks https://github.com/nodejs/benchmarking/blob/master/docs/case_coverage.md
  • 18. 18 Benchmark Workgroup – Build/Identify Benchmarks https://github.com/nodejs/benchmarking/blob/master/docs/case_coverage.md
  • 19. 19 Benchmark Workgroup – Run/Capture Results https://ci.nodejs.org/view/Node.js%20benchmark/
  • 20. 20 Benchmark Workgroup – Run/Capture Results https://benchmarking.nodejs.org/
  • 21. 21 Benchmark Workgroup – Current Work Expand benchmarks Job -> run core benchmarks  https://github.com/nodejs/benchmarking/pull/58
  • 22. 22 Example – The Code var harness = require('../../common/harness.js'); var ARRAY = [1, 2, 23829, 4, 5, 7, 12312321, 2131, 434832, 43792, 23421, 65345, 132210, 77777, 322131, 1, 2, 23829, 4, 5, 7, 12312321, 2131, 434832, 43792, 23421, 65345, 132210, 77777, 322131, 1, 2, 23829, 4, 5, 7, 12312321, 2131, 434832, 43792, 23421, 65345, 132210, 77777, 322131, 1, 2, 23829, 4, 5, 7, 12312321, 2131, 434832, 43792, 23421, 65345, 132210, 77777, 322131]; var ITERATIONS = 300000; var result; function test() { for(var i=0;i<ITERATIONS;i++) { result = new Buffer(ARRAY); } } harness.run_test(test);
  • 23. 23 Example – The Problem Node 4.3.2: ./node benchmark.js total time:5.079s / iterations:54 / ops/sec:10.63 / average time:0.09s / variance:0.89% total time:5.076s / iterations:54 / ops/sec:10.64 / average time:0.09s / variance:0.75% Node 4.4.0: ./node benchmark.js total time:5.131s / iterations:31 / ops/sec:6.04 / average time:0.17s / variance: 2.32% total time:5.106s / iterations:31 / ops/sec:6.07 / average time:0.16s / variance: 0.28% = ~ 40% regression
  • 24. 24 Example – V8 Profiler  Part of Node.js binary  Turn on with – --prof  Test-tick-process to post-process – ./node --prof-process isolate-0x2818130-v8.log  Other helper modules like – https://www.npmjs.com/package/v8-profiler
  • 25. 25 Example – V8 Profiler < 5585 23.7% 23.9% LazyCompile: *fromObject buffer.js:121:20 < 1308 5.6% 5.6% LazyCompile: *subarray native typedarray.js:165:28 < 1263 5.4% 5.4% LazyCompile: *Uint8ArrayConstructByArrayBuffer native typedarray.js:35:42 < 964 4.1% 4.1% Builtin: JSConstructStubGeneric < 854 3.6% 3.7% Stub: InstanceofStub < 677 2.9% 2.9% LazyCompile: *test benchmark.js:7:14 < 669 2.8% 2.9% LazyCompile: *Uint8Array native typedarray.js:122:31 --- > 15227 47.1% 47.3% LazyCompile: *fromObject buffer.js:121:20 > 1240 3.8% 3.9% LazyCompile: *subarray native typedarray.js:165:28 > 1166 3.6% 3.6% LazyCompile: *Uint8ArrayConstructByArrayBuffer native typedarray.js:35:42 > 967 3.0% 3.0% Builtin: JSConstructStubGeneric > 802 2.5% 2.5% Stub: InstanceofStub > 780 2.4% 2.4% LazyCompile: *test benchmark.js:7:14 > 654 2.0% 2.0% LazyCompile: *Uint8Array native typedarray.js:122:31
  • 26. 26 Example – Perf perf record -i -g -e cycles:u -- ./node --perf-basic-prof benchmark.js perf report diff perf_good.out perf_bad.out 327c302 < 91.52% 23.43% node perf-16993.map [.] LazyCompile:*fromObject buffer.js:121 --- > 93.25% 46.56% node perf-16934.map [.] LazyCompile:*fromObject buffer.js:121 331c306
  • 27. --trace-opt --trace-deopt [marking 0x39570dd44951 <JS Function fromObject (SharedFunctionInfo 0x39570dd12f91)> for recompilation, reason: not much type info but very hot, ICs with typeinfo: 14/64 (21%), generic ICs: 0/64 (0%)] [compiling method 0x39570dd44951 <JS Function fromObject (SharedFunctionInfo 0x39570dd12f91)> using Crankshaft] [optimizing 0x39570dd44951 <JS Function fromObject (SharedFunctionInfo 0x39570dd12f91)> - took 0.315, 1.339, 0.511 ms] [completed optimizing 0x39570dd44951 <JS Function fromObject (SharedFunctionInfo 0x39570dd12f91)>] [deoptimizing (DEOPT eager): begin 0x39570dd44951 <JS Function fromObject (SharedFunctionInfo 0x39570dd12f91)> [deoptimizing (eager): end 0x39570dd44951 <JS Function fromObject (SharedFunctionInfo 0x39570dd12f91)>
  • 28. 28 Example – The Narrowing Binary Chop – Compare changes between good & bad – git rev-list ^good bad^ – Potentially time consuming – git bisect is quite useful
  • 29. 29 Example – The Result • Issue in v8 optimiser. • Will be fixed there once TurboFan becomes default • Until then… • github.com/nodejs/node/pull/5819
  • 30. Get Involved !  http://www.github.com/nodejs/benchmarking We need to expand coverage Need help in tracking perf and investigating issues 28
  • 31. Copyrights and Trademarks © IBM Corporation 2016. All Rights Reserved IBM, the IBM logo, ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml Node.js is an official trademark of Joyent. IBM SDK for Node.js is not formally related to or endorsed by the official Joyent Node.js open source or commercial project. Java, JavaScript and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. npm is a trademark of npm, Inc.