SlideShare a Scribd company logo
1 of 22
Performance Analysis Using
Automatic Grouping
July 18th, 2018
Abderrahmane Benbachir
École Polytechnique de Montréal
Laboratoire DORSAL
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
● Intro
● Related Work
● Methodology
● Use Cases
● Discussion
● Conclusion
Outline
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
● Continuous Integration
● Regression tests in several builds
● Performance evaluation
Intro
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
It is the framework proposed to compare traces using the ECCT and
the critical path. It shows the recorded metrics for a trained
user to find the reason of this specific delay.
Doray, F, and M. R. Dagenais, "Diagnosing Performance Variations by Comparing
Multi-Level Execution Traces", IEEE Transactions on Parallel and Distributed Systems,
vol. pp,issue: 99 no. 1, 2016.
Related Work
TraceCompare
POLYTECHNIQUE MONTREAL – Francisco de Melo
Related Work
TraceCompare
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
POLYTECHNIQUE MONTREAL – Francisco de Melo
● Select groups of comparison manually
● Manual comparison task
● Only two groups of comparison
TraceCompare
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
Related Work
POLYTECHNIQUE MONTREAL – Francisco de Melo
● Comparative visualization tool
● CCTs
● Several Mathematical applications
Differential Flame
Graph
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
Related Work
POLYTECHNIQUE MONTREAL – Francisco de Melo
Overview
● Instrument the code
● Run the software with tracing enabled
● Create the tree
● Apply the classification methods
● Compare the groups
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
Methodology
POLYTECHNIQUE MONTREAL – Francisco de MeloPOLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
Methodology - Data Model
POLYTECHNIQUE MONTREAL – Francisco de Melo
Calling Context Model
Construction of tree using lttng and walking technique
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
Methodology - Data Model
POLYTECHNIQUE MONTREAL – Francisco de Melo
● Instrument the source code
● This will enable recording profiling data which will then be saved
together with each function
● Trace with LTTng and Perf Tools
Metrics = instructions, page faults, cache misses
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
Methodology -
Instrumentation
POLYTECHNIQUE MONTREAL – Francisco de Melo
Classification
● Classifying a set of metrics by using a non-supervised method
(e.g. k-means and elbow method)
● Analysis of the classification
● Use the result to deduce the root cause
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
Methodology - Analyzing the data
POLYTECHNIQUE MONTREAL – Francisco de Melo
Methodology - Techniques
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
Elbow heuristic
Find the best k
Compare SSE (sum of squared errors)
Comparison: find the biggest gap → starting from 2
POLYTECHNIQUE MONTREAL – Francisco de Melo
Methodology - Grouping
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
POLYTECHNIQUE MONTREAL – Francisco de Melo
Use Case
Cache Optimization
Php Requests
After several executions (100 times), the performance decreased
Periodic problem
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
POLYTECHNIQUE MONTREAL – Francisco de Melo
Use Case
Cache Optimization
Grouping results relating the caching with the slow executions groups
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
POLYTECHNIQUE MONTREAL – Francisco de Melo
Use Case
Software Regression (OpenCV)
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
POLYTECHNIQUE MONTREAL – Francisco de Melo
Use Case
Software Regression (OpenCV)
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
Pearson correlation (R) of the variables shows that many metrics are directly
proportional. This mean that the development of linear or multilinear models
using those metrics is not adequate and consequently the comparison of
groups, in a pair-wise approach, can be applied.
Correlation among metrics
POLYTECHNIQUE MONTREAL – Francisco de Melo
The differences on each group can indicate normal performance
variations but also can highlight abnormal metric variations
Knowing the abnormal metric and with some knowledge of the
source code, it is possible to find performance regressions
Conclusion
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
POLYTECHNIQUE MONTREAL – Francisco de Melo
Questions
isnaldo-francisco.de-melo-junior@polymtl.ca
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
POLYTECHNIQUE MONTREAL – Francisco de Melo
Want to learn more?
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
POLYTECHNIQUE MONTREAL – Francisco de Melo
Doray, F, and M. R. Dagenais, "Diagnosing Performance Variations by Comparing Multi-
Level Execution Traces", IEEE Transactions on Parallel and Distributed Systems, vol.
pp,issue: 99 no. 1, 2016.
Andrea Adamoli and Matthias Hauswirth. Trevis: A context
tree visualization & analysis framework and its use for classifying
performance failure reports. In SoftVis ’10: Proceedings
of the ACM Symposium on Software Visualization, 2010.
J. M. Spivey. Fast, Accurate Call Graph Profiling. Softw.
Pract. Exper., 34(3):249–264, 2004.
W. N. Sumner, Y. Zheng, D. Weeratunge, and X. Zhang. Precise Calling
Context Encoding. In ACM International Conference on Software Engineering,
2010.
References
POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel

More Related Content

Similar to Performance Analysis Using Automated Grouping Mechanisms - Conference Portugal, 2018.

Electrónica: Diseño y construcción de un prototipo programable orientado la a...
Electrónica: Diseño y construcción de un prototipo programable orientado la a...Electrónica: Diseño y construcción de un prototipo programable orientado la a...
Electrónica: Diseño y construcción de un prototipo programable orientado la a...SANTIAGO PABLO ALBERTO
 
2015-07-08 Paper 38 - ICVS Talk
2015-07-08 Paper 38 - ICVS Talk2015-07-08 Paper 38 - ICVS Talk
2015-07-08 Paper 38 - ICVS TalkThomas Sølund
 
AI-SDV 2022: Accommodating the Deep Learning Revolution by a Development Proc...
AI-SDV 2022: Accommodating the Deep Learning Revolution by a Development Proc...AI-SDV 2022: Accommodating the Deep Learning Revolution by a Development Proc...
AI-SDV 2022: Accommodating the Deep Learning Revolution by a Development Proc...Dr. Haxel Consult
 
Thoreson, Scott Resume
Thoreson, Scott ResumeThoreson, Scott Resume
Thoreson, Scott ResumeScott Thoreson
 
Accelerating the Design of Optical Networks using Surrogate Models
Accelerating the Design of Optical Networks using Surrogate ModelsAccelerating the Design of Optical Networks using Surrogate Models
Accelerating the Design of Optical Networks using Surrogate ModelsCPqD
 
Operations Research Digital Material.pdf
Operations Research Digital Material.pdfOperations Research Digital Material.pdf
Operations Research Digital Material.pdfTANVEERSINGHSOLANKI
 
ZaneRicksResume8-14
ZaneRicksResume8-14ZaneRicksResume8-14
ZaneRicksResume8-14Zane Ricks
 
The Green Lab - Experimentation in Software Energy Efficiency (ICSE)
The Green Lab - Experimentation in Software Energy Efficiency (ICSE)The Green Lab - Experimentation in Software Energy Efficiency (ICSE)
The Green Lab - Experimentation in Software Energy Efficiency (ICSE)Giuseppe Procaccianti
 
Recommendation engine and uncertainty techniques for the efficient calibratio...
Recommendation engine and uncertainty techniques for the efficient calibratio...Recommendation engine and uncertainty techniques for the efficient calibratio...
Recommendation engine and uncertainty techniques for the efficient calibratio...Vform Xsteels
 
VictorMohmmadjaludM._Revision-Matrix.docx
VictorMohmmadjaludM._Revision-Matrix.docxVictorMohmmadjaludM._Revision-Matrix.docx
VictorMohmmadjaludM._Revision-Matrix.docxEugene Embalzado
 
On the topology design of an innovative heterogeneous mechanical test for mat...
On the topology design of an innovative heterogeneous mechanical test for mat...On the topology design of an innovative heterogeneous mechanical test for mat...
On the topology design of an innovative heterogeneous mechanical test for mat...vformxsteels
 
On the topology design of an innovative heterogeneous mechanical test for mat...
On the topology design of an innovative heterogeneous mechanical test for mat...On the topology design of an innovative heterogeneous mechanical test for mat...
On the topology design of an innovative heterogeneous mechanical test for mat...vformxsteels
 
Analysis of Educational Robotics activities using a machine learning approach
Analysis of Educational Robotics activities using a machine learning approachAnalysis of Educational Robotics activities using a machine learning approach
Analysis of Educational Robotics activities using a machine learning approachLorenzo Cesaretti
 
Dbms lab manual
Dbms lab manualDbms lab manual
Dbms lab manualArthyR3
 

Similar to Performance Analysis Using Automated Grouping Mechanisms - Conference Portugal, 2018. (20)

Icsoc16b.ppt
Icsoc16b.pptIcsoc16b.ppt
Icsoc16b.ppt
 
Icsoc16b.ppt
Icsoc16b.pptIcsoc16b.ppt
Icsoc16b.ppt
 
Electrónica: Diseño y construcción de un prototipo programable orientado la a...
Electrónica: Diseño y construcción de un prototipo programable orientado la a...Electrónica: Diseño y construcción de un prototipo programable orientado la a...
Electrónica: Diseño y construcción de un prototipo programable orientado la a...
 
2015-07-08 Paper 38 - ICVS Talk
2015-07-08 Paper 38 - ICVS Talk2015-07-08 Paper 38 - ICVS Talk
2015-07-08 Paper 38 - ICVS Talk
 
AI-SDV 2022: Accommodating the Deep Learning Revolution by a Development Proc...
AI-SDV 2022: Accommodating the Deep Learning Revolution by a Development Proc...AI-SDV 2022: Accommodating the Deep Learning Revolution by a Development Proc...
AI-SDV 2022: Accommodating the Deep Learning Revolution by a Development Proc...
 
Thoreson, Scott Resume
Thoreson, Scott ResumeThoreson, Scott Resume
Thoreson, Scott Resume
 
Accelerating the Design of Optical Networks using Surrogate Models
Accelerating the Design of Optical Networks using Surrogate ModelsAccelerating the Design of Optical Networks using Surrogate Models
Accelerating the Design of Optical Networks using Surrogate Models
 
Operations Research Digital Material.pdf
Operations Research Digital Material.pdfOperations Research Digital Material.pdf
Operations Research Digital Material.pdf
 
ZaneRicksResume8-14
ZaneRicksResume8-14ZaneRicksResume8-14
ZaneRicksResume8-14
 
2021 ME.pdf
2021 ME.pdf2021 ME.pdf
2021 ME.pdf
 
RESUME Fausto Romero M
RESUME Fausto Romero MRESUME Fausto Romero M
RESUME Fausto Romero M
 
The Green Lab - Experimentation in Software Energy Efficiency (ICSE)
The Green Lab - Experimentation in Software Energy Efficiency (ICSE)The Green Lab - Experimentation in Software Energy Efficiency (ICSE)
The Green Lab - Experimentation in Software Energy Efficiency (ICSE)
 
Recommendation engine and uncertainty techniques for the efficient calibratio...
Recommendation engine and uncertainty techniques for the efficient calibratio...Recommendation engine and uncertainty techniques for the efficient calibratio...
Recommendation engine and uncertainty techniques for the efficient calibratio...
 
VictorMohmmadjaludM._Revision-Matrix.docx
VictorMohmmadjaludM._Revision-Matrix.docxVictorMohmmadjaludM._Revision-Matrix.docx
VictorMohmmadjaludM._Revision-Matrix.docx
 
On the topology design of an innovative heterogeneous mechanical test for mat...
On the topology design of an innovative heterogeneous mechanical test for mat...On the topology design of an innovative heterogeneous mechanical test for mat...
On the topology design of an innovative heterogeneous mechanical test for mat...
 
On the topology design of an innovative heterogeneous mechanical test for mat...
On the topology design of an innovative heterogeneous mechanical test for mat...On the topology design of an innovative heterogeneous mechanical test for mat...
On the topology design of an innovative heterogeneous mechanical test for mat...
 
Analysis of Educational Robotics activities using a machine learning approach
Analysis of Educational Robotics activities using a machine learning approachAnalysis of Educational Robotics activities using a machine learning approach
Analysis of Educational Robotics activities using a machine learning approach
 
A1802050102
A1802050102A1802050102
A1802050102
 
Dbms lab manual
Dbms lab manualDbms lab manual
Dbms lab manual
 
02 stein intro_4th-pv_modeling_workshop_2015-10-22_sand2015-8571_c
02 stein intro_4th-pv_modeling_workshop_2015-10-22_sand2015-8571_c02 stein intro_4th-pv_modeling_workshop_2015-10-22_sand2015-8571_c
02 stein intro_4th-pv_modeling_workshop_2015-10-22_sand2015-8571_c
 

More from Francisco de Melo Jr

More from Francisco de Melo Jr (11)

Presentation infonuagique/ cloud computing presentation
Presentation infonuagique/ cloud computing presentationPresentation infonuagique/ cloud computing presentation
Presentation infonuagique/ cloud computing presentation
 
Nutri App
Nutri AppNutri App
Nutri App
 
NeuroGames - TCC Mackenzie Univ
NeuroGames - TCC Mackenzie UnivNeuroGames - TCC Mackenzie Univ
NeuroGames - TCC Mackenzie Univ
 
Honour Thesis Gabriel Alabarse - TCC Anhembi Morumbi
Honour Thesis Gabriel Alabarse - TCC Anhembi MorumbiHonour Thesis Gabriel Alabarse - TCC Anhembi Morumbi
Honour Thesis Gabriel Alabarse - TCC Anhembi Morumbi
 
Metaheuristics for software testing
Metaheuristics for software testingMetaheuristics for software testing
Metaheuristics for software testing
 
Automata research developed at Saint Mary's 2014
Automata research developed at Saint Mary's 2014Automata research developed at Saint Mary's 2014
Automata research developed at Saint Mary's 2014
 
Comparative Framework for Education
Comparative Framework for EducationComparative Framework for Education
Comparative Framework for Education
 
Dreams
DreamsDreams
Dreams
 
Run time
Run timeRun time
Run time
 
Apresentação TDC Floripa 2014
Apresentação TDC Floripa 2014Apresentação TDC Floripa 2014
Apresentação TDC Floripa 2014
 
Projeto mpx
Projeto mpxProjeto mpx
Projeto mpx
 

Recently uploaded

Alluxio Monthly Webinar | Simplify Data Access for AI in Multi-Cloud
Alluxio Monthly Webinar | Simplify Data Access for AI in Multi-CloudAlluxio Monthly Webinar | Simplify Data Access for AI in Multi-Cloud
Alluxio Monthly Webinar | Simplify Data Access for AI in Multi-CloudAlluxio, Inc.
 
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypseTomasz Kowalczewski
 
Lessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfLessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfSrushith Repakula
 
BusinessGPT - Security and Governance for Generative AI
BusinessGPT  - Security and Governance for Generative AIBusinessGPT  - Security and Governance for Generative AI
BusinessGPT - Security and Governance for Generative AIAGATSoftware
 
From Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST APIFrom Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST APIInflectra
 
What Goes Wrong with Language Definitions and How to Improve the Situation
What Goes Wrong with Language Definitions and How to Improve the SituationWhat Goes Wrong with Language Definitions and How to Improve the Situation
What Goes Wrong with Language Definitions and How to Improve the SituationJuha-Pekka Tolvanen
 
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...drm1699
 
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Andreas Granig
 
The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)Roberto Bettazzoni
 
Workshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit Milan
Workshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit MilanWorkshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit Milan
Workshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit MilanNeo4j
 
Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?Maxim Salnikov
 

Recently uploaded (20)

Abortion Clinic In Pretoria ](+27832195400*)[ 🏥 Safe Abortion Pills in Pretor...
Abortion Clinic In Pretoria ](+27832195400*)[ 🏥 Safe Abortion Pills in Pretor...Abortion Clinic In Pretoria ](+27832195400*)[ 🏥 Safe Abortion Pills in Pretor...
Abortion Clinic In Pretoria ](+27832195400*)[ 🏥 Safe Abortion Pills in Pretor...
 
Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...
Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...
Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...
 
Alluxio Monthly Webinar | Simplify Data Access for AI in Multi-Cloud
Alluxio Monthly Webinar | Simplify Data Access for AI in Multi-CloudAlluxio Monthly Webinar | Simplify Data Access for AI in Multi-Cloud
Alluxio Monthly Webinar | Simplify Data Access for AI in Multi-Cloud
 
Abortion Clinic in Midrand [(+27832195400*)]🏥Safe Abortion Pills In Midrand |...
Abortion Clinic in Midrand [(+27832195400*)]🏥Safe Abortion Pills In Midrand |...Abortion Clinic in Midrand [(+27832195400*)]🏥Safe Abortion Pills In Midrand |...
Abortion Clinic in Midrand [(+27832195400*)]🏥Safe Abortion Pills In Midrand |...
 
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
 
Abortion Clinic In Pongola ](+27832195400*)[ 🏥 Safe Abortion Pills In Pongola...
Abortion Clinic In Pongola ](+27832195400*)[ 🏥 Safe Abortion Pills In Pongola...Abortion Clinic In Pongola ](+27832195400*)[ 🏥 Safe Abortion Pills In Pongola...
Abortion Clinic In Pongola ](+27832195400*)[ 🏥 Safe Abortion Pills In Pongola...
 
Lessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfLessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdf
 
BusinessGPT - Security and Governance for Generative AI
BusinessGPT  - Security and Governance for Generative AIBusinessGPT  - Security and Governance for Generative AI
BusinessGPT - Security and Governance for Generative AI
 
From Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST APIFrom Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST API
 
What Goes Wrong with Language Definitions and How to Improve the Situation
What Goes Wrong with Language Definitions and How to Improve the SituationWhat Goes Wrong with Language Definitions and How to Improve the Situation
What Goes Wrong with Language Definitions and How to Improve the Situation
 
Abortion Pill Prices Jane Furse ](+27832195400*)[🏥Women's Abortion Clinic in ...
Abortion Pill Prices Jane Furse ](+27832195400*)[🏥Women's Abortion Clinic in ...Abortion Pill Prices Jane Furse ](+27832195400*)[🏥Women's Abortion Clinic in ...
Abortion Pill Prices Jane Furse ](+27832195400*)[🏥Women's Abortion Clinic in ...
 
Abortion Pill Prices Germiston ](+27832195400*)[ 🏥 Women's Abortion Clinic in...
Abortion Pill Prices Germiston ](+27832195400*)[ 🏥 Women's Abortion Clinic in...Abortion Pill Prices Germiston ](+27832195400*)[ 🏥 Women's Abortion Clinic in...
Abortion Pill Prices Germiston ](+27832195400*)[ 🏥 Women's Abortion Clinic in...
 
Abortion Pill Prices Polokwane ](+27832195400*)[ 🏥 Women's Abortion Clinic in...
Abortion Pill Prices Polokwane ](+27832195400*)[ 🏥 Women's Abortion Clinic in...Abortion Pill Prices Polokwane ](+27832195400*)[ 🏥 Women's Abortion Clinic in...
Abortion Pill Prices Polokwane ](+27832195400*)[ 🏥 Women's Abortion Clinic in...
 
Abortion Pill Prices Rustenburg [(+27832195400*)] 🏥 Women's Abortion Clinic i...
Abortion Pill Prices Rustenburg [(+27832195400*)] 🏥 Women's Abortion Clinic i...Abortion Pill Prices Rustenburg [(+27832195400*)] 🏥 Women's Abortion Clinic i...
Abortion Pill Prices Rustenburg [(+27832195400*)] 🏥 Women's Abortion Clinic i...
 
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
 
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
 
The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)
 
Workshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit Milan
Workshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit MilanWorkshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit Milan
Workshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit Milan
 
Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?
 
Abortion Clinic In Stanger ](+27832195400*)[ 🏥 Safe Abortion Pills In Stanger...
Abortion Clinic In Stanger ](+27832195400*)[ 🏥 Safe Abortion Pills In Stanger...Abortion Clinic In Stanger ](+27832195400*)[ 🏥 Safe Abortion Pills In Stanger...
Abortion Clinic In Stanger ](+27832195400*)[ 🏥 Safe Abortion Pills In Stanger...
 

Performance Analysis Using Automated Grouping Mechanisms - Conference Portugal, 2018.

  • 1. Performance Analysis Using Automatic Grouping July 18th, 2018 Abderrahmane Benbachir École Polytechnique de Montréal Laboratoire DORSAL
  • 2. POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel ● Intro ● Related Work ● Methodology ● Use Cases ● Discussion ● Conclusion Outline
  • 3. POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel ● Continuous Integration ● Regression tests in several builds ● Performance evaluation Intro
  • 4. POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel It is the framework proposed to compare traces using the ECCT and the critical path. It shows the recorded metrics for a trained user to find the reason of this specific delay. Doray, F, and M. R. Dagenais, "Diagnosing Performance Variations by Comparing Multi-Level Execution Traces", IEEE Transactions on Parallel and Distributed Systems, vol. pp,issue: 99 no. 1, 2016. Related Work TraceCompare
  • 5. POLYTECHNIQUE MONTREAL – Francisco de Melo Related Work TraceCompare POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  • 6. POLYTECHNIQUE MONTREAL – Francisco de Melo ● Select groups of comparison manually ● Manual comparison task ● Only two groups of comparison TraceCompare POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Related Work
  • 7. POLYTECHNIQUE MONTREAL – Francisco de Melo ● Comparative visualization tool ● CCTs ● Several Mathematical applications Differential Flame Graph POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Related Work
  • 8. POLYTECHNIQUE MONTREAL – Francisco de Melo Overview ● Instrument the code ● Run the software with tracing enabled ● Create the tree ● Apply the classification methods ● Compare the groups POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Methodology
  • 9. POLYTECHNIQUE MONTREAL – Francisco de MeloPOLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Methodology - Data Model
  • 10. POLYTECHNIQUE MONTREAL – Francisco de Melo Calling Context Model Construction of tree using lttng and walking technique POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Methodology - Data Model
  • 11. POLYTECHNIQUE MONTREAL – Francisco de Melo ● Instrument the source code ● This will enable recording profiling data which will then be saved together with each function ● Trace with LTTng and Perf Tools Metrics = instructions, page faults, cache misses POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Methodology - Instrumentation
  • 12. POLYTECHNIQUE MONTREAL – Francisco de Melo Classification ● Classifying a set of metrics by using a non-supervised method (e.g. k-means and elbow method) ● Analysis of the classification ● Use the result to deduce the root cause POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Methodology - Analyzing the data
  • 13. POLYTECHNIQUE MONTREAL – Francisco de Melo Methodology - Techniques POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Elbow heuristic Find the best k Compare SSE (sum of squared errors) Comparison: find the biggest gap → starting from 2
  • 14. POLYTECHNIQUE MONTREAL – Francisco de Melo Methodology - Grouping POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  • 15. POLYTECHNIQUE MONTREAL – Francisco de Melo Use Case Cache Optimization Php Requests After several executions (100 times), the performance decreased Periodic problem POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  • 16. POLYTECHNIQUE MONTREAL – Francisco de Melo Use Case Cache Optimization Grouping results relating the caching with the slow executions groups POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  • 17. POLYTECHNIQUE MONTREAL – Francisco de Melo Use Case Software Regression (OpenCV) POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  • 18. POLYTECHNIQUE MONTREAL – Francisco de Melo Use Case Software Regression (OpenCV) POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Pearson correlation (R) of the variables shows that many metrics are directly proportional. This mean that the development of linear or multilinear models using those metrics is not adequate and consequently the comparison of groups, in a pair-wise approach, can be applied. Correlation among metrics
  • 19. POLYTECHNIQUE MONTREAL – Francisco de Melo The differences on each group can indicate normal performance variations but also can highlight abnormal metric variations Knowing the abnormal metric and with some knowledge of the source code, it is possible to find performance regressions Conclusion POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  • 20. POLYTECHNIQUE MONTREAL – Francisco de Melo Questions isnaldo-francisco.de-melo-junior@polymtl.ca POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  • 21. POLYTECHNIQUE MONTREAL – Francisco de Melo Want to learn more? POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  • 22. POLYTECHNIQUE MONTREAL – Francisco de Melo Doray, F, and M. R. Dagenais, "Diagnosing Performance Variations by Comparing Multi- Level Execution Traces", IEEE Transactions on Parallel and Distributed Systems, vol. pp,issue: 99 no. 1, 2016. Andrea Adamoli and Matthias Hauswirth. Trevis: A context tree visualization & analysis framework and its use for classifying performance failure reports. In SoftVis ’10: Proceedings of the ACM Symposium on Software Visualization, 2010. J. M. Spivey. Fast, Accurate Call Graph Profiling. Softw. Pract. Exper., 34(3):249–264, 2004. W. N. Sumner, Y. Zheng, D. Weeratunge, and X. Zhang. Precise Calling Context Encoding. In ACM International Conference on Software Engineering, 2010. References POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel