SlideShare a Scribd company logo
1 of 42
H2HC
Isnaldo Francisco de Melo Jr
and Gabriel Alabarse
Software Tracing Comparison
Using Data Mining
Techniques
October 21st, 2017
École Polytechnique de Montréal
Laboratoire DORSAL
POLYTECHNIQUE MONTREAL – Francisco de Melo
1
● -1.Jokes
● 0. Definitions
● 1. Motivation
● 2. Background
● 3. Problem definition
● 4. Research Questions
● 5. Data Model
● 6. Methodology
● 7. Use Cases
● 8. Discussion
● 9. Threats to Validity
● 10. Conclusion and Future Work
Outline
POLYTECHNIQUE MONTREAL – Francisco de Melo
2
● Poutine
● I was born in Diadema
● I actually lost my stick drive
-1. Jokes
POLYTECHNIQUE MONTREAL – Francisco de Melo
2
● Poutine
● Diadema
● I’m sorry/desole
● I actually lost my stick drive
-1. Jokes
POLYTECHNIQUE MONTREAL – Francisco de Melo
2
● Regression tests
● Tracing and Profiling
● Root Cause Analysis
● Calling Context Trees*
0. Definitions
POLYTECHNIQUE MONTREAL – Francisco de Melo
3
● Continuous Integration
● Regression tests in several builds
● Performance evaluation
1. Context and Motivation
POLYTECHNIQUE MONTREAL – Francisco de Melo
3
● Continuous Integration
● Regression tests in several builds
● Performance evaluation
1. Context and Motivation
2
POLYTECHNIQUE MONTREAL – Francisco de Melo
5
TraceCompare: It is the framework proposed to compare
traces using the ECCT and the critical path. It shows the recorded
metrics for a trained user to find the reason of this specific delay.
● Select groups of comparison manually
● Automation of the comparison task
2. Background
TraceCompare
POLYTECHNIQUE MONTREAL – Francisco de Melo
42. Background
TraceCompare
POLYTECHNIQUE MONTREAL – Francisco de Melo
6
● Select groups of comparison manually
● Manual comparison task
● Only two groups of comparison
2. Background
TraceCompare
POLYTECHNIQUE MONTREAL – Francisco de Melo
7
● Visualization tool
● CCTs
● Several Mathematical applications
2. Background
Trevis
POLYTECHNIQUE MONTREAL – Francisco de Melo
8
● Comparative visualization tool
● CCTs
● Several Mathematical applications
2. Background
Differential Flame
Graph
POLYTECHNIQUE MONTREAL – Francisco de Melo
9
Compare several executions of the same software
The current tools are limited to compare several executions of the
same software or require human analysis to find root cause
problems.
3. Problem definition
POLYTECHNIQUE MONTREAL – Francisco de Melo
104. Research Questions
3.1 Hypothesis
Is it possible to find useful information by analyzing segments
of traces from calling context trees?
3.2 Methodology
Which methods can be used to do this process?
3.3 Assumptions
1. The groupings help uncover the causes for differences
2. Only the properties within the groups explain the differences
POLYTECHNIQUE MONTREAL – Francisco de Melo
115. Data Model
Calling Context Model
Construction of tree using lttng and walking technique
POLYTECHNIQUE MONTREAL – Francisco de Melo
125. Data Model
POLYTECHNIQUE MONTREAL – Francisco de Melo
136. Methodology
Overview
● Instrument the code
● Run the software
● Create the tree
● Apply the classification methods
● Compare the groups
POLYTECHNIQUE MONTREAL – Francisco de Melo
146.1 Methodology - Instrumentation
Linear
output
Instrument the source code
This will enable recording profiling data which will then be saved
together with each function
LTTng and Perf Tools
Metrics = instructions, page faults, cache misses
POLYTECHNIQUE MONTREAL – Francisco de Melo
156.2 Methodology - Analyzing the data
6.2 Classification
6.2.1 Classifying a set of metrics by using a non-supervised
method (e.g. k-means and elbow method)
6.2.2 Analysis of the classification
6.2.3 Use the result to deduce the root cause
POLYTECHNIQUE MONTREAL – Francisco de Melo
166.3 Methodology - Mining Techniques
Presentation of two techniques
SVM
k-means (elbow heuristic)
POLYTECHNIQUE MONTREAL – Francisco de Melo
176.3 Methodology - Techniques
Support Vector Machine
Classify the data using an hyperplane
Supervised
Drawbacks
POLYTECHNIQUE MONTREAL – Francisco de Melo
186.3.1 Methodology - Techniques
k-means (elbow heuristic)
Unsupervised
k is pre-defined
optimal division in k clusters
POLYTECHNIQUE MONTREAL – Francisco de Melo
196.3.2 Methodology - Techniques
Elbow heuristic
Find the best k
Compare SSE
Comparison: find the biggest gap → starting from 2
POLYTECHNIQUE MONTREAL – Francisco de Melo
206.3.2 Methodology - Techniques
Elbow heuristic
Find the best k
Compare SSE
POLYTECHNIQUE MONTREAL – Francisco de Melo
216.4 Methodology - Example
Workload cache misses
POLYTECHNIQUE MONTREAL – Francisco de Melo
227. Use Cases
Presentation of three use cases
Inline
OpenCV
Cache Optimization
POLYTECHNIQUE MONTREAL – Francisco de Melo
237.1 Use Case 1
USE CASE 1 - Inline
POLYTECHNIQUE MONTREAL – Francisco de Melo
247.2 Use Case 2
USE CASE 2 - OpenCV
Optical Flow
Instructions difference on the groups
R correlation might not be used
POLYTECHNIQUE MONTREAL – Francisco de Melo
247.2 Use Case 2
POLYTECHNIQUE MONTREAL – Francisco de Melo
257.3 Use Case 3
USE CASE 3 - Cache Optimization
Php Requests
After several executions, the performance decreased
Periodic problem
POLYTECHNIQUE MONTREAL – Francisco de Melo
257.3 Use Case 3
USE CASE 3 - Cache Optimization
POLYTECHNIQUE MONTREAL – Francisco de Melo
268. Discussion
SVM will require a label phase
The kernels of classification can lead to misclassification
Soft classification cannot necessarily help, since a probabilistic
classification can make the performance evaluation more difficult
POLYTECHNIQUE MONTREAL – Francisco de Melo
27
Only for UST
Instrumentation required
Source code comparison is still required
Assumptions must be considered
9. Threats to Validity
POLYTECHNIQUE MONTREAL – Francisco de Melo
28
The differences on each group can indicate normal performance
variations but also can highlight abnormal metric variations
Knowing the abnormal metric and with some knowledge of the
source code, it is possible to find performance regressions
10.1 Conclusion
POLYTECHNIQUE MONTREAL – Francisco de Melo
29
Answers to the research questions
Methods evaluation
10.1 Conclusion
POLYTECHNIQUE MONTREAL – Francisco de Melo
30
Use other techniques
Combination of techniques
Automate the whole process - including source code comparison
Merge with group comparison techniques
10. 2 Future work
POLYTECHNIQUE MONTREAL – Francisco de Melo
Questions
isnaldo-francisco.de-melo-jr@polymtl.ca
Any other info?
31
POLYTECHNIQUE MONTREAL – Francisco de Melo
Merci
32
POLYTECHNIQUE MONTREAL – Francisco de Melo
Want to learn more? 32
POLYTECHNIQUE MONTREAL – Francisco de Melo
Merci
32
POLYTECHNIQUE MONTREAL – Francisco de Melo
33
Doray, F, and M. R. Dagenais, "Diagnosing Performance Variations by Comparing Multi-
Level Execution Traces", IEEE Transactions on Parallel and Distributed Systems, vol.
pp,issue: 99 no. 1, 2016.
Andrea Adamoli and Matthias Hauswirth. Trevis: A context
tree visualization & analysis framework and its use for classifying
performance failure reports. In SoftVis ’10: Proceedings
of the ACM Symposium on Software Visualization, 2010.
J. M. Spivey. Fast, Accurate Call Graph Profiling. Softw.
Pract. Exper., 34(3):249–264, 2004.
W. N. Sumner, Y. Zheng, D. Weeratunge, and X. Zhang. Precise Calling
Context Encoding. In ACM International Conference on Software Engineering,
2010.
References

More Related Content

Similar to H2HC Tracing Profiling Debugging

Student manual CSBS (2).pdf for operating systems lab
Student manual CSBS (2).pdf for operating systems labStudent manual CSBS (2).pdf for operating systems lab
Student manual CSBS (2).pdf for operating systems lab
BalamuruganV28
 

Similar to H2HC Tracing Profiling Debugging (20)

Tte 451 operations research fall 2021 part 1
Tte 451  operations research   fall 2021   part 1Tte 451  operations research   fall 2021   part 1
Tte 451 operations research fall 2021 part 1
 
B tech mechanical engineering syllabus - MITAOE
B tech mechanical engineering syllabus - MITAOEB tech mechanical engineering syllabus - MITAOE
B tech mechanical engineering syllabus - MITAOE
 
Icsoc16b.ppt
Icsoc16b.pptIcsoc16b.ppt
Icsoc16b.ppt
 
Icsoc16b.ppt
Icsoc16b.pptIcsoc16b.ppt
Icsoc16b.ppt
 
B.E. Mechatronics.pdf
B.E. Mechatronics.pdfB.E. Mechatronics.pdf
B.E. Mechatronics.pdf
 
Energy Engineering
Energy Engineering Energy Engineering
Energy Engineering
 
CS8461 Operating System Lab Manual S.Selvi
CS8461 Operating System Lab Manual S.SelviCS8461 Operating System Lab Manual S.Selvi
CS8461 Operating System Lab Manual S.Selvi
 
Student manual CSBS (2).pdf for operating systems lab
Student manual CSBS (2).pdf for operating systems labStudent manual CSBS (2).pdf for operating systems lab
Student manual CSBS (2).pdf for operating systems lab
 
OOAD & ST LAB MANUAL.pdfOose feasibility study in detail Oose feasibility stu...
OOAD & ST LAB MANUAL.pdfOose feasibility study in detail Oose feasibility stu...OOAD & ST LAB MANUAL.pdfOose feasibility study in detail Oose feasibility stu...
OOAD & ST LAB MANUAL.pdfOose feasibility study in detail Oose feasibility stu...
 
8. M.E. CIM 2017.pdf
8. M.E. CIM 2017.pdf8. M.E. CIM 2017.pdf
8. M.E. CIM 2017.pdf
 
m.e.vlsi design
m.e.vlsi designm.e.vlsi design
m.e.vlsi design
 
Sept 6 2021 BTech Artificial Intelligence and Data Science curriculum
Sept 6 2021 BTech Artificial Intelligence and Data Science curriculumSept 6 2021 BTech Artificial Intelligence and Data Science curriculum
Sept 6 2021 BTech Artificial Intelligence and Data Science curriculum
 
Practical Challenges ML Workflows
Practical Challenges ML WorkflowsPractical Challenges ML Workflows
Practical Challenges ML Workflows
 
3.1 ignacio
3.1 ignacio3.1 ignacio
3.1 ignacio
 
Recommendation engine and uncertainty techniques for the efficient calibratio...
Recommendation engine and uncertainty techniques for the efficient calibratio...Recommendation engine and uncertainty techniques for the efficient calibratio...
Recommendation engine and uncertainty techniques for the efficient calibratio...
 
B tech information technology syllabus - MITAOE
B tech information technology syllabus - MITAOEB tech information technology syllabus - MITAOE
B tech information technology syllabus - MITAOE
 
IT6511 Networks Laboratory
IT6511 Networks LaboratoryIT6511 Networks Laboratory
IT6511 Networks Laboratory
 
R2017 Syllabus anna university 2017 .pdf
R2017 Syllabus anna university 2017 .pdfR2017 Syllabus anna university 2017 .pdf
R2017 Syllabus anna university 2017 .pdf
 
Curriculum Architecture Design - November 2010
Curriculum Architecture Design - November 2010Curriculum Architecture Design - November 2010
Curriculum Architecture Design - November 2010
 
Cs8581 networks lab manual 2017
Cs8581 networks lab manual   2017Cs8581 networks lab manual   2017
Cs8581 networks lab manual 2017
 

More from Francisco de Melo Jr

More from Francisco de Melo Jr (11)

Presentation infonuagique/ cloud computing presentation
Presentation infonuagique/ cloud computing presentationPresentation infonuagique/ cloud computing presentation
Presentation infonuagique/ cloud computing presentation
 
Nutri App
Nutri AppNutri App
Nutri App
 
NeuroGames - TCC Mackenzie Univ
NeuroGames - TCC Mackenzie UnivNeuroGames - TCC Mackenzie Univ
NeuroGames - TCC Mackenzie Univ
 
Honour Thesis Gabriel Alabarse - TCC Anhembi Morumbi
Honour Thesis Gabriel Alabarse - TCC Anhembi MorumbiHonour Thesis Gabriel Alabarse - TCC Anhembi Morumbi
Honour Thesis Gabriel Alabarse - TCC Anhembi Morumbi
 
Metaheuristics for software testing
Metaheuristics for software testingMetaheuristics for software testing
Metaheuristics for software testing
 
Automata research developed at Saint Mary's 2014
Automata research developed at Saint Mary's 2014Automata research developed at Saint Mary's 2014
Automata research developed at Saint Mary's 2014
 
Comparative Framework for Education
Comparative Framework for EducationComparative Framework for Education
Comparative Framework for Education
 
Dreams
DreamsDreams
Dreams
 
Run time
Run timeRun time
Run time
 
Apresentação TDC Floripa 2014
Apresentação TDC Floripa 2014Apresentação TDC Floripa 2014
Apresentação TDC Floripa 2014
 
Projeto mpx
Projeto mpxProjeto mpx
Projeto mpx
 

Recently uploaded

Jax, FL Admin Community Group 05.14.2024 Combined Deck
Jax, FL Admin Community Group 05.14.2024 Combined DeckJax, FL Admin Community Group 05.14.2024 Combined Deck
Jax, FL Admin Community Group 05.14.2024 Combined Deck
Marc Lester
 
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
drm1699
 
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Lisi Hocke
 

Recently uploaded (20)

Jax, FL Admin Community Group 05.14.2024 Combined Deck
Jax, FL Admin Community Group 05.14.2024 Combined DeckJax, FL Admin Community Group 05.14.2024 Combined Deck
Jax, FL Admin Community Group 05.14.2024 Combined Deck
 
Auto Affiliate AI Earns First Commission in 3 Hours..pdf
Auto Affiliate  AI Earns First Commission in 3 Hours..pdfAuto Affiliate  AI Earns First Commission in 3 Hours..pdf
Auto Affiliate AI Earns First Commission in 3 Hours..pdf
 
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
 
^Clinic ^%[+27788225528*Abortion Pills For Sale In soweto
^Clinic ^%[+27788225528*Abortion Pills For Sale In soweto^Clinic ^%[+27788225528*Abortion Pills For Sale In soweto
^Clinic ^%[+27788225528*Abortion Pills For Sale In soweto
 
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
 
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
 
Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?
 
Effective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeConEffective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeCon
 
Test Automation Design Patterns_ A Comprehensive Guide.pdf
Test Automation Design Patterns_ A Comprehensive Guide.pdfTest Automation Design Patterns_ A Comprehensive Guide.pdf
Test Automation Design Patterns_ A Comprehensive Guide.pdf
 
Encryption Recap: A Refresher on Key Concepts
Encryption Recap: A Refresher on Key ConceptsEncryption Recap: A Refresher on Key Concepts
Encryption Recap: A Refresher on Key Concepts
 
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
 
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
 
Software Engineering - Introduction + Process Models + Requirements Engineering
Software Engineering - Introduction + Process Models + Requirements EngineeringSoftware Engineering - Introduction + Process Models + Requirements Engineering
Software Engineering - Introduction + Process Models + Requirements Engineering
 
Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...
Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...
Entropy, Software Quality, and Innovation (presented at Princeton Plasma Phys...
 
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
 
^Clinic ^%[+27788225528*Abortion Pills For Sale In witbank
^Clinic ^%[+27788225528*Abortion Pills For Sale In witbank^Clinic ^%[+27788225528*Abortion Pills For Sale In witbank
^Clinic ^%[+27788225528*Abortion Pills For Sale In witbank
 
Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...
Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...
Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...
 
Weeding your micro service landscape.pdf
Weeding your micro service landscape.pdfWeeding your micro service landscape.pdf
Weeding your micro service landscape.pdf
 
Abortion Clinic In Polokwane ](+27832195400*)[ 🏥 Safe Abortion Pills in Polok...
Abortion Clinic In Polokwane ](+27832195400*)[ 🏥 Safe Abortion Pills in Polok...Abortion Clinic In Polokwane ](+27832195400*)[ 🏥 Safe Abortion Pills in Polok...
Abortion Clinic In Polokwane ](+27832195400*)[ 🏥 Safe Abortion Pills in Polok...
 
A Deep Dive into Secure Product Development Frameworks.pdf
A Deep Dive into Secure Product Development Frameworks.pdfA Deep Dive into Secure Product Development Frameworks.pdf
A Deep Dive into Secure Product Development Frameworks.pdf
 

H2HC Tracing Profiling Debugging

  • 1. H2HC Isnaldo Francisco de Melo Jr and Gabriel Alabarse
  • 2. Software Tracing Comparison Using Data Mining Techniques October 21st, 2017 École Polytechnique de Montréal Laboratoire DORSAL
  • 3. POLYTECHNIQUE MONTREAL – Francisco de Melo 1 ● -1.Jokes ● 0. Definitions ● 1. Motivation ● 2. Background ● 3. Problem definition ● 4. Research Questions ● 5. Data Model ● 6. Methodology ● 7. Use Cases ● 8. Discussion ● 9. Threats to Validity ● 10. Conclusion and Future Work Outline
  • 4. POLYTECHNIQUE MONTREAL – Francisco de Melo 2 ● Poutine ● I was born in Diadema ● I actually lost my stick drive -1. Jokes
  • 5. POLYTECHNIQUE MONTREAL – Francisco de Melo 2 ● Poutine ● Diadema ● I’m sorry/desole ● I actually lost my stick drive -1. Jokes
  • 6. POLYTECHNIQUE MONTREAL – Francisco de Melo 2 ● Regression tests ● Tracing and Profiling ● Root Cause Analysis ● Calling Context Trees* 0. Definitions
  • 7. POLYTECHNIQUE MONTREAL – Francisco de Melo 3 ● Continuous Integration ● Regression tests in several builds ● Performance evaluation 1. Context and Motivation
  • 8. POLYTECHNIQUE MONTREAL – Francisco de Melo 3 ● Continuous Integration ● Regression tests in several builds ● Performance evaluation 1. Context and Motivation 2
  • 9. POLYTECHNIQUE MONTREAL – Francisco de Melo 5 TraceCompare: It is the framework proposed to compare traces using the ECCT and the critical path. It shows the recorded metrics for a trained user to find the reason of this specific delay. ● Select groups of comparison manually ● Automation of the comparison task 2. Background TraceCompare
  • 10. POLYTECHNIQUE MONTREAL – Francisco de Melo 42. Background TraceCompare
  • 11. POLYTECHNIQUE MONTREAL – Francisco de Melo 6 ● Select groups of comparison manually ● Manual comparison task ● Only two groups of comparison 2. Background TraceCompare
  • 12. POLYTECHNIQUE MONTREAL – Francisco de Melo 7 ● Visualization tool ● CCTs ● Several Mathematical applications 2. Background Trevis
  • 13. POLYTECHNIQUE MONTREAL – Francisco de Melo 8 ● Comparative visualization tool ● CCTs ● Several Mathematical applications 2. Background Differential Flame Graph
  • 14. POLYTECHNIQUE MONTREAL – Francisco de Melo 9 Compare several executions of the same software The current tools are limited to compare several executions of the same software or require human analysis to find root cause problems. 3. Problem definition
  • 15. POLYTECHNIQUE MONTREAL – Francisco de Melo 104. Research Questions 3.1 Hypothesis Is it possible to find useful information by analyzing segments of traces from calling context trees? 3.2 Methodology Which methods can be used to do this process? 3.3 Assumptions 1. The groupings help uncover the causes for differences 2. Only the properties within the groups explain the differences
  • 16. POLYTECHNIQUE MONTREAL – Francisco de Melo 115. Data Model Calling Context Model Construction of tree using lttng and walking technique
  • 17. POLYTECHNIQUE MONTREAL – Francisco de Melo 125. Data Model
  • 18. POLYTECHNIQUE MONTREAL – Francisco de Melo 136. Methodology Overview ● Instrument the code ● Run the software ● Create the tree ● Apply the classification methods ● Compare the groups
  • 19. POLYTECHNIQUE MONTREAL – Francisco de Melo 146.1 Methodology - Instrumentation Linear output Instrument the source code This will enable recording profiling data which will then be saved together with each function LTTng and Perf Tools Metrics = instructions, page faults, cache misses
  • 20. POLYTECHNIQUE MONTREAL – Francisco de Melo 156.2 Methodology - Analyzing the data 6.2 Classification 6.2.1 Classifying a set of metrics by using a non-supervised method (e.g. k-means and elbow method) 6.2.2 Analysis of the classification 6.2.3 Use the result to deduce the root cause
  • 21. POLYTECHNIQUE MONTREAL – Francisco de Melo 166.3 Methodology - Mining Techniques Presentation of two techniques SVM k-means (elbow heuristic)
  • 22. POLYTECHNIQUE MONTREAL – Francisco de Melo 176.3 Methodology - Techniques Support Vector Machine Classify the data using an hyperplane Supervised Drawbacks
  • 23. POLYTECHNIQUE MONTREAL – Francisco de Melo 186.3.1 Methodology - Techniques k-means (elbow heuristic) Unsupervised k is pre-defined optimal division in k clusters
  • 24. POLYTECHNIQUE MONTREAL – Francisco de Melo 196.3.2 Methodology - Techniques Elbow heuristic Find the best k Compare SSE Comparison: find the biggest gap → starting from 2
  • 25. POLYTECHNIQUE MONTREAL – Francisco de Melo 206.3.2 Methodology - Techniques Elbow heuristic Find the best k Compare SSE
  • 26. POLYTECHNIQUE MONTREAL – Francisco de Melo 216.4 Methodology - Example Workload cache misses
  • 27. POLYTECHNIQUE MONTREAL – Francisco de Melo 227. Use Cases Presentation of three use cases Inline OpenCV Cache Optimization
  • 28. POLYTECHNIQUE MONTREAL – Francisco de Melo 237.1 Use Case 1 USE CASE 1 - Inline
  • 29. POLYTECHNIQUE MONTREAL – Francisco de Melo 247.2 Use Case 2 USE CASE 2 - OpenCV Optical Flow Instructions difference on the groups R correlation might not be used
  • 30. POLYTECHNIQUE MONTREAL – Francisco de Melo 247.2 Use Case 2
  • 31. POLYTECHNIQUE MONTREAL – Francisco de Melo 257.3 Use Case 3 USE CASE 3 - Cache Optimization Php Requests After several executions, the performance decreased Periodic problem
  • 32. POLYTECHNIQUE MONTREAL – Francisco de Melo 257.3 Use Case 3 USE CASE 3 - Cache Optimization
  • 33. POLYTECHNIQUE MONTREAL – Francisco de Melo 268. Discussion SVM will require a label phase The kernels of classification can lead to misclassification Soft classification cannot necessarily help, since a probabilistic classification can make the performance evaluation more difficult
  • 34. POLYTECHNIQUE MONTREAL – Francisco de Melo 27 Only for UST Instrumentation required Source code comparison is still required Assumptions must be considered 9. Threats to Validity
  • 35. POLYTECHNIQUE MONTREAL – Francisco de Melo 28 The differences on each group can indicate normal performance variations but also can highlight abnormal metric variations Knowing the abnormal metric and with some knowledge of the source code, it is possible to find performance regressions 10.1 Conclusion
  • 36. POLYTECHNIQUE MONTREAL – Francisco de Melo 29 Answers to the research questions Methods evaluation 10.1 Conclusion
  • 37. POLYTECHNIQUE MONTREAL – Francisco de Melo 30 Use other techniques Combination of techniques Automate the whole process - including source code comparison Merge with group comparison techniques 10. 2 Future work
  • 38. POLYTECHNIQUE MONTREAL – Francisco de Melo Questions isnaldo-francisco.de-melo-jr@polymtl.ca Any other info? 31
  • 39. POLYTECHNIQUE MONTREAL – Francisco de Melo Merci 32
  • 40. POLYTECHNIQUE MONTREAL – Francisco de Melo Want to learn more? 32
  • 41. POLYTECHNIQUE MONTREAL – Francisco de Melo Merci 32
  • 42. POLYTECHNIQUE MONTREAL – Francisco de Melo 33 Doray, F, and M. R. Dagenais, "Diagnosing Performance Variations by Comparing Multi- Level Execution Traces", IEEE Transactions on Parallel and Distributed Systems, vol. pp,issue: 99 no. 1, 2016. Andrea Adamoli and Matthias Hauswirth. Trevis: A context tree visualization & analysis framework and its use for classifying performance failure reports. In SoftVis ’10: Proceedings of the ACM Symposium on Software Visualization, 2010. J. M. Spivey. Fast, Accurate Call Graph Profiling. Softw. Pract. Exper., 34(3):249–264, 2004. W. N. Sumner, Y. Zheng, D. Weeratunge, and X. Zhang. Precise Calling Context Encoding. In ACM International Conference on Software Engineering, 2010. References