Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Performance Analysis Using Automated Grouping Mechanisms - Conference Portugal, 2018.

Slides presented at the 2018 IEEE International Conference on Software Quality, Reliability & Security in Lisbon, July 2018.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

Performance Analysis Using Automated Grouping Mechanisms - Conference Portugal, 2018.

  1. 1. Performance Analysis Using Automatic Grouping July 18th, 2018 Abderrahmane Benbachir École Polytechnique de Montréal Laboratoire DORSAL
  2. 2. POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel ● Intro ● Related Work ● Methodology ● Use Cases ● Discussion ● Conclusion Outline
  3. 3. POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel ● Continuous Integration ● Regression tests in several builds ● Performance evaluation Intro
  4. 4. POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel It is the framework proposed to compare traces using the ECCT and the critical path. It shows the recorded metrics for a trained user to find the reason of this specific delay. Doray, F, and M. R. Dagenais, "Diagnosing Performance Variations by Comparing Multi-Level Execution Traces", IEEE Transactions on Parallel and Distributed Systems, vol. pp,issue: 99 no. 1, 2016. Related Work TraceCompare
  5. 5. POLYTECHNIQUE MONTREAL – Francisco de Melo Related Work TraceCompare POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  6. 6. POLYTECHNIQUE MONTREAL – Francisco de Melo ● Select groups of comparison manually ● Manual comparison task ● Only two groups of comparison TraceCompare POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Related Work
  7. 7. POLYTECHNIQUE MONTREAL – Francisco de Melo ● Comparative visualization tool ● CCTs ● Several Mathematical applications Differential Flame Graph POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Related Work
  8. 8. POLYTECHNIQUE MONTREAL – Francisco de Melo Overview ● Instrument the code ● Run the software with tracing enabled ● Create the tree ● Apply the classification methods ● Compare the groups POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Methodology
  9. 9. POLYTECHNIQUE MONTREAL – Francisco de MeloPOLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Methodology - Data Model
  10. 10. POLYTECHNIQUE MONTREAL – Francisco de Melo Calling Context Model Construction of tree using lttng and walking technique POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Methodology - Data Model
  11. 11. POLYTECHNIQUE MONTREAL – Francisco de Melo ● Instrument the source code ● This will enable recording profiling data which will then be saved together with each function ● Trace with LTTng and Perf Tools Metrics = instructions, page faults, cache misses POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Methodology - Instrumentation
  12. 12. POLYTECHNIQUE MONTREAL – Francisco de Melo Classification ● Classifying a set of metrics by using a non-supervised method (e.g. k-means and elbow method) ● Analysis of the classification ● Use the result to deduce the root cause POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Methodology - Analyzing the data
  13. 13. POLYTECHNIQUE MONTREAL – Francisco de Melo Methodology - Techniques POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Elbow heuristic Find the best k Compare SSE (sum of squared errors) Comparison: find the biggest gap → starting from 2
  14. 14. POLYTECHNIQUE MONTREAL – Francisco de Melo Methodology - Grouping POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  15. 15. POLYTECHNIQUE MONTREAL – Francisco de Melo Use Case Cache Optimization Php Requests After several executions (100 times), the performance decreased Periodic problem POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  16. 16. POLYTECHNIQUE MONTREAL – Francisco de Melo Use Case Cache Optimization Grouping results relating the caching with the slow executions groups POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  17. 17. POLYTECHNIQUE MONTREAL – Francisco de Melo Use Case Software Regression (OpenCV) POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  18. 18. POLYTECHNIQUE MONTREAL – Francisco de Melo Use Case Software Regression (OpenCV) POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel Pearson correlation (R) of the variables shows that many metrics are directly proportional. This mean that the development of linear or multilinear models using those metrics is not adequate and consequently the comparison of groups, in a pair-wise approach, can be applied. Correlation among metrics
  19. 19. POLYTECHNIQUE MONTREAL – Francisco de Melo The differences on each group can indicate normal performance variations but also can highlight abnormal metric variations Knowing the abnormal metric and with some knowledge of the source code, it is possible to find performance regressions Conclusion POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  20. 20. POLYTECHNIQUE MONTREAL – Francisco de Melo Questions isnaldo-francisco.de-melo-junior@polymtl.ca POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  21. 21. POLYTECHNIQUE MONTREAL – Francisco de Melo Want to learn more? POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel
  22. 22. POLYTECHNIQUE MONTREAL – Francisco de Melo Doray, F, and M. R. Dagenais, "Diagnosing Performance Variations by Comparing Multi- Level Execution Traces", IEEE Transactions on Parallel and Distributed Systems, vol. pp,issue: 99 no. 1, 2016. Andrea Adamoli and Matthias Hauswirth. Trevis: A context tree visualization & analysis framework and its use for classifying performance failure reports. In SoftVis ’10: Proceedings of the ACM Symposium on Software Visualization, 2010. J. M. Spivey. Fast, Accurate Call Graph Profiling. Softw. Pract. Exper., 34(3):249–264, 2004. W. N. Sumner, Y. Zheng, D. Weeratunge, and X. Zhang. Precise Calling Context Encoding. In ACM International Conference on Software Engineering, 2010. References POLYTECHNIQUE MONTREAL – Francisco, Abder and Prof Michel

×