Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Predicting Method Crashes        with Bytecode Operations        Sunghun Kim        Hong Kong University of Science and Te...
© Microsoft Corporation
capture and replay+ prediction© Microsoft Corporation
Capture and Replay© Microsoft Corporation
ReCrash Technique      Goal: Convert a crash into a set of unit tests      1. Monitoring: maintain a shadow stack         ...
Cost of Monitoring     Key cost of ReCrash:     copying arguments to shadow stack     Tradeoff: less information in shadow...
ReCrash+ Technique      Goal: Convert a crash into a set of unit tests      1. Monitoring: maintain a shadow stack        ...
© Microsoft Corporation
crash       defect prediction© Microsoft Corporation
From Defect to Crash     1. The programmer creates a        defect – an error in the code.     2. When executed the defect...
Approach     Identify crashed methods        Gene© Microsoft Corporation
Approach                          Generate features from Bytecode© Microsoft Corporation
Approachfeatures from Bytecode            Build model © Microsoft Corporation
Step 1: Identify Crashed Methods                                     infoZilla                     Bug reportinfoZilla ima...
Step 1: Identify Crashed Methods                                     infoZilla                     Bug reportinfoZilla ima...
Step 2: Generate Features          Bytecode                              Control flow graph                              (...
Step 2: Generate Features© Microsoft Corporation
Step 3: Build Classifier© Microsoft Corporation
Experiments     1. Evaluating crash prediction            – Within-project classification            – Cross-project class...
Evaluating Crash Prediction     • Within-project classification:       ten-fold cross validation     • Cross-project valid...
Within-Project Classification© Microsoft Corporation
Cross-Project Classification© Microsoft Corporation
Reproducing Crashes     • Train classifier using the ECLIPSE corpus     • Classify methods from a different project       ...
Reproducing Crashes     All 3 crashes from SVNKit were successfully     reproduced by ReCrash+.     Runtime overhead decre...
Reproducing Crashes     Only a subset     of methods had     to be monitored:© Microsoft Corporation
Conclusion     • Monitoring crash-prone methods reduced the       overhead significantly at almost no cost.     • Opportun...
Upcoming SlideShare
Loading in …5
×

Predicting Method Crashes with Bytecode Operations

3,437 views

Published on

Presented at ISEC in New Delhi, February 2013

Published in: Technology
  • Be the first to comment

Predicting Method Crashes with Bytecode Operations

  1. 1. Predicting Method Crashes with Bytecode Operations Sunghun Kim Hong Kong University of Science and Technology, China Thomas Zimmermann Microsoft Research, USA Rahul Premraj VU University Amsterdam, The Netherlands Nicolas Bettenburg Queen’s University, Canada Shivkumar Shivaji University of California, Santa Cruz, USA© Microsoft Corporation
  2. 2. © Microsoft Corporation
  3. 3. capture and replay+ prediction© Microsoft Corporation
  4. 4. Capture and Replay© Microsoft Corporation
  5. 5. ReCrash Technique Goal: Convert a crash into a set of unit tests 1. Monitoring: maintain a shadow stack – Contains a copy of each method argument – On program crash, write the shadow stack to a file 2. Test generation: create many unit tests For each stack frame, create one unit test: – Invoke the method using arguments from the shadow stack – If the test does not reproduce the crash, discard the testSlide from: http://www.slideshare.net/hunkim/recrash-making-crashes-reproducible-by-preserving-object-states © Microsoft Corporation
  6. 6. Cost of Monitoring Key cost of ReCrash: copying arguments to shadow stack Tradeoff: less information in shadow stack ⇒ lower chance of reproducing crashes Monitor fewer methods: Ignore methods not likely to crash© Microsoft Corporation
  7. 7. ReCrash+ Technique Goal: Convert a crash into a set of unit tests 1. Monitoring: maintain a shadow stack – Contains a copy of each method argument for methods predicted to crash – On program crash, write the shadow stack to a file 2. Test generation: create many unit tests For each stack frame, create one unit test: – Invoke the method using arguments from the shadow stack – If the test does not reproduce the crash, discard the testSlide adapted from: http://www.slideshare.net/hunkim/recrash-making-crashes-reproducible-by-preserving-object-states © Microsoft Corporation
  8. 8. © Microsoft Corporation
  9. 9. crash defect prediction© Microsoft Corporation
  10. 10. From Defect to Crash 1. The programmer creates a defect – an error in the code. 2. When executed the defect creates an infection – an error in the state. 3. The infection propagates. 4. The infection causes a crash.Slide adapted from companion materials to Why Programs Fail, 2nd Edition.A Guide to Systematic Debugging, by Andreas Zeller, Morgan Kauffman. © Microsoft Corporation
  11. 11. Approach Identify crashed methods Gene© Microsoft Corporation
  12. 12. Approach Generate features from Bytecode© Microsoft Corporation
  13. 13. Approachfeatures from Bytecode Build model © Microsoft Corporation
  14. 14. Step 1: Identify Crashed Methods infoZilla Bug reportinfoZilla image by Nicolas© Microsoft Corporation Bettenburg
  15. 15. Step 1: Identify Crashed Methods infoZilla Bug reportinfoZilla image by Nicolas© Microsoft Corporation Bettenburg
  16. 16. Step 2: Generate Features Bytecode Control flow graph (basic blocks)© Microsoft Corporation
  17. 17. Step 2: Generate Features© Microsoft Corporation
  18. 18. Step 3: Build Classifier© Microsoft Corporation
  19. 19. Experiments 1. Evaluating crash prediction – Within-project classification – Cross-project classification – Significant features (see paper) – Impact of “throws” statements (see paper) 2. Reproducing crashes with ReCrash+© Microsoft Corporation
  20. 20. Evaluating Crash Prediction • Within-project classification: ten-fold cross validation • Cross-project validation: train on one project and test on the other • Baseline: complexity metrics Size of Method (in Bytes), Number of Conditional Statements, Number of Scalar Locals, Number of Vector Locals, Length of Local Identifiers, McCabe Complexity, Data Structure Complexity, Nesting Level Complexity, Halstead complexity measures© Microsoft Corporation
  21. 21. Within-Project Classification© Microsoft Corporation
  22. 22. Cross-Project Classification© Microsoft Corporation
  23. 23. Reproducing Crashes • Train classifier using the ECLIPSE corpus • Classify methods from a different project called SVNKit. – 2,347 methods of which 27% were classified as crash-prone • Apply ReCrash+: monitor only those methods predicted to be crash-prone – Three crashes from original ReCrash paper© Microsoft Corporation
  24. 24. Reproducing Crashes All 3 crashes from SVNKit were successfully reproduced by ReCrash+. Runtime overhead decreased:© Microsoft Corporation
  25. 25. Reproducing Crashes Only a subset of methods had to be monitored:© Microsoft Corporation
  26. 26. Conclusion • Monitoring crash-prone methods reduced the overhead significantly at almost no cost. • Opportunity for capture and replay tools to reduce overhead with prediction models. • Value of project’s history for the identification of crash-prone methods. • Potential value of Bytecode features for prediction models.© Microsoft Corporation

×