Crash Graphs: An aggregated view ofmultiple crashes to improve crash triage                                       Sung Kim...
Windows Error Reporting (WER) System
Windows Error Reporting (WER) System
Windows Error Reporting (WER) Systemcrashes
Windows Error Reporting (WER) SystemIdentifyingCrash causes
Windows Error Reporting (WER) Systembucketing
Windows Error Reporting (WER) System
Windows Error Reporting (WER) System             Bug      Bug      Bug            report   report   report              1 ...
Crash GraphAggregation of multiple crashes
Crash GraphTrace 1   A   B   C DTrace 2   A   E   F   G DTrace 3   C D G D
Crash Graph                                    ATrace 1   A   B   C D           BTrace 2   A   E   F   G DTrace 3   C D G ...
Crash Graph                                    A                                        ETrace 1   A   B   C D           B...
Crash Graph                                    A                                        ETrace 1   A   B   C D           B...
Crash Graph                                    A                                        ETrace 1   A   B   C D           B...
Crash Graph Example
Research Questions}  RQ1: Is   it useful for debugging?}  RQ2: Can           this identify duplicated bugs (second bucke...
RQ1: Useful for Debugging?
Evaluation}  Find fixed bugs reported by Watson(autobug)}  Draw crash graphs for the bugs}  Send the graphs to the corr...
Developer feedback}    “… the graph would be showing me that a single cab      could not…”}    “Your graph looks helpful...
RQ2: Detecting Duplicated Bugs             Bug      Bug      Bug            report   report   report              1       ...
RQ2: Detecting Duplicated Bugs                       Duplicated!               Bug                       Bug          Bug ...
Sub-graph similarity                       ⊇
Sub-graph similarity         !"#(​%↓"( ,  ​%↓*#+,, )=​|​-↓"(   ∩​  -↓*#+,, |/|​-↓*#+,, |    ,  where E is the set of edges...
Evaluation                                     Bug ids      Dup?                                  Bug 1   Bug 2     Bug 1 ...
Evaluation                Bug ids      Dup?             Bug 1   Bug 2             Bug 1   Bug 3             Bug 1   Bug 4 ...
Similarity Computation     Bug ids       Dup?             Similarity   Dup?   threshold=0.9  Bug 1   Bug 2                ...
Subject (WinOS Bugs)         Name                Value         # of bug reports                   X     # of duplicated bu...
Dup-detection Results Similarity    Precision          Recall threshold           1           *70.3               58.8    ...
Why Crash Graph Works?}  Uses   all traces to compare                     trace1
Why Crash Graph Works?}  Uses   all traces to compare            trace1
Why Crash Graph Works?}  Uses   all traces to compare                     90%   trace 2            trace1
Why Crash Graph Works?}  Uses   all traces to compare            trace1            trace 2
Why Crash Graph Works?}  Uses   all traces to compare                      80%   trace 3            trace1            tra...
Why Crash Graph Works?}  Uses   all traces to compare            trace1                 trace 3            trace 2
Why Crash Graph Works?}  Uses   all traces to compare            trace1                 trace 3                          ...
Why Crash Graph Works?}  Partial    tracesBucket 1 Trace 1   A   B   C D Trace 2   D   E   F   G HBucket 2 Trace 3   C D ...
RQ3: Predicting Fixable Crashes}  Not all crashes will be fixed}  There are too many crashes}  Can we prioritize develo...
Extracting Features                      Features   values                      Node #       7                      Edge #...
Extracting Features                     Bug id           Features            Fixed?                          1   0   1   3...
Results                       Subjects/Features      Precision   Recall     F-measureWindows 7 Exchange14                 ...
Results                       Subjects/Features      Precision   Recall     F-measure                              Bug met...
Evaluation                       Subjects/Features      Precision   Recall     F-measure                              Bug ...
Summary: Crash Graph is Useful}  Debugging}  Identifying   duplicated bugs (second buckets)}  Predicting    fixable cra...
Future Work}          Interactive Crash Graphs}          Other trace clustering algorithms       Crash topic analysis   ...
"Crash Graphs: An Aggregated View of Multiple Crashes to Improve Crash Triage" by Sunghun Kim, Thomas Zimmermann and Nachi...
Upcoming SlideShare
Loading in …5
×

"Crash Graphs: An Aggregated View of Multiple Crashes to Improve Crash Triage" by Sunghun Kim, Thomas Zimmermann and Nachiappan Nagappan.

1,605 views

Published on

"Crash Graphs: An Aggregated View of Multiple Crashes to Improve Crash Triage" by Sunghun Kim, Thomas Zimmermann and Nachiappan Nagappan.

Published in: Technology, News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,605
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

"Crash Graphs: An Aggregated View of Multiple Crashes to Improve Crash Triage" by Sunghun Kim, Thomas Zimmermann and Nachiappan Nagappan.

  1. 1. Crash Graphs: An aggregated view ofmultiple crashes to improve crash triage Sung Kim (HKUST) Tom Zimmermann and Nachi Nagappan (MSR)
  2. 2. Windows Error Reporting (WER) System
  3. 3. Windows Error Reporting (WER) System
  4. 4. Windows Error Reporting (WER) Systemcrashes
  5. 5. Windows Error Reporting (WER) SystemIdentifyingCrash causes
  6. 6. Windows Error Reporting (WER) Systembucketing
  7. 7. Windows Error Reporting (WER) System
  8. 8. Windows Error Reporting (WER) System Bug Bug Bug report report report 1 2 3Reportingbugs
  9. 9. Crash GraphAggregation of multiple crashes
  10. 10. Crash GraphTrace 1 A B C DTrace 2 A E F G DTrace 3 C D G D
  11. 11. Crash Graph ATrace 1 A B C D BTrace 2 A E F G DTrace 3 C D G D C D
  12. 12. Crash Graph A ETrace 1 A B C D BTrace 2 A E F G DTrace 3 C D G D F C G D
  13. 13. Crash Graph A ETrace 1 A B C D BTrace 2 A E F G DTrace 3 C D G D F C G D
  14. 14. Crash Graph A ETrace 1 A B C D BTrace 2 A E F G DTrace 3 C D G D F C G D
  15. 15. Crash Graph Example
  16. 16. Research Questions}  RQ1: Is it useful for debugging?}  RQ2: Can this identify duplicated bugs (second buckets)}  RQ3: Can this hold crash properties: can we predict fixable crashes?
  17. 17. RQ1: Useful for Debugging?
  18. 18. Evaluation}  Find fixed bugs reported by Watson(autobug)}  Draw crash graphs for the bugs}  Send the graphs to the corresponding fixers}  Ask fixers for comments
  19. 19. Developer feedback}  “… the graph would be showing me that a single cab could not…”}  “Your graph looks helpful…”}  “Usually developers can guess 50-80% the crash causes by reading call traces. This graph can help developers to see all traces together”
  20. 20. RQ2: Detecting Duplicated Bugs Bug Bug Bug report report report 1 2 3Reportingbugs
  21. 21. RQ2: Detecting Duplicated Bugs Duplicated! Bug Bug Bug Fixed report report report 1 2 3Reportingbugs Second bucket
  22. 22. Sub-graph similarity ⊇
  23. 23. Sub-graph similarity !"#(​%↓"( ,  ​%↓*#+,, )=​|​-↓"(   ∩​  -↓*#+,, |/|​-↓*#+,, |  , where E is the set of edges in G and |​-↓*#+,, |≤|​-↓"( |. ⊇
  24. 24. Evaluation Bug ids Dup? Bug 1 Bug 2 Bug 1 Bug 1 Bug 3 Duplicated! Bug 2 Bug 1 Bug 4 Bug 1 Bug 5 Bug 3 Bug 2 Bug 3 Bug 2 Bug 4 Bug 4 Duplicated! Bug 2 Bug 5 Bug 5 Bug 3 Bug 4 Bug 3 Bug 5From bug reports Bug 4 Bug 5
  25. 25. Evaluation Bug ids Dup? Bug 1 Bug 2 Bug 1 Bug 3 Bug 1 Bug 4 Bug 1 Bug 5 Bug 2 Bug 3 Bug 2 Bug 4 Bug 2 Bug 5 Bug 3 Bug 4 Bug 3 Bug 5 Bug 4 Bug 5
  26. 26. Similarity Computation Bug ids Dup? Similarity Dup? threshold=0.9 Bug 1 Bug 2 0.85 Bug 1 Bug 3 0.95 Bug 1 Bug 4 0.8 Bug 1 Bug 5 0.7 Bug 2 Bug 3 0.8 Bug 2 Bug 4 0.8 Bug 2 Bug 5 0.1 Bug 3 Bug 4 0.4 Bug 3 Bug 5 0.96 Bug 4 Bug 5 0.2 Precision= 50%, recall = 50%
  27. 27. Subject (WinOS Bugs) Name Value # of bug reports X # of duplicated bugs 13.3% # of total bug pair (X*X-1/2) # of duplicated bug pair 0.32% # of non-duplicated bug Remaining
  28. 28. Dup-detection Results Similarity Precision Recall threshold 1 *70.3 58.8 0.99 71.5 62.4 0.98 71.0 63.6 0.97 68.4 64.2 0.96 65.0 64.2 0.95 61.6 64.2
  29. 29. Why Crash Graph Works?}  Uses all traces to compare trace1
  30. 30. Why Crash Graph Works?}  Uses all traces to compare trace1
  31. 31. Why Crash Graph Works?}  Uses all traces to compare 90% trace 2 trace1
  32. 32. Why Crash Graph Works?}  Uses all traces to compare trace1 trace 2
  33. 33. Why Crash Graph Works?}  Uses all traces to compare 80% trace 3 trace1 trace 2
  34. 34. Why Crash Graph Works?}  Uses all traces to compare trace1 trace 3 trace 2
  35. 35. Why Crash Graph Works?}  Uses all traces to compare trace1 trace 3 90% trace 2
  36. 36. Why Crash Graph Works?}  Partial tracesBucket 1 Trace 1 A B C D Trace 2 D E F G HBucket 2 Trace 3 C D E F
  37. 37. RQ3: Predicting Fixable Crashes}  Not all crashes will be fixed}  There are too many crashes}  Can we prioritize developers’ effort? }  If we know which crashes are likely to be fixed }  Developers can focus on these first
  38. 38. Extracting Features Features values Node # 7 Edge # 5 Max-in 4 Max-out 2 Crash graph
  39. 39. Extracting Features Bug id Features Fixed? 1 0 1 3 1 1 5 1 2 1 1 2 1 3 1 1 3 1 1 1 5 1 0 1 Machine1 1 2 1 3 1 0 learner Fixable!
  40. 40. Results Subjects/Features Precision Recall F-measureWindows 7 Exchange14 Crash graph 79.5 69.6 74.5 Bug meta data 69.9 66.1 68.6 Crash graph 72.1 60.3 65 All features 71.8 61.2 65.4Subjects: Several hundred bugs from Windows 7 and a few thousand fromExchange 14 bugs
  41. 41. Results Subjects/Features Precision Recall F-measure Bug meta data 80 57.2 66.3Windows 7 Exchange14 Crash graph 79.5 69.6 74.5 All features 80 70.6 74.7 Bug meta data 69.9 66.1 68.6 Crash graph 72.1 60.3 65 All features 71.8 61.2 65.4Subjects: Several hundred bugs from Windows 7 and a few thousand fromExchange 14 bugs
  42. 42. Evaluation Subjects/Features Precision Recall F-measure Bug meta data 80 57.2 66.3Windows 7 Exchange14 Crash graph 79.5 69.6 74.5 All features 80 70.6 74.7 Bug meta data 69.9 66.1 68.6 Crash graph 72.1 60.3 65 All features 71.8 61.2 65.4Subjects: Several hundred bugs from Windows 7 and a few thousand fromExchange 14 bugs
  43. 43. Summary: Crash Graph is Useful}  Debugging}  Identifying duplicated bugs (second buckets)}  Predicting fixable crashes
  44. 44. Future Work}  Interactive Crash Graphs}  Other trace clustering algorithms Crash topic analysis } }  Applying crash graphs for other problems }  One-hit buckets

×