The document discusses the development and application of crash graphs, which aggregate multiple crash reports to aid in debugging, prioritize fixable crashes, and identify duplicate bugs. It evaluates the effectiveness of crash graphs in providing insights to developers and improving bug triage through quantitative measures like precision and recall. The future work outlined includes enhancing the interactivity of crash graphs and exploring other algorithms for trace clustering.

1. Crash Graphs: An aggregated view of
multiple crashes to improve crash triage
Sung Kim (HKUST)
Tom Zimmermann and Nachi Nagappan (MSR)

2. Windows Error Reporting (WER) System

3. Windows Error Reporting (WER) System

4. Windows Error Reporting (WER) System
crashes

5. Windows Error Reporting (WER) System
Identifying
Crash causes

6. Windows Error Reporting (WER) System
bucketing

7. Windows Error Reporting (WER) System

8. Windows Error Reporting (WER) System
Bug Bug Bug
report report report
1 2 3
Reporting
bugs

9. Crash Graph
Aggregation of multiple crashes

10. Crash Graph
Trace 1 A B C D
Trace 2 A E F G D
Trace 3 C D G D

11. Crash Graph
A
Trace 1 A B C D B
Trace 2 A E F G D
Trace 3 C D G D
C
D

12. Crash Graph
A
E
Trace 1 A B C D B
Trace 2 A E F G D
Trace 3 C D G D F
C
G
D

13. Crash Graph
A
E
Trace 1 A B C D B
Trace 2 A E F G D
Trace 3 C D G D F
C
G
D

14. Crash Graph
A
E
Trace 1 A B C D B
Trace 2 A E F G D
Trace 3 C D G D F
C
G
D

15. Crash Graph Example

16. Research Questions
} RQ1: Is it useful for debugging?
} RQ2: Can
this identify duplicated bugs
(second buckets)
} RQ3: Can
this hold crash properties: can
we predict fixable crashes?

17. RQ1: Useful for Debugging?

18. Evaluation
} Find fixed bugs reported by Watson(autobug)
} Draw crash graphs for the bugs
} Send the graphs to the corresponding fixers
} Ask fixers for comments

19. Developer feedback
} “… the graph would be showing me that a single cab
could not…”
} “Your graph looks helpful…”
} “Usually developers can guess 50-80% the crash
causes by reading call traces. This graph can help
developers to see all traces together”

20. RQ2: Detecting Duplicated Bugs
Bug Bug Bug
report report report
1 2 3
Reporting
bugs

21. RQ2: Detecting Duplicated Bugs
Duplicated!
Bug Bug Bug
Fixed report report report
1 2 3
Reporting
bugs
Second bucket

22. Sub-graph similarity
⊇

23. Sub-graph similarity
!"#(​%↓'"( , ​%↓*#+,, )=​|​-↓'"( ∩​ -↓*#+,, |/|​-↓*#+,, | ,
where E is the set of edges in G and |​-↓*#+,, |≤|​-↓'"( |.
⊇

24. Evaluation
Bug ids Dup?
Bug 1 Bug 2
Bug 1
Bug 1 Bug 3
Duplicated!
Bug 2 Bug 1 Bug 4
Bug 1 Bug 5
Bug 3 Bug 2 Bug 3
Bug 2 Bug 4
Bug 4 Duplicated!
Bug 2 Bug 5
Bug 5 Bug 3 Bug 4
Bug 3 Bug 5
From bug reports Bug 4 Bug 5

25. Evaluation
Bug ids Dup?
Bug 1 Bug 2
Bug 1 Bug 3
Bug 1 Bug 4
Bug 1 Bug 5
Bug 2 Bug 3
Bug 2 Bug 4
Bug 2 Bug 5
Bug 3 Bug 4
Bug 3 Bug 5
Bug 4 Bug 5

26. Similarity Computation
Bug ids Dup? Similarity Dup? threshold=0.9
Bug 1 Bug 2 0.85
Bug 1 Bug 3 0.95
Bug 1 Bug 4 0.8
Bug 1 Bug 5 0.7
Bug 2 Bug 3 0.8
Bug 2 Bug 4 0.8
Bug 2 Bug 5 0.1
Bug 3 Bug 4 0.4
Bug 3 Bug 5 0.96
Bug 4 Bug 5 0.2
Precision= 50%, recall = 50%

27. Subject (WinOS Bugs)
Name Value
# of bug reports X
# of duplicated bugs 13.3%
# of total bug pair (X*X-1/2)
# of duplicated bug pair 0.32%
# of non-duplicated bug Remaining

28. Dup-detection Results
Similarity Precision Recall
threshold
1 *70.3 58.8
0.99 71.5 62.4
0.98 71.0 63.6
0.97 68.4 64.2
0.96 65.0 64.2
0.95 61.6 64.2

29. Why Crash Graph Works?
} Uses all traces to compare
trace1

30. Why Crash Graph Works?
} Uses all traces to compare
trace1

31. Why Crash Graph Works?
} Uses all traces to compare
90% trace 2
trace1

32. Why Crash Graph Works?
} Uses all traces to compare
trace1
trace 2

33. Why Crash Graph Works?
} Uses all traces to compare
80% trace 3
trace1
trace 2

34. Why Crash Graph Works?
} Uses all traces to compare
trace1 trace 3
trace 2

35. Why Crash Graph Works?
} Uses all traces to compare
trace1 trace 3
90%
trace 2

36. Why Crash Graph Works?
} Partial traces
Bucket 1
Trace 1 A B C D
Trace 2 D E F G H
Bucket 2
Trace 3 C D E F

37. RQ3: Predicting Fixable Crashes
} Not all crashes will be fixed
} There are too many crashes
} Can we prioritize developers’ effort?
} If we know which crashes are likely to be fixed
} Developers can focus on these first

38. Extracting Features
Features values
Node # 7
Edge # 5
Max-in 4
Max-out 2
Crash graph

39. Extracting Features
Bug id Features Fixed?
1 0 1 3 1 1 5 1
2 1 1 2 1 3 1 1
3 1 1 1 5 1 0 1
Machine
1 1 2 1 3 1 0 learner Fixable!

40. Results
Subjects/Features Precision Recall F-measure
Windows 7 Exchange14
Crash graph 79.5 69.6 74.5
Bug meta data 69.9 66.1 68.6
Crash graph 72.1 60.3 65
All features 71.8 61.2 65.4
Subjects: Several hundred bugs from Windows 7 and a few thousand from
Exchange 14 bugs

41. Results
Subjects/Features Precision Recall F-measure
Bug meta data 80 57.2 66.3
Windows 7 Exchange14
Crash graph 79.5 69.6 74.5
All features 80 70.6 74.7
Bug meta data 69.9 66.1 68.6
Crash graph 72.1 60.3 65
All features 71.8 61.2 65.4
Subjects: Several hundred bugs from Windows 7 and a few thousand from
Exchange 14 bugs

42. Evaluation
Subjects/Features Precision Recall F-measure
Bug meta data 80 57.2 66.3
Windows 7 Exchange14
Crash graph 79.5 69.6 74.5
All features 80 70.6 74.7
Bug meta data 69.9 66.1 68.6
Crash graph 72.1 60.3 65
All features 71.8 61.2 65.4
Subjects: Several hundred bugs from Windows 7 and a few thousand from
Exchange 14 bugs

43. Summary: Crash Graph is Useful
} Debugging
} Identifying duplicated bugs (second
buckets)
} Predicting fixable crashes

44. Future Work
} Interactive Crash Graphs
} Other trace clustering algorithms
Crash topic analysis
}
} Applying crash graphs for other problems
} One-hit buckets