Which Warnings Should I Fix First?

1,405 views
1,293 views

Published on

2007 European Software Engineering Conference and 2007 Foundations of Software Engineering (ESEC/FSE 2007)

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,405
On SlideShare
0
From Embeds
0
Number of Embeds
28
Actions
Shares
0
Downloads
25
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • We are going to use History since it gives the answer
  • Which Warnings Should I Fix First?

    1. 1. Sung Kim and Michael D. Ernst {hunkim, mernst}@csail.mit.edu CSAIL • MIT
    2. 2. Sung Kim and Michael D. Ernst {hunkim, mernst}@csail.mit.edu CSAIL • MIT
    3. 3. Static analysis I can find bugs for you!
    4. 4. Bug-finding tool
    5. 5. Warnings
    6. 6. Warning examples
    7. 7. Bug candidate - Warning Category : Style - empty try/catch Priority : 3
    8. 8. Are warnings useful? Are all warnings important to be fixed? Does tools’ prioritization help to identify important ones?
    9. 9. Experimental methodology <ul><li>Select source code (a snapshot) </li></ul><ul><li>Mark buggy lines </li></ul><ul><ul><li>Lines are changed in later fixes </li></ul></ul><ul><li>Compare with warnings from bug-finding tools </li></ul><ul><li>Measure precision </li></ul><ul><ul><li>High precision indicates important warnings </li></ul></ul>
    10. 10. Subject tools and projects <ul><li>Three bug-finding tools </li></ul><ul><ul><li>FindBugs, JLint, and PMD </li></ul></ul><ul><li>Three open source projects </li></ul>Projects Software type # of revisions Period LOC(K) Columba Email client 1,703 2002/11~2004/06 121 Lucene Search engine 929 2001/10~2006/11 37 Scarab Issue tracker 870 2002/01~2006/11 64
    11. 11. Buggy lines Rev 100 …… Mark as a buggy line if a line is changed in future fixes.
    12. 12. Change logs indicate fixes …… [bugfix] Fixed bug #3576 fix File 1 Non-fix Non-fix Non-fix Non-fix
    13. 13. Buggy lines Rev 199 Rev 200 Fix If (x=y && Z=x) {
    14. 14. Buggy lines Rev 199 Rev 198 If (x=y && Z=x) { If (x=y && Z=x) { marks Fix
    15. 15. Buggy lines Rev 100 Rev 200 Rev 199 Rev 198 ……
    16. 16. Warning evaluation Rev 100 50% of warnings are fixed later
    17. 17. Evaluate warnings at revision n/2 Rev n/2 Rev n Mark buggy lines Rev n/2+1
    18. 18. Are warnings important? NO only 3~18% are fixed!
    19. 19. Does tools’ prioritization help? NO Tools’ priority 1 warnings are not better.
    20. 20. Does tools’ prioritization help?
    21. 21. Does tools’ prioritization help?
    22. 22. Does tools’ prioritization help? No, high priority warnings do not have high precision!
    23. 23. …… Our solution: using change history Warning A File Warning B Foo ……
    24. 24. <ul><li>Compute score of each warning category using history </li></ul><ul><ul><li>Observe warning removals in history </li></ul></ul><ul><ul><li>If a warning is removed in a change </li></ul></ul><ul><ul><ul><li>Score promotes by non-fix-weight (0.1) </li></ul></ul></ul><ul><ul><li>If a warning is removed in a fix-change </li></ul></ul><ul><ul><ul><li>Score promotes by fix-weight (0.9) </li></ul></ul></ul><ul><ul><li>Assume high score warnings are important </li></ul></ul>Reprioritization algorithm
    25. 25. Reprioritization algorithm …… Score(A) += non-fix-weight (0.1) Score(A) += fix-weight (0.9) Fixed bug #3576 Score(A) += Non-fix-weight(0.1) Score(A)=1.1 fix Score(A)=0 Score(A)=0.1 Score(A)=1 Warning A File 1 Warning B Foo …… Score(B)=0
    26. 26. Evaluation Rev n/2-1 Rev n/2 Rev 1 Train warnings Rev n Mark buggy lines Rev n/2+1
    27. 27. Comparison <ul><li>History -based warning prioritization </li></ul><ul><li>Tools ’ warning prioritization </li></ul><ul><li>Compare precision of top 100 warnings </li></ul>
    28. 28. Columba
    29. 29. Columba
    30. 30. Lucene
    31. 31. Lucene
    32. 32. Scarab
    33. 33. Scarab
    34. 34. Summary
    35. 35. Sung Kim and Michael D. Ernst {hunkim, mernst}@csail.mit.edu CSAIL • MIT

    ×