Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Would Static Analysis Tools Help Developers with Code Reviews?

1,136 views

Published on

Code reviews have been conducted since decades in
software projects, with the aim of improving code quality from
many different points of view. During code reviews, developers are supported by checklists, coding standards and, possibly, by various kinds of static analysis tools. This paper investigates whether warnings highlighted by static analysis tools are taken care of during code reviews and, whether there are kinds of warnings that tend to be removed more than others. Results of a study conducted by mining the Gerrit repository of six Java open source projects indicate that the density of warnings only slightly vary after each review. The overall percentage of warnings removed during reviews is slightly higher than what previous studies found for the overall project evolution history. However, when looking (quantitatively and qualitatively) at specific categories of warnings, we found that during code reviews developers focus on certain kinds of problems. For such
categories of warnings the removal percentage tend to be very high—often above 50% and sometimes 100%. Examples of those are warnings in the imports, regular expression, and type resolution categories. In conclusion, while a broad warning detection might produce way too many false positives, enforcing the removal of certain warnings prior to the patch submission could reduce the
amount of effort provided during the code review process.

  • Be the first to comment

Would Static Analysis Tools Help Developers with Code Reviews?

  1. 1. Would Static Analysis Tools Help Developers with Code Reviews? Sebastiano Panichella Venera Arnaoudova Massimiliano Di Penta Giuliano Antoniol
  2. 2. OUTLINE Context: Code Reviews. Case Study: Code Reviews of 6 Open Source Projects. Results: Warnings Resolved by Developers During Reviews.
  3. 3. CODE REVIEWS Why, What, How?
  4. 4. CODE REVIEWS Why?
  5. 5. CODE REVIEWS Why: concrete benefits… Improved  Code   Quality Fewer  defects   in  Code Improved   Knowledge   Transfer Education  of  Junior   Programmers Benefits “Expectations, Outcomes, and Challenges of Modern Code Review” Alberto Bacchelli and Christian Bird - ICSE 2013 “Common Outcomes of Code Review”
  6. 6. CODE REVIEWS What: types of peer code reviews? Formal  Inspection   Process Over  The  Shoulder   Reviews Email  Pass  Around   Interviews Tool  assisted   reviews Pair  Programming
  7. 7. CODE REVIEWS What: types of peer code reviews? Over  The  Shoulder   Reviews Email  Pass  Around   Interviews Tool  assisted   reviews Pair  Programming “Modern code review is a form of code inspection which has the qualities of being informal, tool-based and frequent.” “Expectations, Outcomes, and Challenges of Modern Code Review” Alberto Bacchelli and Christian Bird - ICSE 2013 Formal  Inspection   Process
  8. 8. MODERN CODE REVIEWS “Modern code review is a form of code inspection which has the qualities of being informal, tool-based and frequent.” “Expectations, Outcomes, and Challenges of Modern Code Review” Alberto Bacchelli and Christian Bird - ICSE 2013
  9. 9. MODERN CODE REVIEWS: TOOLS (I) Code Reviews Management
  10. 10. GERRIT: a Tool to Conduct and Manage Code Reviews
  11. 11. GERRIT: a Tool to Conduct and Manage Code Reviews
  12. 12. GERRIT: a Tool to Conduct and Manage Code Reviews
  13. 13. GERRIT: a Tool to Conduct and Manage Code Reviews
  14. 14. GERRIT: a Tool to Conduct and Manage Code Reviews
  15. 15. MODERN CODE REVIEWS (I) Code Reviews Management
  16. 16. MODERN CODE REVIEWS (I) Code Reviews Management (II) Bugs/Issues Detection
  17. 17. MODERN CODE REVIEWS (I) Code Reviews Management (II) Bugs/Issues Detection LIMITATION: provide a too extensive list of recommendations
  18. 18. Past Work Kim et al. - FSE 2007 Only10%, of suggested warnings are removed by bug fix changes
  19. 19. To What Extend Static Analysis Tools Help Developers During Code Reviews?
  20. 20. To What Extent Static Analysis Tools Help Developers During Code Reviews? Project History
  21. 21. To What Extent Static Analysis Tools Help Developers During Code Reviews? Project History During Code Reviews We argue that the Use of Static AnalysisTools Would be Highly Beneficial During Code Reviews…
  22. 22. CASE STUDY Code Reviews of 6 Open Source Projects.
  23. 23. Goal: understanding how static analysis tools could have helped in dealing with warnings developers solved during code reviews. Quality focus: reducing developers’ effort during the code review task. Perspective: develop tool to support the configuration of static analysis tools towards warnings that are considered relevant by developers. CASE STUDY
  24. 24. RESEARCH QUESTIONS RQ1: To what extent warnings detected by static analysis tools are removed during code reviews? RQ2: What kinds of warnings detected by static analysis tool are mainly considered during code reviews?
  25. 25. Projects Observe Period KLOC # of Reviews Analysed Uses Checkstylee Uses PDM Eclipse CDT 2013-11-29 - 2014-09-22 1,500– 1,550 309 Eclipse Platform UI 2013-06-24 - 2014-09-09 2,092– 2,305 16 Eclipse JDT Core 2013-05-23 - 2014-09-24 2,736– 2,554 113 OpenDaylight Controller 2013-01-01 - 2014-09-24 149–171 161 Motech 2013-07-24 - 2014-09-24 586–1,909 209 Vaadin 2013-06-01 - 2014-09-24 6,174– 6,114 180 CONTEXT Object: Tools Experimented:
  26. 26. STUDY PROCEDURE
  27. 27. PATCH SETS COMPARISON… Given a Code Review
  28. 28. PATCH SETS COMPARISON… Given a Code Review We use…
  29. 29. PATCH SETS COMPARISON… Given a Code Review We use... to compare warnings density variation between… Firstpatch Lastpatch
  30. 30. RQ1 To what extent warnings detected by static analysis tools are removed during code reviews?
  31. 31. Projects Density of Warnings [P-value] # of Warning [P-value] Density of Warnings [P-value] # of Warning [P-value] Eclipse CDT 0.074 0.025 0.028 <001 Eclipse JDT Core 0.450 0.919 0.351 0.624 Eclipse Platform UI 0.132 0.857 0.011 0.2 OpenDaylight Controller 0.080 <0.01 0.614 <0.01 Motech >0.01 <0.01 0.205 <0.01 Vaadin NA NA 0.148 0.209 Changes of Warnings Density (and Absolute Number) During Code Reviews.
  32. 32. Projects Density of Wornings [P-value] # of Warning [P-value] Density of Wornings [P-value] # of Warning [P-value] Eclipse CDT 0.074 0.025 0.028 <001 Eclipse JDT Core 0.450 0.919 0.351 0.624 Eclipse Platform UI 0.132 0.857 0.011 0.2 OpenDaylight Controller 0.080 <0.01 0.614 <0.01 Motech >0.01 <0.01 0.205 <0.01 Vaadin NA NA 0.148 0.209 Changes of Warnings Density (and Absolute Number) During Code Reviews.
  33. 33. Projects Density of Wornings [P-value] # of Warning [P-value] Density of Wornings [P-value] # of Warning [P-value] Eclipse CDT 0.074 0.025 0.028 <001 Eclipse JDT Core 0.450 0.919 0.351 0.624 Eclipse Platform UI 0.132 0.857 0.011 0.2 OpenDaylight Controller 0.080 <0.01 0.614 <0.01 Motech >0.01 <0.01 0.205 <0.01 Vaadin NA NA 0.148 0.209 Changes of Warnings Density (and Absolute Number) During Code Reviews.
  34. 34. Cumulative Percentage of Removed Warnings Projects Uses Checkstyle Uses PDM % of Resolved Warnings % of Resolved Warnings Eclipse CDT 11% 11% Eclipse Platform UI 5% 7% Eclipse JDT Core 11% 9% OpenDaylight Controller 15% 15% Motech 23% 13% Vaadin - 13%
  35. 35. Cumulative Percentage of Removed Warnings Projects Uses Checkstyle Uses PDM % of Resolved Warnings % of Resolved Warnings Eclipse CDT 11% 11% Eclipse Platform UI 5% 7% Eclipse JDT Core 11% 9% OpenDaylight Controller 15% 15% Motech 23% 13% Vaadin - 13%
  36. 36. RQ2 What kinds of warnings detected by static analysis tool are mainly considered during code reviews?
  37. 37. Qualitative Analysis
  38. 38. Qualitative Analysis
  39. 39. Qualitative Analysis
  40. 40. Qualitative Analysis “We randomly sampled 10% of code reviews that resolved at least one warning”
  41. 41. Qualitative AnalysisQualitative Analysis “Warning that Developers Fix During Code Reviews:”
  42. 42. Qualitative Analysis “Warning that Developers Fix During Code Reviews:” Type Resolution
  43. 43. Qualitative Analysis “Warning that Developers Fix During Code Reviews:” Unused code Type Resolution
  44. 44. Qualitative Analysis “Warning that Developers Fix During Code Reviews:” Imports Regular Expression Type Resolution Unused code
  45. 45. Qualitative Analysis “Warning that Developers Fix During Code Reviews:” Imports Regular Expression Type Resolution Unused code
  46. 46. Eclipse CDT: Percentage of PDM’ Resolved Warnings Warning Types % Resolved Warnings Type Resolution 100% Import 100% Basic 75% Sunsecure 67% Codesize 59% Unusedcode 58% Logging-java 51% j2ee 47% Design 42% junit 38% Empty 33% Javabeans 26% Naming 14% Braces 14% …. …..
  47. 47. Eclipse CDT: Percentage of PDM’ Resolved Warnings Warning Types % Resolved Warnings Type Resolution 100% Import 100% Basic 75% Sunsecure 67% Codesize 59% Unusedcode 58% Logging-java 51% j2ee 47% Design 42% junit 38% Empty 33% Javabeans 26% Naming 14% Braces 14% …. ….. “Quantitative Analisys Confirms Findings of the Qualitative analysis..”
  48. 48. OpenDaylight Controller: Percentage of Checkstyle’ Resolved Warnings Warning Types % Resolved Warnings Regular Expressions 100% Modifiers 100% Metrics 100% import 53% Whitespace 48% Class Design 47% Annotations 40% Naming 16% Coding 15% %Javadoc Comments 12% Size Violations 11% Javabeans 26% Block Checks 10% Miscellaneous 8% …. ….. “Similar Results for Checkstyle Warnings..”
  49. 49. OpenDaylight Controller: Percentage of Checkstyle’ Resolved Warnings Warning Types % Resolved Warnings Regular Expressions 100% Modifiers 100% Metrics 100% import 53% Whitespace 48% Class Design 47% Annotations 40% Naming 16% Coding 15% % Javadoc Comments 12% Size Violations 11% Javabeans 26% Block Checks 10% Miscellaneous 8% …. ….. Developers Fix also Warnings related to: 1) naming convention 2) code formatting 3) code comments
  50. 50. By implication… “Enforcing the removal of certain warnings before submitting a patch..”

×