OSS Java Analysis - What You Might Be Missing


Published on

We think FindBugs is a great tool for finding coding style and best practice types of issues, so we conducted a little experiment a few months ago. We analyzed Jenkins core code with both FindBugs and Coverity. These slides provide a high-level summary of our results.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

OSS Java Analysis - What You Might Be Missing

  1. 1. OSS Java Analysis What You Might Be Missing
  2. 2. Overview • Open Source Static Analysis Options for Java • Case Study: Analysis of Jenkins Project • Detailed Findings 2 Copyright 2013, Coverity, Inc.
  3. 3. Consistent Use = More Maintainable Code Numerous Static Analysis Options for Java Copyright 2013, Coverity, Inc.3 Checkstyle Defines and enforces coding standards Finds common programming flaws and duplicated code Finds bugs Finds coding errors, ensures proper testing coverage and more
  4. 4. • Will you use it? • Does it find issues that you care about? Does it fit into your team’s workflow? Is it worth the effort? • Is it actionable? • Does it just find issues, or does it help you understand how and where to fix them? • Is it accurate? • Will you waste significant time on false positives? Are results consistent across builds? • Is it integrated? • Do you have to go out of your way to use it, or does it bring the problems to you? Key Considerations Copyright 2013, Coverity, Inc.4
  5. 5. Case Study: Jenkins Coverity Development Testing Platform and FindBugs 5 Copyright 2013, Coverity, Inc.
  6. 6. Background • Analyzed Jenkins 1.496 core code using up-to-date Coverity and FindBugs (as of Dec 2012) • Both solutions focus on finding defects, have IDE and command line interfaces Copyright 2013, Coverity, Inc.6 FindBugs • Often used automatically in Eclipse • Extensible tool • Management capabilities focused on individual developers Coverity Platform • Most often run in Eclipse or via automated build • Extensible tool, open platform • Sophisticated management capabilities addressing the needs of developers, teams & organizations
  7. 7. Little Overlap 7 Copyright 2013, Coverity, Inc. Only 28 relevant issues found by both solutions
  8. 8. Defect Summary (Real Bugs Only) 8 Type Coverity FindBugs Both Resource leaks 86 12 13 Security problems 1 0 0 Concurrency problems 22 10 9 High Severity Subtotal 109 22 22 Coding Standards, Best Practices, Other 88 605 6 Total Bugs 197 627 28 Coverity FindBug s Impact Distribution High Med-Low Copyright 2013, Coverity, Inc. Each solution found unique issues (neither is sufficient to replace the other)
  9. 9. Here is a resource leak identified by FindBugs: • Identifies the problem and general location • But there’s no indication of the affected object/variable, the relevant code path or events • Developers need to figure it out on their own • Not a big deal in this simple code • Complex code will be a pain! Example: FindBugs Resource Leak Copyright 2013, Coverity, Inc.9 A leak might affect something on this line
  10. 10. Here is a leak identified by Coverity: The object will continue to hold resources until the garbage collector eventually finalizes it—which may never happen. Example: Coverity Resource Leak Copyright 2013, Coverity, Inc.10 “listener” allocated. Implements Closeable, so it needs to be closed “listener” used, but still needs to be closed Leak: no longer usable, and never closed
  11. 11. Example: Coverity Race Condition 11 Copyright 2013, Coverity, Inc. This is an HTTP request handler “o” is associated with session Another thread may modify the session Session attribute is deleted. The right one? Erroneous code path identified You may delete the wrong key! Good luck finding this manually!
  12. 12. Example: Coverity Null Dereference Copyright 2013, Coverity, Inc.12 Passing null to function Dereferenced two layers deep You should check for null before or during the call to rebuild()
  13. 13. Conclusion • Different analysis tools often find different (important and complementary) issues • Use multiple tools that find issues important to you • Avoid tools that take too long to analyze, require too much manual work or have too many false positives 13 Copyright 2013, Coverity, Inc.
  14. 14. Copyright 2013 Coverity, Inc.