The document discusses extracting patterns from bug fix changes in software projects to build a "bug fix memory". The memory captures abstracted bug and fix code patterns by parsing code, normalizing variables, filtering common elements, and storing unique components. The memory can then detect bugs and suggest fixes by matching new code. An evaluation on 5 projects found the memory captured 19.3-40.3% of actual bug fixes but also non-bug changes. Bugs detected differed from a static analysis tool, suggesting memories complement other approaches.

1. Memories of Bug-Fixes Sunghun Kim, Kai Pan, Jim Whitehead {hunkim, pankai, ejw}@cs.ucsc.edu University of California, Santa Cruz

2. What is a bug (Zeller 2006)? This pointer, being null, is a bug An incorrect program state This software crashes; this is a bug An incorrect program execution This line 11 is buggy An incorrect program code

3. Bugs? //null dereference public nullDeref () { MyObject o = null; if (isGoodDay) { o = new MyObject(“Hi”); } System.out.println(o.toString()); }

4. Bugs? //null dereference public nullDeref () { MyObject o = null; if (isGoodDay) { o = new MyObject(“Hi”); } System.out.println(o.toString()); }

5. Bugs? //stack buffer overun for sizes greater than 14 stack_buffer(void* src, int size ) { char buffer[14]; memcpy(buffer, src, size ); }

6. Bugs? //stack buffer over-run for sizes greater than 14 stack_buffer(void* src, int size ) { char buffer[14]; memcpy(buffer, src, size ); }

7. Bugs? if (…) { setSelectedText("\t"); }

8. There are many bug fix patterns that are specific to an individual project, and may not match one of the static patterns Example from jEdit project: Project-Specific Bug Fix Patterns JEditTextArea.java at transaction 114 - setSelectedText("\t"); + insertTab(); JEditTextArea.java at transaction 86 - setSelectedText("\t"); + insertTab();

9. Bug? if (requiredProjectRsc.exists() && requiredProjectRsc.isOpen()) { … }

10. Example from Eclipse project: JavaProject.java, transaction 2024 (“Fix for bug 28434”) - if (requiredProjectRsc.exists() && - requiredProjectRsc.isOpen ()) { + if ( JavaProject.hasJavaNature (requiredProjectRsc)) { DeltaProcessor.java, transaction 1945 (“Fix for bug 27499”) - boolean isOpened= proj.isOpen (); - if (isOpened && this.hasJavaNature(proj)) + if ( JavaProject.hasJavaNature (proj)) Project-Specific Bug Fix Patterns

11. Horizontal and Vertical Bug Patterns Buffer over run Horizontal : general bugs Vertical : project specific Null dereference JEdit example Eclipse example

12. Bug-Fix Memories – Basic Idea Extract patterns in bug fix change history …… Bug fix changes in revision 1 .. n-1 Memory

13. Bug-Fix Memories – Basic Idea Extract patterns in bug fix change history …… Search for patterns against Memory Bug fix changes in revision 1 .. n-1 Memory Code to examine

14. Talk Overview Detection of bug fix changes Mining vertical bugs Abstracting code Evaluation Conclusions Future Work

15. Retrieving Bug Fix Changes Software projects today record their development history using Software Configuration Management tools As developers make changes, they record a reason along with the change In the change log message When developers fix a bug in the software, they tend to record log messages with some variation of the words “fixed” or “bug” “ Fixed null pointer bug” It is possible to mine the change history of a software project to uncover these bug-fix changes That is, we retrospectively recover those changes that developers have marked as containing a bug fix We assume they are not lying

16. Bug-introducing and Bug-fix Changes Development history of foo.java SCM log message: “Bug #567 fixed” “ bug fix” Bug #567 entered into issue tracking system (bug finally observed and recorded) Software change that introduces the bug “bug-introducing”

17. Kenyon Processing SCM Repository Filesystem Extract Automated configuration extraction Save Persist gathered metrics & facts Kenyon Repository (RDBMS/ Hibernate) Analyze Query DB, add new facts Analysis Software (e.g., IVA) Compute Fact extraction (metrics, static analysis) Kenyon

18. Commits, Transactions & Configurations transactions configurations CVS file commits Added feature X Fixed null ptr bug Modified button text Added feature Y log message

19. Hunks, and Hunk Pairs Revision n-1 (has bug hunks) Revision n (has fix hunks) modification addition deletion added hunk hunk pair type deleted hunk empty deleted hunk empty added hunk

20. Detecting Vertical Bugs (Patterns) Detecting bug patterns Saving exact code in bug and fix hunks doesn’t work, since there is rarely an exact match. Need a method for abstracting changes to find patterns Approach Abstract code in each bug fix change Save abstracted bug and fix code in a database (the “bug fix memory”) Can search existing code to see if it matches a bug fix pattern Can suggest code to fix the bug

21. Process for Abstracting Code Four step process Raw component extraction Parse source code, and burst out individual syntactic elements Normalization Substitute type names for variables, string literals, constants (abstract to types) Information filtering Remove elements that are too common to yield project-specific patterns Diff filtering Remove code components that are common in bug and fix hunks, yielding only code unique to the change

22. Raw Component Extraction Step 1: Convert statements inside change hunks so they lie on a single line Eliminate whitespace Concatenate multi-line statements to one line Concatenate conditionals for complex statements (if, while, etc.) to one line Step 2: Extract raw components Component is a non-leaf node in the syntax tree of a single line Bursts out complex statements into constituent parts Each portion of a complex conditional is a separate component Additionally, separate out a method call and its parameters

23. Raw Component Extraction Example Initial code if (foo.flag > 5 && foo.ready()) { i=1; foo.create(“example”); initiate(6,bar); } Extracted Raw Components foo.flag foo.flag > 5 foo.ready() ready() foo.flag > 5 && foo.ready () if (foo.flag > 5 && foo.ready()) i=1 “ example” foo.create(.) “example” create(.) “example” initiate(,) 6, bar if > && . . foo flag 5 foo ready() ready

24. Normalization To further improve the ability to match code, perform abstraction of instances to types Replace variable instance with its type Permits matching on type, rather than instance foo.flag >= 5  Foo.flag >= 5 (type of foo is Foo) For literals, insert new component with type i=1 yields int=1 and int=int For method calls, replace each parameter with type of parameter Use “*” for unknown types (we only do one-pass parse) initiate(,) 6, bar  initiate(,) int,* (type of bar is unknown)

25. Information Filtering Goal After normalization, resulting components are candidates for insertion into database Problem: many commonly occurring statement types int=int Want to eliminate these, and others that don’t contribute unique information about bug fixes

26. Information Filtering Approach Assign an “information value” to component elements Value 2: method call, string literal longer than 8 chars Value 1: predicates for: if, do, while, for, as well as conditional expressions return, case, switch, synchronized, throw string literal, length 3-8 chars variable name, field name, class name, variable type Value 0: Everything else Information value for an entire component is the sum of its elemental information values We remove components with information value < 2 int=1 (info value = 1), int=int (info value = 0) “ example” (info value = 1), String (info value = 0)

27. Diff Filtering and Storing Memories As a final filtering step, keep only those components that are unique to either bug or fix hunks Duplicate components are eliminated, since they do not represent the bug or its fix After diff filtering step, store all components into the database (“memory”) Components record their transaction, file name, bug or fix hunk, etc. Also store initial source code of bug and fix hunks

28. Searching the Memory The memory database contains extracted adaptive bug and fix patterns for a given project Can use this memory to find code that matches bug code in the memory Use scenario Developer working in their favorite development environment Receives feedback when code they are developing matches a stored bug pattern Can also suggest potential fixes from stored bug fix code

29. IDE Integration Bug detection Fix suggestion

30. Evaluation We evaluated the memory to determine how well it captures new bug fix changes Online learning approach Specifically, we create a memory for transactions 1 to n-1 At transaction n , for bug fix changes we examine whether the bug hunks are found in the memory This is a “half hit” If found, we also examine whether the fix hunk is found too This is a “full hit” Examined same 5 project histories ArgoUML, Columba, Eclipse, jEdit, Scarab This can be viewed as a proxy for how well the approach might work for bug and fix prediction

31. Half and Full Hit Build memories based on transaction 1 .. n-1 …… Transaction 1 .. n-1 Memories Bug | Fix Fix change case at transaction n Half hit Full hit

32. True and False Positives Build memories based on transaction 1 .. n-1 …… False positive half hit, if found True positive half hit, if found Transaction 1 .. n-1 Memories Non-fix change case at transaction n Fix change case at transaction n

33. True Positive Hit Rates

34. False Positive Hit Rates

35. True Positive and False Positive Full Hit Rates

36. True Positive and False Positive Full Hit Rates Bug fix memories work well Captures 19.3%-40.3% of bugs (half-hits) But, also captures a lot of non-bug changes (20.8%-32.5%)

37. PMD VS Fix Memories PMD is a bug finding tool based on a static syntax checker Bug

38. PMD VS Fix Memories PMD is a bug finding tool based on a static syntax checker Bug PMD

39. PMD VS Fix Memories PMD is a bug finding tool based on a static syntax checker Bug PMD Fix Memories

40. PMD VS Fix Memories PMD is a bug finding tool based on a static syntax checker Bug PMD Fix Memories

41. PMD VS Fix Memories PMD is a bug finding tool based on a static syntax checker Found bugs by PMD and Fix memories are largely exclusive 40.3% 6.5% PMD Fix Memories 3% ArgoUML 38.7% 6.5% PMD Fix Memories 2.3% Eclipse

42. Conclusions It is now possible to reliably extract bug fix memories from software project evolution data Bug fix memories work well Captures 19.3%-40.3% of bugs (half-hits) But, also captures a lot of non-bug changes (20.8%-32.5%) Found bugs using fix memories and PMD are mostly exclusive Our approach complements other bug finding tools

43. Future Work Developing other pattern extracting algorithms To remove false positives AST, Slicing, Control flow, etc. Comparing fix memories with more bug finding tools FindBugs, JLint, etc.