Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

iFixR: Bug Report Driven Program Repair

32 views

Published on

Issue tracking systems are commonly used in modern software development for collecting feedback from users and developers. An ultimate automation target of software maintenance is then the systematization of patch generation for user-reported bugs. Although this ambition is aligned with the momentum of automated program repair, the literature has, so far, mostly focused on generate-and- validate setups where fault localization and patch generation are driven by a well-defined test suite. On the one hand, however, the common (yet strong) assumption on the existence of relevant test cases does not hold in practice for most development settings: many bugs are reported without the available test suite being able to reveal them. On the other hand, for many projects, the number of bug reports generally outstrips the resources available to triage them. Towards increasing the adoption of patch generation tools by practitioners, we investigate a new repair pipeline, iFixR, driven by bug reports: (1) bug reports are fed to an IR-based fault localizer; (2) patches are generated from fix patterns and validated via regression testing; (3) a prioritized list of generated patches is proposed to developers. We evaluate iFixR on the Defects4J dataset, which we enriched (i.e., faults are linked to bug reports) and carefully-reorganized (i.e., the timeline of test-cases is naturally split). iFixR generates genuine/plausible patches for 21/44 Defects4J faults with its IR-based fault localizer. iFixR accurately places a genuine/plausible patch among its top-5 recommendation for 8/13 of these faults (without using future test cases in generation-and-validation).

Published in: Technology
  • Be the first to comment

  • Be the first to like this

iFixR: Bug Report Driven Program Repair

  1. 1. iFixR: Bug Report driven Program Repair Anil Koyuncu, Kui Liu, Tegawendé F. Bissyandé, Dongsun Kim, Martin Monperrus, Jacques Klein, Yves Le Traon SnT, University of Luxembourg, Luxembourg 2019-08-28, Wednesday @FSE 2019
  2. 2. Typical Generate-validate pipeline 2 > Code changes in Software repositories Bug fix patches Enhanced AST diff representations Diff hunk search space construction Rooted tree isomorphism computation Clustering based on subgraph Identification Step 0 Step 1 Step 2 Step 3 Step 4 Step 5 Iterative folding Patch Candidates Patch Generation Patch Validation Suspicious Code Locations Buggy Program Fault Localization Test Suite Pass Fail Testing
  3. 3. Assumption of Test suite 3 > Code changes in Software repositories Bug fix patches Enhanced AST diff representations Diff hunk search space construction Rooted tree isomorphism computation Clustering based on subgraph Identification Step 0 Step 1 Step 2 Step 3 Step 4 Step 5 Iterative folding Patch Candidates Patch Generation Patch Validation Suspicious Code Locations Buggy Program Fault Localization Test Suite Pass Fail Testing Test Cases in Defects4J , 92% of bugs do not have a failing test case when a bug report submitted
  4. 4. What practical repair pipeline to account for limited test-suite? 4 > Code changes in Software repositories Bug fix patches Enhanced AST diff representations Diff hunk search space construction Rooted tree isomorphism computation Clustering based on subgraph Identification Step 0 Step 1 Step 2 Step 3 Step 4 Step 5 Iterative folding Patch Candidates Patch Generation Patch Validation Suspicious Code Locations Buggy Program Fault Localization Test Suite Pass Fail Testing Test Cases ? ?
  5. 5. What practical repair pipeline to account for limited test-suite? 5 > Code changes in Software repositories Bug fix patches Enhanced AST diff representations Diff hunk search space construction Rooted tree isomorphism computation Clustering based on subgraph Identification Step 0 Step 1 Step 2 Step 3 Step 4 Step 5 Iterative folding Patch Candidates Patch Generation Patch Validation Suspicious Code Locations Buggy Program Fault Localization Test Suite Pass Fail Testing ? Bug Report
  6. 6. What practical repair pipeline to account for limited test-suite? 6 > Code changes in Software repositories Bug fix patches Enhanced AST diff representations Diff hunk search space construction Rooted tree isomorphism computation Clustering based on subgraph Identification Step 0 Step 1 Step 2 Step 3 Step 4 Step 5 Iterative folding Patch Candidates Patch Generation Patch Validation Suspicious Code Locations Buggy Program Fault Localization Test Suite Pass Fail Regression Tests Bug Report
  7. 7. Approach 7
  8. 8. iFixR: Bug Report driven Program Repair IRBL FeaturesDistribute to Regions ... ... ... ... Regions Step 2 Step 3 Step 4 Step 5 Divide Conquer Best Models Leaf-wise Weights Computations Code changes in Software repositories Bug fix patches Enhanced AST diff representations Diff hunk search space construction Rooted tree isomorphism computation Clustering based on subgraph Identification Step 0 Step 1 Step 2 Step 3 Step 4 Step 5 Iterative folding Patch Candidates Patch Generation Patch Validation Regression Testing Fix Pattern Matching IR-based fault localization Suspicious Code Locations Buggy Program Select fix pattern Mutate suspicious code Fault Localization Bug Report Fix patterns Manual Validation Developer Test Code Elements (AST) 8 > A relevant test case reproducing the bug may not be readily available when a bug report is submitted to the issue tracking system.
  9. 9. Where is the Bug? IRBL FeaturesDistribute to Regions ... ... ... ... Regions Step 2 Step 3 Step 4 Step 5 Divide Conquer Best Models Leaf-wise Weights Computations Code changes in Software repositories Bug fix patches Enhanced AST diff representations Diff hunk search space construction Rooted tree isomorphism computation Clustering based on subgraph Identification Step 0 Step 1 Step 2 Step 3 Step 4 Step 5 Iterative folding Patch Candidates Patch Generation Patch Validation Regression Testing Fix Pattern Matching IR-based fault localization Suspicious Code Locations Buggy Program Select fix pattern Mutate suspicious code Fault Localization Bug Report Fix patterns Manual Validation Developer Test Code Elements (AST) 9 > Information Retrieval based Fault Localization (IRFL)
  10. 10. Information Retrieval based Fault Localization • gener 10 >
  11. 11. Statement-level IR-based Fault Localization 11 Top-K Suspicious Code Files > … Statement Suspiciousness scores Statement-level Fault Localizer
  12. 12. How should we change the code? IRBL FeaturesDistribute to Regions ... ... ... ... Regions Step 2 Step 3 Step 4 Step 5 Divide Conquer Best Models Leaf-wise Weights Computations Code changes in Software repositories Bug fix patches Enhanced AST diff representations Diff hunk search space construction Rooted tree isomorphism computation Clustering based on subgraph Identification Step 0 Step 1 Step 2 Step 3 Step 4 Step 5 Iterative folding Patch Candidates Patch Generation Patch Validation Regression Testing Fix Pattern Matching IR-based fault localization Suspicious Code Locations Buggy Program Select fix pattern Mutate suspicious code Fault Localization Bug Report Fix patterns Manual Validation Developer Test Code Elements (AST) 12 > Fix Pattern-based Selection
  13. 13. 13 > Fix Pattern Taxonomy FP1. Insert Cast Checker. FP2. Insert Null Pointer Checker. FP3. Insert Range Checker. FP4. Insert Missed Statement. FP5. Mutate Class Instance Creation. FP6. Mutate Conditional Expression. FP7. Mutate Data Type. FP8. Mutate Integer Division Operation. FP9. Mutate Literal Expression. FP10. Mutate Method Invocation Expression. FP11. Mutate Operator. FP12. Mutate Return Statement. FP13. Mutate Variable. FP14. Move Statement. FP15. Remove Statement.
  14. 14. 14 > Fix Pattern Selection (see TBAR@ISSTA 2019) Selected fix pattern A ranked list of suspicious statements Fix pattern data base Code AST … // Buggy code of Closure-13 in Defects4J dataset. if(options.dependencyOptions.needsManagement() && !options.skipAllPasses && options.closurePass) { // AST. IfStatement ---InfixExpression ------MethodInvocation --------- …… ------Operator ------PrefixExpression --------- …… ------Operator ------QualifiedName --------- …… FP6. Mutate Conditional Expression. FP6.1: - ...condExp1... + ...condExp2... FP6.2: - ...condExp1 Op condExp2... + ...condExp1... FP6.3: - ...condExp1... + ...condExp1 Op condExp2... Breadth First Search
  15. 15. How should we generate the patch? IRBL FeaturesDistribute to Regions ... ... ... ... Regions Step 2 Step 3 Step 4 Step 5 Divide Conquer Best Models Leaf-wise Weights Computations Code changes in Software repositories Bug fix patches Enhanced AST diff representations Diff hunk search space construction Rooted tree isomorphism computation Clustering based on subgraph Identification Step 0 Step 1 Step 2 Step 3 Step 4 Step 5 Iterative folding Patch Candidates Patch Generation Patch Validation Regression Testing Fix Pattern Matching IR-based fault localization Suspicious Code Locations Buggy Program Select fix pattern Mutate suspicious code Fault Localization Bug Report Fix patterns Manual Validation Developer Test Code Elements (AST) 15 > Fix Pattern-based Patch Generation
  16. 16. 16 > Patch Generation // Buggy code of Closure-13 in Defects4J dataset. if(options.dependencyOptions.needsManagement() && !options.skipAllPasses && options.closurePass) { // Patch Candidate I. - if(options.dependencyOptions.needsManagement() && - !options.skipAllPasses && + if(!options.skipAllPasses && options.closurePass) { // Patch Candidate II. if (options.dependencyOptions.needsManagement() && - !options.skipAllPasses && options.closurePass) { // Patch Candidate III. if (options.dependencyOptions.needsManagement() && - !options.skipAllPasses && - options.closurePass) { + !options.skipAllPasses) { FP6. Mutate Conditional Expression. FP6.2: - ...condExp1 Op condExp2... + ...condExp1... Patch CandidatesSelected fix pattern Mutate suspicious code
  17. 17. Which patches are valid? IRBL FeaturesDistribute to Regions ... ... ... ... Regions Step 2 Step 3 Step 4 Step 5 Divide Conquer Best Models Leaf-wise Weights Computations Code changes in Software repositories Bug fix patches Enhanced AST diff representations Diff hunk search space construction Rooted tree isomorphism computation Clustering based on subgraph Identification Step 0 Step 1 Step 2 Step 3 Step 4 Step 5 Iterative folding Patch Candidates Patch Generation Patch Validation Regression Testing Fix Pattern Matching IR-based fault localization Suspicious Code Locations Buggy Program Select fix pattern Mutate suspicious code Fault Localization Bug Report Fix patterns Manual Validation Developer Test Code Elements (AST) 17 > Available test cases to validate patch candidates ! Available test cases may not even trigger the bug
  18. 18. iFixR: Patch Validation 18 Regression tests to validate patch candidates A patch ordering strategy to recommend patches with priority Heuristics to re-prioritize the patch candidates 1. Minimal changes 2. Higher fault localization score 3. Affected code elements >
  19. 19. 19 RQ1: [Fault localization] : To what extent does IR-based fault localization provide reliable results for an APR scenario? RQ2: [Overfitting] : To what extent does IR-based fault localization point to locations that are less subject to overfitting? RQ3: [Patch ordering] : What is the effectiveness of iFixR’s patch ordering strategy? > RQs: Research Questions
  20. 20. 20 > RQ1: Fault localization performance Spectrum Fault Localization (SFL) vs Information Retrieval Fault Localization(IRFL) IR-based fault localization in iFixR is as accurate as Spectrum-based fault localization
  21. 21. 21 > RQ2: Reparability IRFL vs SFL impacts on the number of generated correct/plausible patches. IRFL and SBFL localization information lead to similar repair performance in terms of the number of fixed bugs plausibly/correctly
  22. 22. 22 IR-based fault localization lead less to overfitted patches than the code locations suggested by Spectrum-based fault localization > RQ2: Overfitting Dissection of why patches are plausible.
  23. 23. 23 > RQ3: Patch ordering Patch re-prioritization recommends more plausible and even correct patches
  24. 24. 24 > RQ3: Patch Recommendation [iFixRtop5] - only top 5 recommended patches validated only with regression tests: [iFixRall] - all generated patches validated only with regression tests [iFixRopt]- all generated patches validated with augmented test suites Scenarios: Scenario
  25. 25. 25 > Take-aways • Test cases can be incomplete / buggy • Bug reports deserve more interest • Better approaches to selecting fix ingredients are necessary • Prioritization techniques must be investigated • More comprehensive benchmarks are needed
  26. 26. 26 > Conclusion • Feasibility of automating patch generation from bug reports • No assumptions on the availability of future test cases • Patch ordering strategy to recommend patches for developer validation
  27. 27. 27 iFixR: Bug Report driven Program Repair IRBL FeaturesDistribute to Regions ... ... ... ... Regions Step 2 Step 3 Step 4 Step 5 Divide Conquer Best Models Leaf-wise Weights Computations Code changes in Software repositories Bug fix patches Enhanced AST diff representations Diff hunk search space construction Rooted tree isomorphism computation Clustering based on subgraph Identification Step 0 Step 1 Step 2 Step 3 Step 4 Step 5 Iterative folding Patch Candidates Patch Generation Patch Validation Regression Testing Fix Pattern Matching IR-based fault localization Suspicious Code Locations Buggy Program Select fix pattern Mutate suspicious code Fault Localization Bug Report Fix patterns Manual Validation Developer Test Code Elements (AST) 10 > A relevant test case reproducing the bug may not be readily available when a bug report is submitted to the issue tracking system. 25 > RQ1: Fault localization performance Spectrum Fault Localization (SFL) vs Information Retrieval Fault Localization(IRFL) IR-based fault localization in iFixR is as accurate as Spectrum-based fault localization 27 IR-based fault localization lead less to overfitted patches than the code locations suggested by Spectrum-based fault localization > RQ2: Overfitting Dissection of why patches are plausible.
  28. 28. 28
  29. 29. Strong assumption limits the practicality of APR • Incomplete • Few tests are written • Future • Tests written after the source code • Regression • Tests to validate the bugs are fixed and will not reoccur 29 >
  30. 30. Typical Information Retrieval Fault Localization 30 Suspicious Code Files >
  31. 31. Tests in Suites: Defects4J Cases 33 96% 4% Test Suite Evolution Future test cases Available test cases > Test case changes in fix commits of Defects4J bugs.
  32. 32. Test case availability when the bug is reported 34 8% 92% After Removing Future Test Cases Failing test cases No failure > Failing test cases after removing future test cases Fault Localization and Patch validation may be challenged in the case of user-reported bugs, given the lack of relevant test cases.
  33. 33. How to repair without complete test suite? 35 • Fault localization without the future test cases? Bug Reports • Patch validation without the future test cases? Regression Tests Patch suggestions for developers > A relevant test case reproducing the bug may not be readily available when a bug report is submitted to the issue tracking system.
  34. 34. Statement-level IR-based Fault Localization 37 Top-K Suspicious Code Files Extract Statements Extract Text Similarity Feature Vectors File Suspiciousness scores X > Preprocess & Tokenization … … … Statement Suspiciousness scores Weighted Suspiciousness scores … Statement Suspiciousness scores
  35. 35. What practical repair pipeline to account for limited test-suite? 38 > Code changes in Software repositories Bug fix patches Enhanced AST diff representations Diff hunk search space construction Rooted tree isomorphism computation Clustering based on subgraph Identification Step 0 Step 1 Step 2 Step 3 Step 4 Step 5 Iterative folding Patch Candidates Patch Generation Patch Validation Suspicious Code Locations Buggy Program Fault Localization Test Suite Pass Fail Regression Tests Bug Report 1. 2. 5. Patch Patch Patch Recommended Patches …

×