Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

TBar: Revisiting Template-based Automated Program Repair

25 views

Published on

We revisit the performance of template-based APR to build comprehensive knowledge about the effectiveness of fix patterns, and to highlight the importance of complementary steps such as fault localization or donor code retrieval. To that end, we first investigate the literature to collect, summarize and label recurrently-used fix patterns. Based on the investigation, we build TBar, a straightforward APR tool that systematically attempts to apply these fix patterns to program bugs. We thoroughly evaluate TBar on the Defects4J benchmark. In particular, we assess the actual qualitative and quantitative diversity of fix patterns, as well as their effectiveness in yielding plausible or correct patches. Eventually, we find that, assuming a perfect fault localization, TBar correctly/plausibly fixes 74/101 bugs. Replicating a standard and practical pipeline of APR assessment, we demonstrate that TBar correctly fixes 43 bugs from Defects4J, an unprecedented performance in the literature (including all approaches, i.e., template-based, stochastic mutation-based or synthesis-based APR).

Published in: Technology
  • Be the first to comment

  • Be the first to like this

TBar: Revisiting Template-based Automated Program Repair

  1. 1. TBar: Revisiting Template-based Automated Program Repair Kui LIU, Anil Koyuncu, Dongsun Kim and Tegawendé F. Bissyandé SnT, University of Luxembourg, Luxembourg 2019-07-17, Wednesday @ISSTA 2019, Beijing, China
  2. 2. 1 > Fix Pattern Fix pattern: e.g., PAR, kim2013ICSE [23]. Fix template: e.g., SOFix, liu2018SANER [40]. Abstract modification: e.g., SimFix, jiang2018ISSTA [18]. Mutation operator: e.g., CapGen, wen2018ICSE [69]. Modification rule: e.g., ssFix, xin2017ASE [74]. Code transform: e.g., Genesis, long2017FSE [41]. Program transformation schemas: e.g., SketchFix, hua2018ICSE[17]. “A fix pattern is a pair of a code context representing a buggy code entity and a set of related change operations, which can be applied to a given buggy code entity to generate fixing code.” ---- Liu et al. TSE 2018 [35]. FP: <Code Context, {Change Operations}>
  3. 3. 2 > Limited Characterization of Fix Patterns APR Tool Venue Number of Fix Patterns PAR [23] ICSE-2013 10 jMutRepair [40] ISSTA-2016 2 HDRepair [29] SANER-2016 11 NPEfix [13] SANER-2017 9 ssFix [74] ASE-2017 34 Genesis [41] FSE-2017 108 S3 [25] ASE-2017 4 ELIXIR [62] ASE-2017 8 SketchFix [17] ICSE-2018 6 SOFix [40] SANER-2018 12 SimFix [18] ISSTA-2018 16 CapGen [69] ICSE-2018 30 AVATAR [39] SANER-2019 13 1.Diversity of fix patterns. 2.Bug fix performance of each fix pattern. 3.Fault localization sensitiveness of fix patterns.
  4. 4. 3 > Study Methodology 1. SLR: Systematically review the APR literature related to fix patterns. 2. Taxonomy: enumerate and summarize fix patterns. 3. A Baseline Repair Tool: implement a comprehensive pattern based APR tool. 4. Empirical Study on Performance and Sensitiveness.
  5. 5. 4 SLR and Taxonomy
  6. 6. 5 > Fix Pattern Inference in the Literature 1. Manual Summarization. 2. Automated Mining. 3. Pre-definition. 4. Statistics of Actions. 1) Pan et al. EMSE-2009, 27 fix patterns, 5 Java projects. 2) Kim et al. ICSE-2013, 10 fix patterns, 62,656 patches. 1) Long et al. FSE-2017, 108 fix patterns for 3 kinds of defects. 2) Liu and Zhong, SANER-2018, 12 fix patterns, Stack Overflow. 1) Saha et al. ASE-2017, 11 fix patterns, on the top of PAR. 2) Hua et al. ICSE-2018, 6 fix patterns. 1) Wen et al. ICSE-2018, top-30 frequent code change actions. 2) Jiang et al. ISSTA-2018, top-16 frequent code change actions.
  7. 7. 6 > Fixing a Bug with a Fix Pattern --- a/source/org/jfree/chart/renderer/category/AbstractCategoryItemRenderer.java +++ b/source/org/jfree/chart/renderer/category/AbstractCategoryItemRenderer.java @ -1796,4 +1796,4 @ CategoryDataset dataset = this.plot.getDataset(index); - if (dataset != null) { + if (dataset == null) { return result; } A Defects4J Bug Chart-1. Action Type: Update, Granularity: Expression. Spread: Single statement. Fix Pattern: Mutate Operator.
  8. 8. 7 > Fix Pattern Taxonomy FP1. Insert Cast Checker. FP2. Insert Null Pointer Checker. FP3. Insert Range Checker. FP4. Insert Missed Statement. FP5. Mutate Class Instance Creation. FP6. Mutate Conditional Expression. FP7. Mutate Data Type. FP8. Mutate Integer Division Operation. FP9. Mutate Literal Expression. FP10. Mutate Method Invocation Expression. FP11. Mutate Operator. FP12. Mutate Return Statement. FP13. Mutate Variable. FP14. Move Statement. FP15. Remove Statement. 15 Fix Pattern Categories  35 fix patterns.
  9. 9. 8 > Change Properties of Fix Patterns Action Type # fix patterns Update 17 Delete 4 Insert 13 Move 1 Granularity # fix patterns Expression 21 Statement 17 Method 1 Spread # fix patterns Single- statement 30 Multi- statement 7
  10. 10. 9 TBar: a Baseline APR System
  11. 11. 10 > TBar: Template-Based automated program repair Overall Workflow of TBar. GZoltar and Ochiai
  12. 12. 11 > Fix Pattern Selection Selected fix pattern A ranked list of suspicious statements Fix pattern data base Code AST … // Buggy code of Closure-13 in Defects4J dataset. if(options.dependencyOptions.needsManagement() && !options.skipAllPasses && options.closurePass) { // AST. IfStatement ---InfixExpression ------MethodInvocation --------- …… ------Operator ------PrefixExpression --------- …… ------Operator ------QualifiedName --------- …… FP6. Mutate Conditional Expression. FP6.1: - ...condExp1... + ...condExp2... FP6.2: - ...condExp1 Op condExp2... + ...condExp1... FP6.3: - ...condExp1... + ...condExp1 Op condExp2... Breadth First Search
  13. 13. 12 > Patch Generation // Buggy code of Closure-13 in Defects4J dataset. if(options.dependencyOptions.needsManagement() && !options.skipAllPasses && options.closurePass) { // Patch Candidate I. - if(options.dependencyOptions.needsManagement() && - !options.skipAllPasses && + if(!options.skipAllPasses && options.closurePass) { // Patch Candidate II. if (options.dependencyOptions.needsManagement() && - !options.skipAllPasses && options.closurePass) { // Patch Candidate III. if (options.dependencyOptions.needsManagement() && - !options.skipAllPasses && - options.closurePass) { + !options.skipAllPasses) { FP6.2: - ...condExp1 Op condExp2... + ...condExp1... Patch CandidatesSelected fix pattern Mutate suspicious code FP6. Mutate Conditional Expression. FP6.1: - ...condExp1... + ...condExp2... FP6.2: - ...condExp1 Op condExp2... + ...condExp1... FP6.3: - ...condExp1... + ...condExp1 Op condExp2...
  14. 14. 13 > Assessment Benchmark Project Chart (C) Closure (Cl) Lang (L) Math (M) Mockito (Mc) Time (T) Total # bugs 26 133 65 106 38 27 395 # test cases 2,205 7,927 2,245 3,602 1,457 4,130 21,566 Benchmark Defects4J dataset information
  15. 15. 14 Repair Suitability of Fix Patterns
  16. 16. 15 > Experiment #1: TBar with Perfect Fault Localization Research Questions : RQ1. How many bugs from Defects4J can be correctly fixed by fix patterns from our taxonomy? RQ2. Can each Defects4J bug be fixed by different fix patterns? RQ3. What are the properties of fix patterns that are successfully used to fix bugs?
  17. 17. 16 > RQ1: Number of Fixed Bugs Project Chart (C) Closure (Cl) Lang (L) Math (M) Mockito (Mc) Time (T) Total # fully fixed bugs 12/13 20/26 13/18 23/25 3/3 3/6 74/101 # partially fixed bugs 2/4 3/6 1/4 0/4 0/0 1/1 7/20 Number of bugs fixed by fix patterns in TBar with perfect fault localization
  18. 18. 17 > Limitation and Discussion 1. Insufficient fix patterns. 2. Less effective search of fix ingredients. public String generateToolTipFragment(String toolTipText) { - return " title="" + toolTipText + return " title="" + ImageMapUtilities.htmlEscape(toolTipText) + "" alt="""; } Matchable fix pattern: FP9.2. Donor code searching in “local” file
  19. 19. 18 > RQ2: Number of Fix Patterns vs. Fixing Bugs 28 9 2 1 38 10 5 3 10 4 4 3 1 1 1 1 1 2 3 4 5 >5#FIX PATTERNS P #1 #2 #3 #4 It is important to select appropriate fix patterns for different bugs.
  20. 20. 19 > RQ3: Fix Pattern Properties 85 49 24 4 43 32 18 2 Update Insert Delete Move #fixedbugs 122 38 2 54 39 2 Expression Statement Method 140 22 71 24 Single Multiple Plausibly BUT Incorrectly fixed bugs. Correctly fixed bugs. a. Change Actions b. Change Granularity c. Changed Spread Repair performance is different among fix patterns properties.
  21. 21. 20 Repair Performance Comparison
  22. 22. 21 > Experiment #2: TBar with Normal Fault Localization Research Questions: RQ4. What performance can be achieved by TBar in a standard and practical repair scenario? RQ5. To what extent are the fix pattern based repair tool sensitive to noise in fault localization (i.e., spotting bug positions)?
  23. 23. 22 > RQ4: Performance Comparison with the State-of-the-Art TBar outperforms all recent state-of-the-art APR tools that were evaluated on the Defects4J dataset.
  24. 24. 23 > RQ5: Fault Localization Sensitiveness Fault localization noise has a significant impact on the performance of TBar. Plausibly but incorrectly fixed bugs 24 1514 1 Fully fixed Partially fixed #fixed bugs incorrect position correct position
  25. 25. 24 > DISCUSSION 1. Fix pattern diversity. 2. Fix pattern selection. 3. Donor code searching. 4. Patch prioritization. 5. Fault Localization Sensitiveness.
  26. 26. 25 > SUMMARY 10 > TBar: Template-Based automated program repair Overall Workflow of TBar. GZoltar and Ochiai 23 > Standard Practical Performance https://github.com/SerVal-DTF/TBar

×