Successfully reported this slideshow.
Your SlideShare is downloading. ×

Automatic Patch Generation Learned from Human-Written Patches

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 171 Ad
Advertisement

More Related Content

Advertisement

Recently uploaded (20)

Automatic Patch Generation Learned from Human-Written Patches

  1. 1. Automatic Patch Generation Learned from Human-Written Patches Dongsun Kim, Jaechang Nam, Jaewoo Song, and Sunghun Kim The Hong Kong University of Science and Technology, China 24 May 2013 the 35th International Conference on Software Engineering (ICSE 2013)
  2. 2. 2 Repair
  3. 3. 2 Repair
  4. 4. 2 Repair GenProgGenProg
  5. 5. 3 GenProg C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  6. 6. 3 GenProg State-of-the-art C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  7. 7. 3 GenProg State-of-the-art Genetic Programming C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  8. 8. 3 GenProg State-of-the-art Genetic Programming Random Mutation C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  9. 9. 3 GenProg State-of-the-art Genetic Programming Random Mutation Systematically Evaluated C. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  10. 10. 4 Buggy Code
  11. 11. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 4 in Interpreter.java reported as Mozilla Bug #76683 Buggy Code
  12. 12. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 4 in Interpreter.java reported as Mozilla Bug #76683 Buggy Code
  13. 13. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 4 in Interpreter.java reported as Mozilla Bug #76683 Null Pointer Exception Buggy Code
  14. 14. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 5 GenProg repairs bugs
  15. 15. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 5 GenProg repairs bugs 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; Buggy  Code GenProg
  16. 16. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 5 GenProg repairs bugs 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; Buggy  Code GenProg
  17. 17. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 5 GenProg repairs bugs 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; Buggy  Code GenProg This patch passes ALL test cases.
  18. 18. 6 GenProg repairs bugs1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num;
  19. 19. 6 GenProg repairs bugs1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num;
  20. 20. 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 6 GenProg repairs bugs1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num;
  21. 21. 7 Would you accept?
  22. 22. 7 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept?
  23. 23. 7 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept?
  24. 24. 7 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept? 17  Students 68  Developers
  25. 25. 8 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept? 17  Students 68  Developers
  26. 26. 8 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // do nothing. 1507 } 1508 state.parenCount = num; Would you accept? 9.4% 90.6% 17  Students 68  Developers
  27. 27. 9
  28. 28. 9
  29. 29. 9 •Random
  30. 30. 9 •Random •Unnatural
  31. 31. 9 •Random •Unnatural •Harder  to  understand
  32. 32. 10
  33. 33. 10
  34. 34. 10 Random Evolution
  35. 35. 10 Random Evolution Explores the large search space
  36. 36. 10 Random Evolution Explores the large search space but sometimes produces random results
  37. 37. 10 Random Evolution Intelligent Design Explores the large search space but sometimes produces random results
  38. 38. 10 Random Evolution Intelligent Design Explores the large search space Leverages human knowledge but sometimes produces random results
  39. 39. 11 Human-written Patches
  40. 40. 11 Human-written Patches Readable
  41. 41. 11 Human-written Patches Readable Natural
  42. 42. 11 Human-written Patches Readable Natural Easy to understand
  43. 43. 11 Human-written Patches Readable Natural Easy to understand We can learn how to generate patches from human knowledge.
  44. 44. 12
  45. 45. 12 JDT
  46. 46. 12 >60,000 Patches JDT
  47. 47. 12 Manual Classification >60,000 Patches JDT
  48. 48. 12 Manual Classification #patches Patterns >60,000 Patches JDT
  49. 49. 12 Manual Classification #patches Patterns >60,000 Patches JDT
  50. 50. 12 Manual Classification #patches Patterns Top frequent patterns account for >20~30% >60,000 Patches JDT
  51. 51. 13 Common Fix Patterns Altering method parameters obj.method(v1,v2)0→0obj.method(v1,v3) Altering method parameters obj.method(v1,v2)0→0obj.method(v1,v3)
  52. 52. 13 Common Fix Patterns Adding a checker obj.m1())→)if(obj'!='null)){obj.m1()} Adding a checker obj.m1())→)if(obj'!='null)){obj.m1()}
  53. 53. 14 PAR Pattern-based Automatic Program Repair
  54. 54. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 15
  55. 55. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 15 + - + Human Knowledge
  56. 56. 16 + - + Using Human Knowledge for patch generation
  57. 57. 16 + - + Fix Templates Using Human Knowledge for patch generation
  58. 58. 16 + - + Fix Templates Program Edit Script Using Human Knowledge for patch generation
  59. 59. 16 + - + Fix Templates Program Edit Script Using Human Knowledge for patch generation 10
  60. 60. 16 + - + Fix Templates Program Edit Script Manually created from fix patterns Using Human Knowledge for patch generation 10 JDT
  61. 61. 16 + - + Fix Templates Program Edit Script Manually created from fix patterns Highly reusable Using Human Knowledge for patch generation 10 JDT
  62. 62. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 17
  63. 63. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 18 if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Fault Localization
  64. 64. 19 Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  65. 65. 19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  66. 66. 19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  67. 67. 19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  68. 68. 19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  69. 69. 19 TP TF Fault Locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  70. 70. 19 TP TF Fault Locations Fault locations J.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedings of the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  71. 71. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 20 if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Fault Localization
  72. 72. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 21
  73. 73. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 22 Template-based Patch Candidate Generation + - + - + + Fix Template Patch Candidate Fault Location
  74. 74. 23 How a Fix Template works
  75. 75. 23 How a Fix Template works
  76. 76. 23 How a Fix Template works
  77. 77. + - + 23 How a Fix Template works
  78. 78. + - + 23 How a Fix Template works
  79. 79. + - + 23 How a Fix Template works
  80. 80. + - + 23 AST Analysis →Collects necessary AST nodes How a Fix Template works
  81. 81. + - + 23 AST Analysis →Collects necessary AST nodes Context Check →Examines applicability How a Fix Template works
  82. 82. + - + 23 AST Analysis →Collects necessary AST nodes Context Check →Examines applicability Program Editing →Modifies the source code How a Fix Template works
  83. 83. Using a Fix Template:An Example 24 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  84. 84. Using a Fix Template:An Example 24 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  85. 85. + - +Null Pointer Checker Using a Fix Template:An Example 24 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  86. 86. + - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  87. 87. + - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  88. 88. + - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  89. 89. + - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  90. 90. + - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 // deleted. 1507 } 1508 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 if( state != null && state.parens[i] != null) 1507 state.parens[i].length = 0; 1508 } 1509 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  91. 91. + - +Null Pointer Checker Using a Fix Template:An Example 24 obj ref.: state, parens[i], ... Check obj ref.: PASS Edit: Insert ... ... + if( ) { state.parens[i].length = 0; + } ... ... state != null && state.parens[i] != null 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 if( state != null && state.parens[i] != null) 1507 state.parens[i].length = 0; 1508 } 1509 state.parenCount = num; 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != -1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 { 1506 if( state != null && state.parens[i] != null) 1507 state.parens[i].length = 0; 1508 } 1509 state.parenCount = num; 1500 num = state.parenCount; 1501 int kidMatch = matchRENodes(state, (RENode)ren.kid, 1502 stop, index); 1503 if (kidMatch != 1) return kidMatch; 1504 for (int i = num; i < state.parenCount; i++) 1505 state.parens[i].length = 0; 1506 state.parenCount = num;
  92. 92. 25 List of Templates
  93. 93. 25 List of Templates Parameter Replacer Method Replacer Parameter Adder and Remover Expression Replacer Expression Adder and Remover Object Initializer Range Checker Collection Size Checker Null Pointer Checker Class Cast Checker
  94. 94. 26 Creating Patch Candidates
  95. 95. 26 Creating Patch Candidates
  96. 96. 26 Fault location #1 Fault location #2 Fault location #3 Fault location #4 Fault location #N ... Creating Patch Candidates
  97. 97. 26 Fault location #1 Fault location #2 Fault location #3 Fault location #4 Fault location #N ... Creating Patch Candidates + - + ... + - + + - + + - + + - + Fix Templates
  98. 98. 26 Fault location #1 Fault location #2 Fault location #3 Fault location #4 Fault location #N ... Creating Patch Candidates + - + ... + - + + - + + - + + - + Fix Templates ... Patch Candidates
  99. 99. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 27 Template-based Patch Candidate Generation + - + - + + Fix Template Patch Candidate Fault Location
  100. 100. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 28 Patch Evaluation
  101. 101. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation
  102. 102. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired Test Cases
  103. 103. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired Test Cases
  104. 104. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired PassTest Cases
  105. 105. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired T Repaired PassTest Cases
  106. 106. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired T Repaired Pass Fail Test Cases
  107. 107. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 29 Patch Evaluation T Repaired T Repaired + - + - + + Fix Template Patch Candidate Fault Location Pass Fail Test Cases
  108. 108. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
  109. 109. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
  110. 110. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
  111. 111. if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 30
  112. 112. 31 Evaluation: Experiment Design
  113. 113. 31 Evaluation: Experiment Design
  114. 114. 31 Evaluation: Experiment Design PAR GenProg
  115. 115. 31 Evaluation: Experiment Design PAR GenProg
  116. 116. 31 Evaluation: Experiment Design PAR GenProg
  117. 117. 31 Evaluation: Experiment Design PAR GenProg # #
  118. 118. 32 RQ1(Fixability): How many bugs are fixed successfully? RQ2(Acceptability):Which approach can generate more acceptable bug patches? Evaluation: Research Questions #
  119. 119. 33 Subject # bugs LOC # test cases Rhino 17 51,001 5,578 AspectJ 18 180,394 1,602 log4j 15 27,855 705 Math 29 121,168 3,538 Lang 20 54,537 2,051 Collections 20 48,049 11,577 Total 119 351,406 25,051 Experiment Subjects
  120. 120. 34 RQ1: Fixability
  121. 121. 34 RQ1: Fixability PAR GenProg 0 6 12 18 24 30
  122. 122. 34 RQ1: Fixability PAR GenProg 0 6 12 18 24 30 27
  123. 123. 34 RQ1: Fixability PAR GenProg 0 6 12 18 24 30 27 16
  124. 124. 34 RQ1: Fixability PAR GenProg 0 6 12 18 24 30 27 16 PAR GenProg 27 16 >
  125. 125. 35 RQ2: Acceptability
  126. 126. 35 RQ2: Acceptability User Study #1: Ranking between PAR and GenProg
  127. 127. 36
  128. 128. 36 PAR
  129. 129. 36 PAR GenProg
  130. 130. 36 PAR GenProg 5
  131. 131. 37 User Study #1 Ranking Patches
  132. 132. 37 User Study #1 Bug Description Ranking Patches
  133. 133. 37 User Study #1 Bug Description Buggy Code Ranking Patches
  134. 134. 37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Ranking Patches
  135. 135. 37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Ranking Patches
  136. 136. 37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 Ranking Patches
  137. 137. 37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 Ranking Patches
  138. 138. 37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 Rank patches 17  Students 68  Developers Ranking Patches
  139. 139. 37 User Study #1 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Anonymized Patch #3 2 1 3 Rank patches 17  Students 68  Developers Ranking Patches
  140. 140. User Study #1: Results Student  group  (avg.  ranking) 38 (the lower the better) Developer  group  (avg.  ranking) (the lower the better)
  141. 141. User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) (the lower the better)
  142. 142. User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) (the lower the better) Significantly Different
  143. 143. User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) 1 1.35 1.7 2.05 2.4 1.81 1.82 2.36 (the lower the better) PAR GenProgHuman Significantly Different
  144. 144. User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) 1 1.35 1.7 2.05 2.4 1.81 1.82 2.36 (the lower the better) PAR GenProgHuman Significantly Different Significantly Different
  145. 145. User Study #1: Results Student  group  (avg.  ranking) 38 0 0.75 1.5 2.25 3 1.72 1.57 2.67 PAR GenProgHuman (the lower the better) Developer  group  (avg.  ranking) 1 1.35 1.7 2.05 2.4 1.81 1.82 2.36 (the lower the better) PAR GenProgHuman Significantly Different Significantly Different PAR generates better ranking patches than GenProg
  146. 146. 39 RQ2: Acceptability User Study #1: Ranking between PAR and GenProg
  147. 147. 39 RQ2: Acceptability User Study #1: Ranking between PAR and GenProg User Study #2: Pair-wise Comparison between Human-written Patches Vs. PAR or GenProg
  148. 148. 40
  149. 149. 40 PAR 27
  150. 150. 40 PAR 27 GenProg 16
  151. 151. 41 User Study #2 Pair-wise Comparison
  152. 152. 41 User Study #2 Bug DescriptionPair-wise Comparison
  153. 153. 41 User Study #2 Bug Description Buggy Code Pair-wise Comparison
  154. 154. 41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Pair-wise Comparison
  155. 155. 41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Pair-wise Comparison
  156. 156. 41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Choose acceptable patch(es) 72  Students 96  Developers Pair-wise Comparison
  157. 157. 41 User Study #2 Bug Description Buggy Code Anonymized Patch #1 Anonymized Patch #2 Patch #1 Patch #2 Both Not sure Choose acceptable patch(es) 72  Students 96  Developers Pair-wise Comparison
  158. 158. User Study #2: Results GenProg 42 PAR
  159. 159. User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR
  160. 160. User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure
  161. 161. User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure 49%
  162. 162. User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure 49% 32%
  163. 163. User Study #2: Results GenProg 42 0 10 20 30 40 21 28 37 14 responses(%) PAR HumanBoth Not Sure PAR 0 15 30 45 60 20 12 51 17responses(%) GenProg HumanBoth Not Sure 49% 32% PAR generates more acceptable patches than GenProg
  164. 164. 43 Limitations • Fix templates are written manually. • But it is one-time cost, these are highly reusable. •We entirely re-implemented GenProg by Java. •All subjects are collected from open source projects. • Some participants may not be thoroughly qualified.
  165. 165. 44 Summary
  166. 166. 44 Summary Observed common patches #patches Patterns
  167. 167. 44 Summary Observed common patches #patches Patterns Fix Templates and PAR if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location
  168. 168. 44 Summary Can fix more bugs with more acceptability Observed common patches #patches Patterns Fix Templates and PAR if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 0 6 12 18 24 30 16 27 0 0.75 1.5 2.25 3 1 1.35 1.7 2.05 2.4 0 10 20 30 40 PAR HumanBoth Not Sure 0 15 30 45 60 GenProg HumanBoth Not Sure 49% 32%
  169. 169. 45 Future Work Automatic Fix Template Identification • More templates can fix more bugs. More Test Cases • More test cases may lead us to better patches.
  170. 170. 46 Acknowledgement •Westley Weimer • Claire Le Goues •Thomas Zimmermann • Christian Bird
  171. 171. 47 Summary Fix Templates and PAR Can fix more bugs with more acceptability Observed common patches #patches Patterns if(lhs == DBL_MRK) lhs = ...; if(lhs == undefined) { lhs = strings[pc + 1]; } Scriptable calleeScope = ...; Buggy Program (a) Fault Localization + - + - + + (b) Template-based Patch Candidate Generation Fail Pass (c) Patch Evaluation T Repaired Fix Template Patch Candidate Repaired Program Fault Location 0 6 12 18 24 30 16 27 0 0.75 1.5 2.25 3 1 1.35 1.7 2.05 2.4 0 10 20 30 40 PAR HumanBoth Not Sure 0 15 30 45 60 GenProg HumanBoth Not Sure 49% 32%

×