Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automatic patch generation learned from human written patches

8,576 views

Published on

Dongsun's ICSE 2013 Presentation. ACM SIGSOFT Distinguished Paper Award Winner!

Automatic patch generation learned from human written patches

  1. 1. Automatic Patch Generation Learnedfrom Human-Written PatchesDongsun Kim, Jaechang Nam, Jaewoo Song, and Sunghun KimThe Hong Kong University of Science and Technology, China24 May 2013the 35th International Conference on Software Engineering (ICSE 2013)
  2. 2. 2Repair
  3. 3. 2Repair
  4. 4. 2RepairGenProgGenProg
  5. 5. 3GenProgC. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automatedprogram repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  6. 6. 3GenProgState-of-the-artC. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automatedprogram repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  7. 7. 3GenProgState-of-the-artGenetic ProgrammingC. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automatedprogram repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  8. 8. 3GenProgState-of-the-artGenetic ProgrammingRandom MutationC. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automatedprogram repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  9. 9. 3GenProgState-of-the-artGenetic ProgrammingRandom MutationSystematically EvaluatedC. Le Goues, M. Dewey-Vogt, S. Forrest, and W.Weimer,“A systematic study of automatedprogram repair: Fixing 55 out of 105 bugs for $8 each,” in ICSE ’12.
  10. 10. 4Buggy Code
  11. 11. 1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;4in Interpreter.javareported as Mozilla Bug #76683Buggy Code
  12. 12. 1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;4in Interpreter.javareported as Mozilla Bug #76683Buggy Code
  13. 13. 1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;4in Interpreter.javareported as Mozilla Bug #76683Null Pointer ExceptionBuggy Code
  14. 14. 1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;5GenProg repairs bugs
  15. 15. 1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // deleted.1507 }1508 state.parenCount = num;5GenProg repairs bugs1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;Buggy  CodeGenProg
  16. 16. 1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // deleted.1507 }1508 state.parenCount = num;5GenProg repairs bugs1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;Buggy  CodeGenProg
  17. 17. 1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // deleted.1507 }1508 state.parenCount = num;5GenProg repairs bugs1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;Buggy  CodeGenProgThis patch passes ALL test cases.
  18. 18. 6GenProg repairs bugs1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // deleted.1507 }1508 state.parenCount = num;
  19. 19. 6GenProg repairs bugs1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // deleted.1507 }1508 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // deleted.1507 }1508 state.parenCount = num;
  20. 20. 1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;6GenProg repairs bugs1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // deleted.1507 }1508 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // deleted.1507 }1508 state.parenCount = num;
  21. 21. 7Would you accept?
  22. 22. 71500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // do nothing.1507 }1508 state.parenCount = num;Would you accept?
  23. 23. 71500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // do nothing.1507 }1508 state.parenCount = num;Would you accept?
  24. 24. 71500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // do nothing.1507 }1508 state.parenCount = num;Would you accept?17  Students68  Developers
  25. 25. 81500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // do nothing.1507 }1508 state.parenCount = num;Would you accept?17  Students68  Developers
  26. 26. 81500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // do nothing.1507 }1508 state.parenCount = num;Would you accept?9.4%90.6%17  Students68  Developers
  27. 27. 9
  28. 28. 9
  29. 29. 9•Random
  30. 30. 9•Random•Unnatural
  31. 31. 9•Random•Unnatural•Harder  to  understand
  32. 32. 10
  33. 33. 10
  34. 34. 10RandomEvolution
  35. 35. 10RandomEvolutionExplores the largesearch space
  36. 36. 10RandomEvolutionExplores the largesearch spacebut sometimesproducesrandom results
  37. 37. 10RandomEvolutionIntelligentDesignExplores the largesearch spacebut sometimesproducesrandom results
  38. 38. 10RandomEvolutionIntelligentDesignExplores the largesearch space Leverages humanknowledgebut sometimesproducesrandom results
  39. 39. 11Human-written Patches
  40. 40. 11Human-written PatchesReadable
  41. 41. 11Human-written PatchesReadableNatural
  42. 42. 11Human-written PatchesReadableNaturalEasy to understand
  43. 43. 11Human-written PatchesReadableNaturalEasy to understandWe can learn how to generatepatches from human knowledge.
  44. 44. 12
  45. 45. 12JDT
  46. 46. 12>60,000PatchesJDT
  47. 47. 12ManualClassification>60,000PatchesJDT
  48. 48. 12ManualClassification#patchesPatterns>60,000PatchesJDT
  49. 49. 12ManualClassification#patchesPatterns>60,000PatchesJDT
  50. 50. 12ManualClassification#patchesPatternsTop frequentpatternsaccount for >20~30%>60,000PatchesJDT
  51. 51. 13Common Fix PatternsAltering method parametersobj.method(v1,v2)0→0obj.method(v1,v3)Altering method parametersobj.method(v1,v2)0→0obj.method(v1,v3)
  52. 52. 13Common Fix PatternsAdding a checkerobj.m1())→)if(obj!=null)){obj.m1()}Adding a checkerobj.m1())→)if(obj!=null)){obj.m1()}
  53. 53. 14PARPattern-based Automatic Program Repair
  54. 54. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation15
  55. 55. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation15+-+Human Knowledge
  56. 56. 16+-+Using HumanKnowledgefor patch generation
  57. 57. 16+-+Fix TemplatesUsing HumanKnowledgefor patch generation
  58. 58. 16+-+Fix TemplatesProgram Edit ScriptUsing HumanKnowledgefor patch generation
  59. 59. 16+-+Fix TemplatesProgram Edit ScriptUsing HumanKnowledgefor patch generation10
  60. 60. 16+-+Fix TemplatesProgram Edit ScriptManually created from fix patternsUsing HumanKnowledgefor patch generation10
  61. 61. 16+-+Fix TemplatesProgram Edit ScriptManually created from fix patternsHighly reusableUsing HumanKnowledgefor patch generation10
  62. 62. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation17
  63. 63. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation18if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;Fault Localization
  64. 64. 19FaultLocationsJ.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedingsof the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  65. 65. 19TP TFFaultLocationsJ.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedingsof the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  66. 66. 19TP TFFaultLocationsJ.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedingsof the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  67. 67. 19TP TFFaultLocationsJ.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedingsof the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  68. 68. 19TP TFFaultLocationsJ.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedingsof the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  69. 69. 19TP TFFaultLocationsJ.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedingsof the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  70. 70. 19TP TFFaultLocationsFaultlocationsJ.A. Jones, M. J. Harrold, and J. Stasko,“Visualization of test information to assist fault localization,” in Proceedingsof the 24th International Conference on Software Engineering, NewYork, NY, USA, 2002, pp. 467–477.
  71. 71. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation20if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;Fault Localization
  72. 72. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation21
  73. 73. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation22Template-basedPatch Candidate Generation+-+-++FixTemplatePatchCandidateFaultLocation
  74. 74. 23How a Fix Template works
  75. 75. 23How a Fix Template works
  76. 76. 23How a Fix Template works
  77. 77. +-+23How a Fix Template works
  78. 78. +-+23How a Fix Template works
  79. 79. +-+23How a Fix Template works
  80. 80. +-+23AST Analysis→Collects necessary AST nodesHow a Fix Template works
  81. 81. +-+23AST Analysis→Collects necessary AST nodesContext Check→Examines applicabilityHow a Fix Template works
  82. 82. +-+23AST Analysis→Collects necessary AST nodesContext Check→Examines applicabilityProgram Editing→Modifies the source codeHow a Fix Template works
  83. 83. Using a Fix Template:An Example241500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;
  84. 84. Using a Fix Template:An Example241500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;
  85. 85. +-+Null Pointer CheckerUsing a Fix Template:An Example241500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;
  86. 86. +-+Null Pointer CheckerUsing a Fix Template:An Example24obj ref.: state, parens[i], ...1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;
  87. 87. +-+Null Pointer CheckerUsing a Fix Template:An Example24obj ref.: state, parens[i], ...Check obj ref.: PASS1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;
  88. 88. +-+Null Pointer CheckerUsing a Fix Template:An Example24obj ref.: state, parens[i], ...Check obj ref.: PASSEdit: Insert......+ if( ) {state.parens[i].length = 0;+ }......state != null && state.parens[i] != null1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;
  89. 89. +-+Null Pointer CheckerUsing a Fix Template:An Example24obj ref.: state, parens[i], ...Check obj ref.: PASSEdit: Insert......+ if( ) {state.parens[i].length = 0;+ }......state != null && state.parens[i] != null1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;
  90. 90. +-+Null Pointer CheckerUsing a Fix Template:An Example24obj ref.: state, parens[i], ...Check obj ref.: PASSEdit: Insert......+ if( ) {state.parens[i].length = 0;+ }......state != null && state.parens[i] != null1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != -1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != -1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 // deleted.1507 }1508 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != -1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != -1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 if( state != null && state.parens[i] != null)1507 state.parens[i].length = 0;1508 }1509 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;
  91. 91. +-+Null Pointer CheckerUsing a Fix Template:An Example24obj ref.: state, parens[i], ...Check obj ref.: PASSEdit: Insert......+ if( ) {state.parens[i].length = 0;+ }......state != null && state.parens[i] != null1502 stop, index);1503 if (kidMatch != -1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != -1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 if( state != null && state.parens[i] != null)1507 state.parens[i].length = 0;1508 }1509 state.parenCount = num;1502 stop, index);1503 if (kidMatch != -1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != -1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 {1506 if( state != null && state.parens[i] != null)1507 state.parens[i].length = 0;1508 }1509 state.parenCount = num;1500 num = state.parenCount;1501 int kidMatch = matchRENodes(state, (RENode)ren.kid,1502 stop, index);1503 if (kidMatch != 1) return kidMatch;1504 for (int i = num; i < state.parenCount; i++)1505 state.parens[i].length = 0;1506 state.parenCount = num;
  92. 92. 25List of Templates
  93. 93. 25List of TemplatesParameter ReplacerMethod ReplacerParameter Adderand RemoverExpression ReplacerExpression Adder andRemoverObject InitializerRange CheckerCollection SizeCheckerNull Pointer CheckerClass Cast Checker
  94. 94. 26Creating Patch Candidates
  95. 95. 26Creating Patch Candidates
  96. 96. 26Fault location #1Fault location #2Fault location #3Fault location #4Fault location #N...Creating Patch Candidates
  97. 97. 26Fault location #1Fault location #2Fault location #3Fault location #4Fault location #N...Creating Patch Candidates+-+...+-++-++-++-+FixTemplates
  98. 98. 26Fault location #1Fault location #2Fault location #3Fault location #4Fault location #N...Creating Patch Candidates+-+...+-++-++-++-+FixTemplates...PatchCandidates
  99. 99. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation27Template-basedPatch Candidate Generation+-+-++FixTemplatePatchCandidateFaultLocation
  100. 100. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation28Patch Evaluation
  101. 101. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation29Patch Evaluation
  102. 102. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation29Patch EvaluationTRepairedTestCases
  103. 103. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation29Patch EvaluationTRepairedTestCases
  104. 104. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation29Patch EvaluationTRepairedPassTestCases
  105. 105. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation29Patch EvaluationTRepairedTRepairedPassTestCases
  106. 106. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation29Patch EvaluationTRepairedTRepairedPassFailTestCases
  107. 107. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation29Patch EvaluationTRepairedTRepaired+-+-++FixTemplatePatchCandidateFaultLocationPassFailTestCases
  108. 108. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation30
  109. 109. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation30
  110. 110. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation30
  111. 111. if(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation30
  112. 112. 31Evaluation:Experiment Design
  113. 113. 31Evaluation:Experiment Design
  114. 114. 31Evaluation:Experiment DesignPAR GenPro
  115. 115. 31Evaluation:Experiment DesignPAR GenPro
  116. 116. 31Evaluation:Experiment DesignPAR GenPro
  117. 117. 31Evaluation:Experiment DesignPAR GenPro# #
  118. 118. 32RQ1(Fixability): How many bugsare fixed successfully?RQ2(Acceptability):Which approachcan generate more acceptable bugpatches?Evaluation:Research Questions#
  119. 119. 33Subject # bugs LOC # test casesRhino 17 51,001 5,578AspectJ 18 180,394 1,602log4j 15 27,855 705Math 29 121,168 3,538Lang 20 54,537 2,051Collections 20 48,049 11,577Total 119 351,406 25,051Experiment Subjects
  120. 120. 34RQ1: Fixability
  121. 121. 34RQ1: FixabilityPAR GenProg0612182430
  122. 122. 34RQ1: FixabilityPAR GenProg061218243027
  123. 123. 34RQ1: FixabilityPAR GenProg06121824302716
  124. 124. 34RQ1: FixabilityPAR GenProg06121824302716PAR GenProg27 16>
  125. 125. 35RQ2: Acceptability
  126. 126. 35RQ2: AcceptabilityUser Study #1: Ranking betweenPAR and GenProg
  127. 127. 36
  128. 128. 36PAR
  129. 129. 36PAR GenProg
  130. 130. 36PAR GenProg5
  131. 131. 37User Study #1Ranking Patches
  132. 132. 37User Study #1Bug DescriptionRanking Patches
  133. 133. 37User Study #1Bug DescriptionBuggy CodeRanking Patches
  134. 134. 37User Study #1Bug DescriptionBuggy CodeAnonymized Patch #1Ranking Patches
  135. 135. 37User Study #1Bug DescriptionBuggy CodeAnonymized Patch #1Anonymized Patch #2Ranking Patches
  136. 136. 37User Study #1Bug DescriptionBuggy CodeAnonymized Patch #1Anonymized Patch #2Anonymized Patch #3Ranking Patches
  137. 137. 37User Study #1Bug DescriptionBuggy CodeAnonymized Patch #1Anonymized Patch #2Anonymized Patch #3Ranking Patches
  138. 138. 37User Study #1Bug DescriptionBuggy CodeAnonymized Patch #1Anonymized Patch #2Anonymized Patch #3Rank patches17  Students68  DevelopersRanking Patches
  139. 139. 37User Study #1Bug DescriptionBuggy CodeAnonymized Patch #1Anonymized Patch #2Anonymized Patch #3213Rank patches17  Students68  DevelopersRanking Patches
  140. 140. User Study #1: ResultsStudent  group  (avg.  ranking)38(the lower the better)Developer  group  (avg.  ranking)(the lower the better)
  141. 141. User Study #1: ResultsStudent  group  (avg.  ranking)3800.751.52.2531.72 1.572.67PAR GenProgHuman(the lower the better)Developer  group  (avg.  ranking)(the lower the better)
  142. 142. User Study #1: ResultsStudent  group  (avg.  ranking)3800.751.52.2531.72 1.572.67PAR GenProgHuman(the lower the better)Developer  group  (avg.  ranking)(the lower the better)SignificantlyDifferent
  143. 143. User Study #1: ResultsStudent  group  (avg.  ranking)3800.751.52.2531.72 1.572.67PAR GenProgHuman(the lower the better)Developer  group  (avg.  ranking)11.351.72.052.41.81 1.822.36(the lower the better)PARGenProgHumanSignificantlyDifferent
  144. 144. User Study #1: ResultsStudent  group  (avg.  ranking)3800.751.52.2531.72 1.572.67PAR GenProgHuman(the lower the better)Developer  group  (avg.  ranking)11.351.72.052.41.81 1.822.36(the lower the better)PARGenProgHumanSignificantlyDifferentSignificantlyDifferent
  145. 145. User Study #1: ResultsStudent  group  (avg.  ranking)3800.751.52.2531.72 1.572.67PAR GenProgHuman(the lower the better)Developer  group  (avg.  ranking)11.351.72.052.41.81 1.822.36(the lower the better)PARGenProgHumanSignificantlyDifferentSignificantlyDifferentPAR generates better rankingpatches than GenProg
  146. 146. 39RQ2: AcceptabilityUser Study #1: Ranking betweenPAR and GenProg
  147. 147. 39RQ2: AcceptabilityUser Study #1: Ranking betweenPAR and GenProgUser Study #2: Pair-wise ComparisonbetweenHuman-written PatchesVs.PAR or GenProg
  148. 148. 40
  149. 149. 40PAR27
  150. 150. 40PAR27GenProg16
  151. 151. 41User Study #2Pair-wise Comparison
  152. 152. 41User Study #2Bug DescriptionPair-wise Comparison
  153. 153. 41User Study #2Bug DescriptionBuggy CodePair-wise Comparison
  154. 154. 41User Study #2Bug DescriptionBuggy CodeAnonymized Patch #1Pair-wise Comparison
  155. 155. 41User Study #2Bug DescriptionBuggy CodeAnonymized Patch #1Anonymized Patch #2Pair-wise Comparison
  156. 156. 41User Study #2Bug DescriptionBuggy CodeAnonymized Patch #1Anonymized Patch #2Choose acceptablepatch(es)72  Students96  DevelopersPair-wise Comparison
  157. 157. 41User Study #2Bug DescriptionBuggy CodeAnonymized Patch #1Anonymized Patch #2Patch #1 Patch #2Both Not sureChoose acceptablepatch(es)72  Students96  DevelopersPair-wise Comparison
  158. 158. User Study #2: ResultsGenProg42PAR
  159. 159. User Study #2: ResultsGenProg4201020304021283714responses(%)PAR HumanBoth NotSurePAR
  160. 160. User Study #2: ResultsGenProg4201020304021283714responses(%)PAR HumanBoth NotSurePAR01530456020125117responses(%)GenProg HumanBoth NotSure
  161. 161. User Study #2: ResultsGenProg4201020304021283714responses(%)PAR HumanBoth NotSurePAR01530456020125117responses(%)GenProg HumanBoth NotSure49%
  162. 162. User Study #2: ResultsGenProg4201020304021283714responses(%)PAR HumanBoth NotSurePAR01530456020125117responses(%)GenProg HumanBoth NotSure49%32%
  163. 163. User Study #2: ResultsGenProg4201020304021283714responses(%)PAR HumanBoth NotSurePAR01530456020125117responses(%)GenProg HumanBoth NotSure49%32%PAR generates moreacceptable patches than GenProg
  164. 164. 43Limitations• Fix templates are written manually.• But it is one-time cost, these are highly reusable.•We entirely re-implemented GenProg by Java.•All subjects are collected from open source projects.• Some participants may not be thoroughly qualified.
  165. 165. 44Summary
  166. 166. 44SummaryObserved common patches#patchesPatterns
  167. 167. 44SummaryObserved common patches#patchesPatternsFix Templates and PARif(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation
  168. 168. 44SummaryCan fix more bugs with more acceptabilityObserved common patches#patchesPatternsFix Templates and PARif(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation0612182430162700.751.52.25311.351.72.052.4010203040PAR HumanBoth NotSure015304560GenProg HumanBoth NotSure49%32%
  169. 169. 45Future WorkAutomatic Fix Template Identification• More templates can fix more bugs.More Test Cases• More test cases may lead us to better patches.
  170. 170. 46AcknowledgementSpecial Thanks to:Westley Weimer,Claire Le Goues,Thomas Zimmermann,and Christian Bird
  171. 171. 47SummaryFix Templates and PARCan fix more bugs with more acceptabilityObserved common patches#patchesPatternsif(lhs == DBL_MRK) lhs = ...;if(lhs == undefined) {lhs = strings[pc + 1];}Scriptable calleeScope = ...;BuggyProgram(a) FaultLocalization+-+-++(b) Template-basedPatch Candidate GenerationFailPass(c) Patch EvaluationTRepairedFixTemplatePatchCandidateRepairedProgramFaultLocation0612182430162700.751.52.25311.351.72.052.4010203040PAR HumanBoth NotSure015304560GenProg HumanBoth NotSure49%32%

×