Downloaded 38 times
Uploaded byThe University of Adelaide
Who Should Review My Code?
The document discusses a file location-based code-reviewer recommendation tool called RevFinder, aimed at improving the efficiency of code reviews in software development. It highlights the impact of code-reviewer assignment problems on review times, with findings indicating that such problems can lead to approvals taking significantly longer. The evaluation results show that RevFinder outperforms previous methods, accurately recommending reviewers for up to 87% of cases.
![[India Merge World Tour] Coverity](https://cdn.slidesharecdn.com/ss_thumbnails/india-mergeworldtourcoverity-130819053319-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Who Should Review My Code?
- 1. Who Should ReviewMy Code? A File Location-Based Code-Reviewer Recommendation Approach for Modern Code Review Patanamon (Pick) Thongtanunam Chakkrit Tantithamtavorn Raula G. Kula Norihiro Yoshida Hajimu Iida Ken-ichi Matsumoto patanamon-t@is.naist.jp @pamon 22nd IEEE International Conference on Software Analysis, Evolution, and Reengineering Montréal, Quebec, Canada March 3 - 6, 2015 1
- 2. Modern Code Reviewis a tool-based, and used regularly in practice nowadays at industrial and OSS projects 2
- 3. Gerrit Code Review Rietveld Modern CodeReview is a tool-based, and used regularly in practice nowadays at industrial and OSS projects 2
- 4. Gerrit Code Review Rietveld Modern CodeReview is a tool-based, and used regularly in practice nowadays at industrial and OSS projects 2
- 5.
- 6. Modern Code Review Developer Codechange Gerrit Code Review 3
- 7. Modern Code Review Developer Codechange Gerrit Code Review 3
- 8. Modern Code Review Reviewers Developer Canyou review my code? Code change Gerrit Code Review Invite Reviewers 3
- 9. Modern Code Review Codingstyle passed :) There’s a bug! Reviewers Developer Can you review my code? Code change Gerrit Code Review Invite Reviewers 3
- 10. Code-Reviewer Assignment Problem Codechange Gerrit Code Review Invite Reviewers Developer 4
- 11. Code-Reviewer Assignment Problem Whoshould review my code? Code change Gerrit Code Review Invite Reviewers NewDeveloper 4
- 12. Code-Reviewer Assignment Problem Whoshould review my code? Code change Gerrit Code Review Invite Reviewers NewDeveloper 4
- 13. RQ1: How doreviews with code-reviewer assignment problem impact reviewing time? 5
- 14. RQ1: How doreviews with code-reviewer assignment problem impact reviewing time? Studied Projects 5
- 15. RQ1: How doreviews with code-reviewer assignment problem impact reviewing time? Studied Projects Sample Reviews 357 363 378 363 5
- 16. RQ1: How doreviews with code-reviewer assignment problem impact reviewing time? Studied Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% 5
- 17. RQ1: How doreviews with code-reviewer assignment problem impact reviewing time? Studied Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment pro Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviewing Time (Days) Reviews with code−reviewer assignment problem viewingtime(Days) 102030 eviews with code−reviewer assignment problem Reviews without code−revReview w/ Assignment Problem Review w/o Assignment Problem 18 Days Longer 13 Days Longer 9 Days Longer 6 Days Longer 5
- 18. RQ1: How doreviews with code-reviewer assignment problem impact reviewing time? Studied Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% Reviews with code-reviewer assignment problem significantly take 12 days longer to approve a code change. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment pro Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviewing Time (Days) Reviews with code−reviewer assignment problem viewingtime(Days) 102030 eviews with code−reviewer assignment problem Reviews without code−revReview w/ Assignment Problem Review w/o Assignment Problem 18 Days Longer 13 Days Longer 9 Days Longer 6 Days Longer 5
- 19. 6 RQ1: How doreviews with code-reviewer assignment problem impact reviewing time?
- 20. 6 RQ1: How doreviews with code-reviewer assignment problem impact reviewing time?
- 21. 6 RQ1: How doreviews with code-reviewer assignment problem impact reviewing time?
- 22. 6 RQ1: How doreviews with code-reviewer assignment problem impact reviewing time?
- 23. A code-reviewer recommendationtool will be useful in distributed software development to speed up the code review process. 6 RQ1: How do reviews with code-reviewer assignment problem impact reviewing time?
- 24. RevFinder: A FileLocation-based Code-Reviewer Recommendation Approach 7
- 25. RevFinder: Overview R1 R2 ChangedFiles - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat Review History 8
- 26. RevFinder: Overview R1 R2 ChangedFiles - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? Review History 8
- 27. RevFinder: Overview R1 R2 ChangedFiles - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History Find reviewers who have reviewed the most similar file paths 8
- 28. RevFinder: Overview R1 R2 ChangedFiles - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History 8
- 29. RevFinder: Overview R1 R2 ChangedFiles - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare Compare(FilePath_R2,FilePath_R3) = 0.3 Review History 8
- 30. RevFinder: Overview R1 R2 ChangedFiles - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Compare(FilePath_R2,FilePath_R3) = 0.3 Compare(FilePath_R1,FilePath_R3) = 0.6 Review History 8 Compare
- 31. RevFinder: Overview R1 R2 ChangedFiles - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History Compare(FilePath_R2,FilePath_R3) = 0.3 Compare(FilePath_R1,FilePath_R3) = 0.6 9
- 32. RevFinder: Overview R1 R2 ChangedFiles - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History Compare(FilePath_R1,FilePath_R3) = 0.6 “Pat should review R3” 9
- 33. RevFinder: Overview R1 R2 ChangedFiles - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History Compare(FilePath_R1,FilePath_R3) = 0.6 “Pat should review R3” 9
- 34. RevFinder: FilePath ComparisonTechniques Longest Common Prefix Longest Common Suffix Longest Common Substring Longest Common Subsequence 4 String Comparison Techniques 10
- 35. RevFinder: FilePath ComparisonTechniques Longest Common Prefix Longest Common Suffix Longest Common Substring Longest Common Subsequence 4 String Comparison Techniques 10
- 36. RevFinder: FilePath ComparisonTechniques Longest Common Prefix Longest Common Suffix Longest Common Substring Longest Common Subsequence Recommendation List Combine (Borda Count) 4 String Comparison Techniques 10
- 37. RevFinder: FilePath ComparisonTechniques Longest Common Prefix Longest Common Suffix Longest Common Substring Longest Common Subsequence Recommendation List Combine (Borda Count) 4 String Comparison Techniques Voting Ranks 10
- 38. RevFinder: FilePath ComparisonTechniques Longest Common Prefix Longest Common Suffix Longest Common Substring Longest Common Subsequence Recommendation List Combine (Borda Count) 4 String Comparison Techniques Voting Ranks 10 Bubble-up correct reviewers
- 39. Empirical Evaluation Studied Projects Period~4 years ~1year ~1 year ~2 year #Reviews 5,126 6,586 23,810 6,523 #Reviewers 94 82 202 63 11
- 40. Empirical Evaluation Studied Projects Period~4 years ~1year ~1 year ~2 year #Reviews 5,126 6,586 23,810 6,523 #Reviewers 94 82 202 63 Baseline Approach: ReviewBot (Balachandran ICSE’13) R1 R2 R3 (New) JohnPat “Recommend reviewers who have reviewed the same line of code changes” 11
- 41. Empirical Evaluation Studied Projects Period~4 years ~1year ~1 year ~2 year #Reviews 5,126 6,586 23,810 6,523 #Reviewers 94 82 202 63 Baseline Approach: ReviewBot (Balachandran ICSE’13) R1 R2 R3 (New) JohnPat “Recommend reviewers who have reviewed the same line of code changes” John should review R3 11
- 42.
- 43. Empirical Evaluation Accuracy “RQ2: DoesRevFinder accurately recommend reviewers?” 12
- 44. Empirical Evaluation Accuracy “RQ2: DoesRevFinder accurately recommend reviewers?” Ranking Performance “RQ3: Does RevFinder provide a better ranking of correct reviewers?” 12
- 45. Empirical Evaluation Accuracy “RQ2: DoesRevFinder accurately recommend reviewers?” Ranking Performance “RQ3: Does RevFinder provide a better ranking of correct reviewers?” 12
- 46. Empirical Evaluation Accuracy “RQ2: DoesRevFinder accurately recommend reviewers?” Measure: Top-k Accuracy Ranking Performance “RQ3: Does RevFinder provide a better ranking of correct reviewers?” 12
- 47. Empirical Evaluation: Accuracy Top-kAccuracy measures how many reviews does an approach can correctly recommend with a Top-k recommendation. 13
- 48. Empirical Evaluation: Accuracy Top-kAccuracy measures how many reviews does an approach can correctly recommend with a Top-k recommendation. 13 Top-5 Recommendation
- 49. Empirical Evaluation: Accuracy Top-kAccuracy measures how many reviews does an approach can correctly recommend with a Top-k recommendation. 13 Top-5 Recommendation Actual Reviewer
- 50. Empirical Evaluation: Accuracy Top-kAccuracy measures how many reviews does an approach can correctly recommend with a Top-k recommendation. 13 Top-5 Recommendation Actual Reviewer Correct!
- 51. Empirical Evaluation: Accuracy Top-kAccuracy measures how many reviews does an approach can correctly recommend with a Top-k recommendation. Top-10Accuracy 0% 22.5% 45% 67.5% 90% 10% 28% 41% 29% 74% 69% 87%86% RevFinder ReviewBot RevFinder can correctly recommended 69%-87% of reviews with a top-10 recommendation. RevFinder is 4 times more accurate than ReviewBot. 13 Top-5 Recommendation Actual Reviewer Correct!
- 52. Empirical Evaluation Accuracy “Does RevFinderaccurately recommend reviewers?” Measure: Top-k Accuracy Ranking Performance “Does RevFinder provide a better ranking of correct reviewers?” 14
- 53. Empirical Evaluation Accuracy “Does RevFinderaccurately recommend reviewers?” Measure: Top-k Accuracy Ranking Performance “Does RevFinder provide a better ranking of correct reviewers?” 14 On average, RevFinder can correctly recommend reviewers for 79% of reviews with a top-10 recommendation.
- 54. Empirical Evaluation Accuracy “Does RevFinderaccurately recommend reviewers?” Measure: Top-k Accuracy Ranking Performance “Does RevFinder provide a better ranking of correct reviewers?” 14 On average, RevFinder can correctly recommend reviewers for 79% of reviews with a top-10 recommendation.
- 55. Empirical Evaluation Accuracy “Does RevFinderaccurately recommend reviewers?” Measure: Top-k Accuracy Ranking Performance “Does RevFinder provide a better ranking of correct reviewers?” 14 On average, RevFinder can correctly recommend reviewers for 79% of reviews with a top-10 recommendation. Measure: Mean Reciprocal Rank (MRR)
- 56. Empirical Evaluation: RankingPerformance Mean Reciprocal Rank measures an average rank of correct reviewer that an approach can recommend. 15
- 57. Empirical Evaluation: RankingPerformance Mean Reciprocal Rank measures an average rank of correct reviewer that an approach can recommend. 15 Recommendation Lists vs Actual Reviewer
- 58. Empirical Evaluation: RankingPerformance Mean Reciprocal Rank measures an average rank of correct reviewer that an approach can recommend. 15 Recommendation Lists vs Actual Reviewer MRR = 1/3
- 59. Empirical Evaluation: RankingPerformance Mean Reciprocal Rank measures an average rank of correct reviewer that an approach can recommend. 15 Recommendation Lists vs Actual Reviewer MRR = 1/3 MRR = 1/1
- 60. Empirical Evaluation: RankingPerformance Mean Reciprocal Rank measures an average rank of correct reviewer that an approach can recommend. RevFinder recommended the correct code-reviewers with a median rank of 2 - 8. The code-reviewers ranking of RevFinder is 3 times better than that of ReviewBot. 15 Recommendation Lists vs Actual Reviewer MRR = 1/3 MRR RevFinder 0.6 0.55 0.31 0.40 ReviewBot 0.25 0.30 0.22 0.07 Rankof CorrectReviewers Android OpenStack Qt LibreOffice ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●● ● ● ● ●●●●● ●●● ●●●●●●●●●●●● ● ●●●●●● ●●● ●●●● ● ● ●●● ●●●●●● ● ●●●●● ● ●● ● ●●●●●●● ● ●●●● ● ●● ● ●●●●●●●●●● ● ●●● ● ●●● ● ●●●●●● ● ● ● ●● ● ●●● ● ● ●● ● ●●● ●● ●● ● ● ●●●● ● ●●●●● ●● ● ● ●● ● ●●●●●●●●●●●● ● ● ●●●● ● ●● ● ●●●●●●●●● ● ● ● ●●● ● ● ● ● ● ●●● ● ●● ● ● ●● ●●●●● ● ●●● ● ● ●● ● ●● ● ● ●●●● ● ●●●●●●● ●● ●●●●● ●● ● ● ●●●●●●●●● ● ●●●● ● ● ●●●●●●●●●●●●●● ●● ● ● ●● ● ●● ● ●● ● ● ●● ●● ●●●●●● ● ●●● ●● ● ● ● ● ●●● ● ●●●●●●●● ● ●●●●●● ●● ●● ● ●●●● ● ● ● ●●● ● ● ● ● ● ●● ● ●●● ● ● ●●● ● ● ●●●●● ● ● ● ●●●● ● ● ● ● ●●●●●●●●●● ● ● ● ●●● ●●● ●●●●●●●●●●● ● ● ● ● ●●●● ● ●● ●● ● ● ● ● ●●●●●●● ● ● ● ● ●● ●● ● ● ● ● ● ●●●●● ● ●●● ● ●●●●● ● ● ● ● ●●● ● ● ● ●●●●●●● ● ● ● ●● ● ●●●●● ●● ● ●●●● ● ●●●●● ● ● ●● ● ● ● ●●●●● ● ●●●●●● ● ●●● ●● ●●●● ● ●● ● ● ●● ● ● ●●●●● ● ● ●● ● ●●●●●●●●●● ● ●●●●● ●● ●●● ●● ●● ● ● ●● ● ● ● ● ●● ● ●● ●●●●● ● 1 5 15 50 100 200 RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot Approach Rank RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot MRR = 1/1
- 61. Empirical Evaluation: RankingPerformance Mean Reciprocal Rank measures an average rank of correct reviewer that an approach can recommend. RevFinder recommended the correct code-reviewers with a median rank of 2 - 8. The code-reviewers ranking of RevFinder is 3 times better than that of ReviewBot. 15 Recommendation Lists vs Actual Reviewer MRR = 1/3 MRR RevFinder 0.6 0.55 0.31 0.40 ReviewBot 0.25 0.30 0.22 0.07 Rankof CorrectReviewers Android OpenStack Qt LibreOffice ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●● ● ● ● ●●●●● ●●● ●●●●●●●●●●●● ● ●●●●●● ●●● ●●●● ● ● ●●● ●●●●●● ● ●●●●● ● ●● ● ●●●●●●● ● ●●●● ● ●● ● ●●●●●●●●●● ● ●●● ● ●●● ● ●●●●●● ● ● ● ●● ● ●●● ● ● ●● ● ●●● ●● ●● ● ● ●●●● ● ●●●●● ●● ● ● ●● ● ●●●●●●●●●●●● ● ● ●●●● ● ●● ● ●●●●●●●●● ● ● ● ●●● ● ● ● ● ● ●●● ● ●● ● ● ●● ●●●●● ● ●●● ● ● ●● ● ●● ● ● ●●●● ● ●●●●●●● ●● ●●●●● ●● ● ● ●●●●●●●●● ● ●●●● ● ● ●●●●●●●●●●●●●● ●● ● ● ●● ● ●● ● ●● ● ● ●● ●● ●●●●●● ● ●●● ●● ● ● ● ● ●●● ● ●●●●●●●● ● ●●●●●● ●● ●● ● ●●●● ● ● ● ●●● ● ● ● ● ● ●● ● ●●● ● ● ●●● ● ● ●●●●● ● ● ● ●●●● ● ● ● ● ●●●●●●●●●● ● ● ● ●●● ●●● ●●●●●●●●●●● ● ● ● ● ●●●● ● ●● ●● ● ● ● ● ●●●●●●● ● ● ● ● ●● ●● ● ● ● ● ● ●●●●● ● ●●● ● ●●●●● ● ● ● ● ●●● ● ● ● ●●●●●●● ● ● ● ●● ● ●●●●● ●● ● ●●●● ● ●●●●● ● ● ●● ● ● ● ●●●●● ● ●●●●●● ● ●●● ●● ●●●● ● ●● ● ● ●● ● ● ●●●●● ● ● ●● ● ●●●●●●●●●● ● ●●●●● ●● ●●● ●● ●● ● ● ●● ● ● ● ● ●● ● ●● ●●●●● ● 1 5 15 50 100 200 RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot Approach Rank RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot RevFinder ReviewBot MRR = 1/1
- 62. Empirical Evaluation Accuracy “Does RevFinderaccurately recommend reviewers?” Measure: Top-k Accuracy Ranking Performance “Does RevFinder provide a better ranking of correct reviewers?” 16 On average, RevFinder can correctly recommend reviewers for 79% of reviews with a top-10 recommendation. Measure: Mean Reciprocal Rank (MRR)
- 63. Empirical Evaluation Accuracy “Does RevFinderaccurately recommend reviewers?” Measure: Top-k Accuracy Ranking Performance “Does RevFinder provide a better ranking of correct reviewers?” 16 On average, RevFinder can correctly recommend reviewers for 79% of reviews with a top-10 recommendation. Measure: Mean Reciprocal Rank (MRR) On average, RevFinder can recommend reviewers with a median rank of 4.
- 64. Empirical Evaluation Accuracy “Does RevFinderaccurately recommend reviewers?” Measure: Top-k Accuracy Ranking Performance “Does RevFinder provide a better ranking of correct reviewers?” 16 On average, RevFinder can correctly recommend reviewers for 79% of reviews with a top-10 recommendation. Measure: Mean Reciprocal Rank (MRR) On average, RevFinder can recommend reviewers with a median rank of 4. We can use a similarity of previously reviewed file paths to recommend code-reviewers for new code changes.
- 65.
- 66. Code-Reviewer Assignment Problem Whoshould review my code? Code change Gerrit Code Review Invite Reviewers NewDeveloper 4
- 67. Code-Reviewer Assignment Problem Whoshould review my code? Code change Gerrit Code Review Invite Reviewers NewDeveloper 4
- 68. Code-Reviewer Assignment Problem Whoshould review my code? Code change Gerrit Code Review Invite Reviewers NewDeveloper 4 5 How do reviews with code-reviewer assignment problem impact reviewing time? Studied Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% Reviews with code-reviewer assignment problem significantly take 12 days longer to approve a code change. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment pro Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviewing Time (Days) Reviews with code−reviewer assignment problem viewingtime(Days) 102030 eviews with code−reviewer assignment problem Reviews without code−revReview w/ Assignment Problem Review w/o Assignment Problem 18 Days Longer 13 Days Longer 9 Days Longer 6 Days Longer
- 69. Code-Reviewer Assignment Problem Whoshould review my code? Code change Gerrit Code Review Invite Reviewers NewDeveloper 4 5 How do reviews with code-reviewer assignment problem impact reviewing time? Studied Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% Reviews with code-reviewer assignment problem significantly take 12 days longer to approve a code change. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment proble Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviewing Time (Days) Reviews with code−reviewer assignment problem Studie Reviewingtime(Days) Android OpenStack 0102030 Reviews with code−reviewer assignment problem Reviews without code−revie Studied systems Reviewingtime(Days) Android OpenStack Qt 0102030 Review w/ Assignment Problem Review w/o Assignment Problem 18 Days Longer 13 Days Longer 9 Days Longer 6 Days Longer
- 70. 5 How do reviewswith code-reviewer assignment problem impact reviewing time? Studied Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% Reviews with code-reviewer assignment problem significantly take 12 days longer to approve a code change. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment proble Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviewing Time (Days) Reviews with code−reviewer assignment problem Studie Reviewingtime(Days) Android OpenStack 0102030 Reviews with code−reviewer assignment problem Reviews without code−revi Studied systems Reviewingtime(Days) Android OpenStack Qt 0102030 Review w/ Assignment Problem Review w/o Assignment Problem 18 Days Longer 13 Days Longer 9 Days Longer 6 Days Longer Code-Reviewer Assignment Problem Who should review my code? Code change Gerrit Code Review Invite Reviewers NewDeveloper 4 7 RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History Find reviewers who have reviewed the most similar file paths
- 71. 5 How do reviewswith code-reviewer assignment problem impact reviewing time? Studied Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% Reviews with code-reviewer assignment problem significantly take 12 days longer to approve a code change. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment proble Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviewing Time (Days) Reviews with code−reviewer assignment problem Studie Reviewingtime(Days) Android OpenStack 0102030 Reviews with code−reviewer assignment problem Reviews without code−revi Studied systems Reviewingtime(Days) Android OpenStack Qt 0102030 Review w/ Assignment Problem Review w/o Assignment Problem 18 Days Longer 13 Days Longer 9 Days Longer 6 Days Longer Code-Reviewer Assignment Problem Who should review my code? Code change Gerrit Code Review Invite Reviewers NewDeveloper 4 7 RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History Find reviewers who have reviewed the most similar file paths
- 72. 5 How do reviewswith code-reviewer assignment problem impact reviewing time? Studied Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% Reviews with code-reviewer assignment problem significantly take 12 days longer to approve a code change. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment proble Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviewing Time (Days) Reviews with code−reviewer assignment problem Studie Reviewingtime(Days) Android OpenStack 0102030 Reviews with code−reviewer assignment problem Reviews without code−revi Studied systems Reviewingtime(Days) Android OpenStack Qt 0102030 Review w/ Assignment Problem Review w/o Assignment Problem 18 Days Longer 13 Days Longer 9 Days Longer 6 Days Longer 7 RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History Find reviewers who have reviewed the most similar file paths Code-Reviewer Assignment Problem Who should review my code? Code change Gerrit Code Review Invite Reviewers NewDeveloper 4 Empirical Evaluation Accuracy “Does RevFinder accurately recommend reviewers?” Measure: Top-k Accuracy Ranking Performance “Does RevFinder provide a better ranking of correct reviewers?” 15 On average, RevFinder can correctly recommend reviewers for 79% of reviews with a top-10 recommendation. Measure: Mean Reciprocal Rank (MRR) On average, RevFinder can recommend reviewers with a median rank of 4. We can use a similarity of previously reviewed file paths to recommend code-reviewers for new code changes.
- 73. 5 How do reviewswith code-reviewer assignment problem impact reviewing time? Studied Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% Reviews with code-reviewer assignment problem significantly take 12 days longer to approve a code change. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment proble Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviewing Time (Days) Reviews with code−reviewer assignment problem Studie Reviewingtime(Days) Android OpenStack 0102030 Reviews with code−reviewer assignment problem Reviews without code−revi Studied systems Reviewingtime(Days) Android OpenStack Qt 0102030 Review w/ Assignment Problem Review w/o Assignment Problem 18 Days Longer 13 Days Longer 9 Days Longer 6 Days Longer 7 RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History Find reviewers who have reviewed the most similar file paths Code-Reviewer Assignment Problem Who should review my code? Code change Gerrit Code Review Invite Reviewers NewDeveloper 4 Empirical Evaluation Accuracy “Does RevFinder accurately recommend reviewers?” Measure: Top-k Accuracy Ranking Performance “Does RevFinder provide a better ranking of correct reviewers?” 15 On average, RevFinder can correctly recommend reviewers for 79% of reviews with a top-10 recommendation. Measure: Mean Reciprocal Rank (MRR) On average, RevFinder can recommend reviewers with a median rank of 4. We can use a similarity of previously reviewed file paths to recommend code-reviewers for new code changes.
- 74. 5 How do reviewswith code-reviewer assignment problem impact reviewing time? Studied Projects Sample Reviews 357 363 378 363 #Reviews w/ Assignment Problem 10% 5% 30% 4% Reviews with code-reviewer assignment problem significantly take 12 days longer to approve a code change. Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment problem Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviews with code−reviewer assignment problem Reviews without code−reviewer assignment proble Studied systems Reviewingtime(Days) Android OpenStack Qt LibreOffice 0102030 Reviewing Time (Days) Reviews with code−reviewer assignment problem Studie Reviewingtime(Days) Android OpenStack 0102030 Reviews with code−reviewer assignment problem Reviews without code−revi Studied systems Reviewingtime(Days) Android OpenStack Qt 0102030 Review w/ Assignment Problem Review w/o Assignment Problem 18 Days Longer 13 Days Longer 9 Days Longer 6 Days Longer 7 RevFinder: Overview R1 R2 Changed Files - src/video/camera.java - src/video/player.java Code-Reviewers Reviewed by John Changed Files - src/devices/screen.java Code-Reviewers Reviewed by Pat R3 (New) Changed Files - src/video/recorder.java Code-Reviewers ? RevFinder Review History Find reviewers who have reviewed the most similar file paths Empirical Evaluation Accuracy “Does RevFinder accurately recommend reviewers?” Measure: Top-k Accuracy Ranking Performance “Does RevFinder provide a better ranking of correct reviewers?” 15 On average, RevFinder can correctly recommend reviewers for 79% of reviews with a top-10 recommendation. Measure: Mean Reciprocal Rank (MRR) On average, RevFinder can recommend reviewers with a median rank of 4. We can use a similarity of previously reviewed file paths to recommend code-reviewers for new code changes. Code-Reviewer Assignment Problem Who should review my code? Code change Gerrit Code Review Invite Reviewers NewDeveloper 4