In this webinar session, we share a comprehensive list of peer code review best practices, distilled down years of SmartBear research and case studies. At the end, we shared how our code and document review tool, Collaborator, can help teams put these tactics into practice.

1. The Best Kept Secrets
of Code Review

2. Jonathan FortunatiPatrick Londa
Your Hosts Today
Justin Collier
Marketing Manager,
Collaborator
Product Owner, Collaborator Sr. Sales Engineer, Collaborator

3. 2009
6.5M+
users
11
22,000+
5 525+
customers
employees
founded global offices
free tools

4. SmartBear Product Portfolio = Breadth and Depth
4
Design
HipTest
Create Test Monitor
SwaggerHub
LoadUI Pro
LoadNinja
CBT
ServiceV Pro SoapUI Pro
TestComplete
AlertSite
TestLeftZephyr
Collaborator

5. Agenda
Review Process Recommendations
Code Review Approaches
Why Focus on Code Reviews?
Inside the Code Review
The Current Tool Landscape
1
2
3
4
5
5

6. Main Sources
Proprietary & Confidential
6

7. Why Focus on Code Review?

8. The 2018 State of Code Review

9. The 2018 State of Code Review

10. 10
Code Review is the #1 Way to Improve Code Quality
According to: The 2018 State of Code Review
The 2017 State of Code Review
The 2016 State of Code Review

11. The 2018 State of Code Review

12. 10,000-line project with 10 developers
Bugs Remaining = 463
Bugs Remaining = 321
Bugs Remaining = 194
0
50
100
150
200
250
300
350
400
450
500
After Development After QA/Test ($200/fix) After Customer ($1000/fix)
BugsRemainingintheApplication
BEFORE CODE REVIEW
Cost of Fixing Bugs: $174k
+ Cost of 194 Latent Bugs: 194k
Total Cost: $368k
SmartBear Customer Case Study

13. Bugs Remaining = 463
Bugs Remaining = 180
Bugs Remaining = 113
Bugs Remaining = 32
0
50
100
150
200
250
300
350
400
450
500
After Development After Code Review After QA/Test ($200/fix) After Customer ($1000/fix)
BugsRemainingintheApplication
AFTER CODE REVIEW
10,000-line project with 10 developers
Cost of Fixing Bugs: $120k
+ Cost of 32 Latent Bugs: 32k
Total Cost: $152k
Resulting Savings = $216k
SmartBear Customer Case Study

14. Code Review Approaches

15. • Formal Inspections
Types of Code Review Planning
Introductory Meeting
Inspection Meeting
Rework
Verification Meeting
Complete
Follow-Up Meeting
Readers and reviewers
inspect the code privately.
If no defects are found,
review is complete.
If more defects are
found, back to rework.

16. • Formal Inspections
• Over-the-Shoulder Reviews
Types of Code Review
Preparation
Inspection Meeting
Rework
Complete

17. • Formal Inspections
• Over-the-Shoulder Reviews
• Email Pass-Around Process
Types of Code Review
Code Check-In
Inspections
Rework
Complete
If no problems, review
fades into “Complete”.

18. • Formal Inspections
• Over-the-Shoulder Reviews
• Email Pass-Around Process
• Tool-Assisted Reviews
Types of Code Review

19. • Formal Inspections
• Over-the-Shoulder Reviews
• Email Pass-Around Process
• Tool-Assisted Reviews
• Pair Programming
Types of Code Review

20. Review Process
Recommendations

21. • Conduct Reviews Daily
What Works Best:
The 2018 State of Code Review

22. • Conduct Reviews Daily
• Set Clear Expectations
The 2018 State of Code Review
What Works Best:

23. • Conduct Reviews Daily
• Set Clear Expectations
*Utilize a checklist of things to
check for at the class, method,
& class-hierarchy levels
Research by Alastair Dunsmoore, Marc Roper, and Murray Wood. (2000)
The Best Kept Secrets of Peer Code Review
What Works Best:

24. • Conduct Reviews Daily
• Set Clear Expectations
• Go Tool-Assisted
The 2018 State of Code Review
What Works Best:

25. • Conduct Reviews Daily
• Set Clear Expectations
• Go Tool-Assisted + Meetings
We never perform meeting-based reviews...
28% Satisfied with Code Quality
48% Unsatisfied with Code Quality
The 2018 State of Code Review
What Works Best:

26. • Conduct Reviews Daily
• Set Clear Expectations
• Go Tool-Assisted + Meetings
Research by Lawrence Votta from AT&T Bell Laboratories. (1993)
Supporting research by Diane Kelly and Terry Shepard. (2003)
Supporting research by Reidar Conradi. (2003)
The Best Kept Secrets of Peer Code Review
What Works Best:

27. • Conduct Reviews Daily
• Set Clear Expectations
• Go Tool-Assisted + Meetings
• Report on Their Process
The 2018 State of Code Review
What Works Best:

28. • Conduct Reviews Daily
• Set Clear Expectations
• Go Tool-Assisted + Meetings
• Report on Their Process
• Foster a Learning Culture
What Works Best:

29. Inside the Code Review

30. Review Guidelines
• Limit reviews to 60 minute
sessions at most
30
Research by Alastair Dunsmoore, Marc Roper, and Murray Wood. (2000)
The Best Kept Secrets of Peer Code Review

31. Review Guidelines
• Limit reviews to 60 minute
sessions at most
• Consider shorter function
length to improve readability
3131
Research by Uwano at Nara Institute of Science and Technology. (2006)
The Best Kept Secrets of Peer Code Review

32. Review Guidelines
• Limit reviews to 60 minute
sessions at most
• Consider limiting function
length to improve readability
3232
Research by Uwano at Nara Institute of Science and Technology. (2006)
The Best Kept Secrets of Peer Code Review

33. Review Guidelines
• Limit reviews to 60 minute
sessions at most
• Consider shorter function
length to improve readability
3333
Research by Uwano at Nara Institute of Science and Technology. (2006)
The Best Kept Secrets of Peer Code Review

34. Review Guidelines
• Limit reviews to 60 minute
sessions at most
• Consider shorter function
length to improve readability
• Slow down your reading and
take time to scan
3434
Research by Uwano at Nara Institute of Science and Technology. (2006)
The Best Kept Secrets of Peer Code Review

35. Ideal Review
• Author annotates their code
prior to reviewers reading
35
Cisco Case Study with SmartBear. (2006)
The Best Kept Secrets of Peer Code Review

36. Ideal Review
• Author annotates their code
prior to reviewers reading
• 100 – 300 lines of code
36
Cisco Case Study with SmartBear. (2006)
The Best Kept Secrets of Peer Code Review

37. Ideal Review
• Author annotates their code
prior to reviewers reading
• 100 – 300 lines of code
• Reviewers spend 30-60 minutes
reviewing the code change
37
Cisco Case Study with SmartBear. (2006)
The Best Kept Secrets of Peer Code Review

38. Goals & Reading Metrics
• Internal Metrics
• Inspection Rate
• Defect Rate
• Defect Density
• Defect Count
• Defect Type and Severity
• External Metrics
• Number of support calls
• Number of bugs injected by Development
Proprietary & Confidential
38

39. Reviewing Artifacts
39
The 2018 State of Code Review

40. Reviewing Artifacts
90% of teams are conducting document review of some kind.
35% of respondents are using a tool to do this.
Teams using a tool for reviewing documents are
26% more likely
to be satisfied with their code review process.
40
The 2018 State of Code Review

41. Review Best Practices
• Conduct reviews daily
• Set clear expectations with checklists
• Conduct tool-assisted reviews + meetings
• Create goals and report on your process
• Foster a learning culture
• Limit reviews to 60 minute sessions
• Shorten function length for readability
• Slow down your reading and take time to scan
• Author annotates code prior to adding reviewers
• 100 – 300 lines of code at a time
• Reviewers should spend 30-60 minutes at a time
• Verify that defects have been addressed
• Conduct peer reviews on other docs & artifacts
• Maintain review history for long-term success
41

42. Current Review Tool Landscape

43. The 2018 State of Code Review

44. Review Best Practices
• Conduct reviews daily
• Set clear expectations with checklists
• Conduct tool-assisted reviews + meetings
• Create goals and report on your process
• Foster a learning culture
• Limit reviews to 60 minute sessions
• Shorten function length for readability
• Slow down your reading and take time to scan
• Author annotates code prior to adding reviewers
• 100 – 300 lines of code at a time
• Reviewers should spend 30-60 minutes at a time
• Verify that defects have been addressed
• Conduct peer reviews on other docs & artifacts
• Maintain review history for long-term success
44

45. Review Best Practices
45
• Conduct reviews daily
• Set clear expectations with checklists
• Conduct tool-assisted reviews + meetings
• Create goals and report on your process
• Foster a learning culture
• Limit reviews to 60 minute sessions
• Shorten function length for readability
• Slow down your reading and take time to scan
• Author annotates code prior to adding reviewers
• 100 – 300 lines of code at a time
• Reviewers should spend 30-60 minutes at a time
• Verify that defects have been addressed
• Conduct peer reviews on other docs & artifacts
• Maintain review history for long-term success

46. The 2018 State of Code Review

47. Review Best Practices
• Conduct reviews daily
• Set clear expectations with checklists
• Conduct tool-assisted reviews + meetings
• Create goals and report on your process
• Foster a learning culture
• Limit reviews to 60 minute sessions
• Shorten function length for readability
• Slow down your reading and take time to scan
• Author annotates code prior to adding reviewers
• 100 – 300 lines of code at a time
• Reviewers should spend 30-60 minutes at a time
• Verify that defects have been addressed
• Conduct peer reviews on other docs & artifacts
• Maintain review history for long-term success
47

48. Smart Review Practices
• Set clear expectations with checklists
• Create goals and report on your process
• Author annotates code prior to adding reviewers
• Verify that defects have been addressed
• Conduct peer reviews on other docs & artifacts
• Maintain review history for long-term success
48
Teams that report on their code
reviews are 5X as likely to be
satisfied with their process
Teams that review documents in
tool are 26% more likely to be
satisfied with their code reviews.

49. Review Best Practices
49
• Conduct reviews daily
• Set clear expectations with checklists
• Conduct tool-assisted reviews + meetings
• Create goals and report on your process
• Foster a learning culture
• Limit reviews to 60 minute sessions
• Shorten function length for readability
• Slow down your reading and take time to scan
• Author annotates code prior to adding reviewers
• 100 – 300 lines of code at a time
• Reviewers should spend 30-60 minutes at a time
• Verify that defects have been addressed
• Conduct peer reviews on other docs & artifacts
• Maintain review history for long-term success

50. Reviews in Collaborator

51. Thank you!