SlideShare a Scribd company logo

Confusion in Code Reviews: Reasons, Impacts, and Coping Strategies

Code review is a software quality assurance practice widely employed in both open source and commercial software projects to detect defects, transfer knowledge and encourage adherence to coding standards. Notwithstanding, code reviews can also delay the incorporation of a code change into a code base, thus slowing down the overall development process. Part of this delay is often a consequence of reviewers not understanding, becoming confused by, or being uncertain about the intention, behavior, or effect of a code change. We investigate the reasons and impacts of confusion in code reviews, as well as the strategies developers adopt to cope with confusion. We employ a concurrent triangulation strategy to combine the analyses of survey responses and of the code review comments, and build a comprehensive confusion framework structured along the dimensions of the review process, the artifact being reviewed, the developers themselves and the relation between the developer and the artifact. The most frequent reasons for confusion are the missing rationale, discussion of non- functional requirements of the solution, and lack of familiarity with existing code. Developers report that confusion delays the merge decision, decreases review quality, and results in additional discussions. To cope with confusion developers request informa- tion, improve familiarity with existing code, and discuss off-line. Based on the results, we provide a series of implications for tool builders, as well as insights and suggestions for researchers. The results of our work offer empirical justification for the need to improve code review tools to support developers facing confusion.

1 of 35
Download to read offline
Confusion in
Code Reviews
Reasons, Impacts, and
Coping Strategies
Felipe Ebert1
Nicole Novielli2
1 Federal University of Pernambuco, Brazil
2 University of Bari, Italy
3 Eindhoven University of Technology, The Netherlands
Fernando Castor1
Alexander Serebrenik3
Code Review
3
Why Code Reviewing?!
4
Find Defects
Why Code Reviewing?!
5
Find Defects Code Improvement
Alternative Solutions
Why Code Reviewing?!
6
Knowledge Transfer
Find Defects Code Improvement
Alternative Solutions
Why Code Reviewing?!
7
Knowledge Transfer
Team Awareness
Find Defects Code Improvement
Alternative Solutions
Code Reviews Are Not Free!!!
8
9
What is confusion?!
a situation where a person is
uncertain about or unable
to understand something
10
The impacts of confusion!
11
The impacts of confusion!
12
The impacts of confusion!
13
14
Reasons
Impacts
Coping
strategies
Methodology
15
“what developers say” “what developers do”
Survey
• How often do you feel confused...?
• What usually makes you confused...?
• What is the impact of confusion…?
• What do you usually do to overcome
confusion…?
16
Code Review Comments[1]
• Extracted from Android code reviews
• Including expressions of confusion
• 156 general comments
• 151 inline comments
• Discussed and agreed upon by the
authors
17
[1] F. Ebert, F. Castor, N. Novielli, and A.Serebrenik. Confusion Detection in Code
Reviews. In Proceedings of ICSME 2017, pp. 549–553.
Card sorting! Card sorting! Card sorting!
18
What Developers Say?
• 1st survey:
– Android developers
– 17 responses
– Response rate: 0.45%
• 2nd survey:
– Facebook & Twitter
– 24 responses
• 3rd survey:
– Facebook & Twitter
– 13 responses
19
What Developers Say?
• 1st survey:
– Android developers
– 17 responses
– Response rate: 0.45%
• 2nd survey:
– Facebook & Twitter
– 24 responses
• 3rd survey:
– Facebook & Twitter
– 13 responses
20
What Developers Say?
• 1st survey:
– Android developers
– 17 responses
– Response rate: 0.45%
• 2nd survey:
– Facebook & Twitter
– 24 responses
• 3rd survey:
– Facebook & Twitter
– 13 responses
21
We treat survey respondents
as a single population
• 2nd and 3rd
• ANOSIM R = −0.0171 and p-value = 0.542
• ADONIS p-value = 0.975
• 1st and the later two
• ANOSIM R = 0.126 and p-value = 0.01
• ADONIS p-value = 0.191
22
What Developers Do?
• 156 General Comments
• 151 Inline Comments
23
24
30 reasons
14 impacts
13 coping strategies
25
Frequency of Confusion
26
41%
27
28
29
30
31
32
33
34
Concluding Remarks
• Some topics have not been studied yet! 35
• Confusion is present!
– “Developers said!” (survey)
– “Developers did!” (code review comments)
• Confusion in context:
– 30 reasons
– 14 impacts
– 13 coping strategies

Recommended

Game Design and Procution: Introduction to Studies
Game Design and Procution: Introduction to StudiesGame Design and Procution: Introduction to Studies
Game Design and Procution: Introduction to StudiesPetri Lankoski
 
Adriana Moscatelli - Robot Games for Girls
Adriana Moscatelli - Robot Games for GirlsAdriana Moscatelli - Robot Games for Girls
Adriana Moscatelli - Robot Games for GirlsSeriousGamesAssoc
 
A Gamified Tutorial for Learning about Security Requirements Engineering
A Gamified Tutorial for Learning about Security Requirements EngineeringA Gamified Tutorial for Learning about Security Requirements Engineering
A Gamified Tutorial for Learning about Security Requirements EngineeringFabiano Dalpiaz
 
How not to run code reviews
How not to run code reviewsHow not to run code reviews
How not to run code reviewsVictor Maliy
 
Survey knowledge
Survey knowledgeSurvey knowledge
Survey knowledgeTu Tran
 
Surveys that work: using questionnaires to gather useful data, November 2010
Surveys that work: using questionnaires to gather useful data, November 2010Surveys that work: using questionnaires to gather useful data, November 2010
Surveys that work: using questionnaires to gather useful data, November 2010Caroline Jarrett
 

More Related Content

Similar to Confusion in Code Reviews: Reasons, Impacts, and Coping Strategies

Technovation challenge work plan for week 3
Technovation challenge work plan for week 3Technovation challenge work plan for week 3
Technovation challenge work plan for week 3wetech_global
 
Usability reports for prototype jam Nov. 30 2015 presentation
Usability reports for prototype jam Nov. 30 2015 presentationUsability reports for prototype jam Nov. 30 2015 presentation
Usability reports for prototype jam Nov. 30 2015 presentationMissouri Western State University
 
Education & Game Principles: Context, Theory & Application
Education & Game Principles: Context, Theory & ApplicationEducation & Game Principles: Context, Theory & Application
Education & Game Principles: Context, Theory & ApplicationLa Trobe University
 
WEBINAR: How to Coach Problem Solvers to Build Their Soft Skills
WEBINAR: How to Coach Problem Solvers to Build Their Soft SkillsWEBINAR: How to Coach Problem Solvers to Build Their Soft Skills
WEBINAR: How to Coach Problem Solvers to Build Their Soft SkillsGoLeanSixSigma.com
 
Higher Education & Game Principles: Context, Theory & Application - Daniel La...
Higher Education & Game Principles: Context, Theory & Application - Daniel La...Higher Education & Game Principles: Context, Theory & Application - Daniel La...
Higher Education & Game Principles: Context, Theory & Application - Daniel La...Blackboard APAC
 
Big Challenges in Data Modeling - Data Modelers and Project Managers
Big Challenges in Data Modeling - Data Modelers and Project ManagersBig Challenges in Data Modeling - Data Modelers and Project Managers
Big Challenges in Data Modeling - Data Modelers and Project ManagersDATAVERSITY
 
A Successful DevOps Initiative Starts with Knowning Your Numbers by Anne Hungate
A Successful DevOps Initiative Starts with Knowning Your Numbers by Anne HungateA Successful DevOps Initiative Starts with Knowning Your Numbers by Anne Hungate
A Successful DevOps Initiative Starts with Knowning Your Numbers by Anne HungateQA or the Highway
 
How to Effectively Lead Focus Groups: Presented at Product School NYC
How to Effectively Lead Focus Groups: Presented at Product School NYC How to Effectively Lead Focus Groups: Presented at Product School NYC
How to Effectively Lead Focus Groups: Presented at Product School NYC Tremis Skeete
 
Caroline Jarrett: Forms and their Users
Caroline Jarrett: Forms and their UsersCaroline Jarrett: Forms and their Users
Caroline Jarrett: Forms and their UsersFrancis Rowland
 
On to code review lessons learned at microsoft
On to code review lessons learned at microsoftOn to code review lessons learned at microsoft
On to code review lessons learned at microsoftMichaela Greiler
 
Communication skill
Communication skillCommunication skill
Communication skillArise Roby
 
Ep interview flow training
Ep interview flow trainingEp interview flow training
Ep interview flow trainingAIESECGreece
 
Personal branding CVvdT_en_2014/12/8
Personal branding CVvdT_en_2014/12/8Personal branding CVvdT_en_2014/12/8
Personal branding CVvdT_en_2014/12/8Tom Scholte
 
personal branding cvvdt_en_2014
personal branding cvvdt_en_2014personal branding cvvdt_en_2014
personal branding cvvdt_en_2014Tom Scholte
 
Designing forms for technical specialists by @cjforms
Designing forms for technical specialists by @cjformsDesigning forms for technical specialists by @cjforms
Designing forms for technical specialists by @cjformsCaroline Jarrett
 
Using Developer Conversations to Resolve Uncertainty in Software Development:...
Using Developer Conversations to Resolve Uncertainty in Software Development:...Using Developer Conversations to Resolve Uncertainty in Software Development:...
Using Developer Conversations to Resolve Uncertainty in Software Development:...Michalis Famelis
 
If Universities Won't Help Us, How do we #MakeATester?
If Universities Won't Help Us, How do we #MakeATester?If Universities Won't Help Us, How do we #MakeATester?
If Universities Won't Help Us, How do we #MakeATester?Simon Prior
 
Manual QA Testing Interview Questions From H2KInfosys
Manual QA Testing Interview Questions From H2KInfosysManual QA Testing Interview Questions From H2KInfosys
Manual QA Testing Interview Questions From H2KInfosysH2kInfosys
 
How I Hire Developers
How I Hire DevelopersHow I Hire Developers
How I Hire DevelopersTristan Gomez
 

Similar to Confusion in Code Reviews: Reasons, Impacts, and Coping Strategies (20)

Technovation challenge work plan for week 3
Technovation challenge work plan for week 3Technovation challenge work plan for week 3
Technovation challenge work plan for week 3
 
Usability reports for prototype jam Nov. 30 2015 presentation
Usability reports for prototype jam Nov. 30 2015 presentationUsability reports for prototype jam Nov. 30 2015 presentation
Usability reports for prototype jam Nov. 30 2015 presentation
 
Education & Game Principles: Context, Theory & Application
Education & Game Principles: Context, Theory & ApplicationEducation & Game Principles: Context, Theory & Application
Education & Game Principles: Context, Theory & Application
 
WEBINAR: How to Coach Problem Solvers to Build Their Soft Skills
WEBINAR: How to Coach Problem Solvers to Build Their Soft SkillsWEBINAR: How to Coach Problem Solvers to Build Their Soft Skills
WEBINAR: How to Coach Problem Solvers to Build Their Soft Skills
 
Higher Education & Game Principles: Context, Theory & Application - Daniel La...
Higher Education & Game Principles: Context, Theory & Application - Daniel La...Higher Education & Game Principles: Context, Theory & Application - Daniel La...
Higher Education & Game Principles: Context, Theory & Application - Daniel La...
 
Big Challenges in Data Modeling - Data Modelers and Project Managers
Big Challenges in Data Modeling - Data Modelers and Project ManagersBig Challenges in Data Modeling - Data Modelers and Project Managers
Big Challenges in Data Modeling - Data Modelers and Project Managers
 
A Successful DevOps Initiative Starts with Knowning Your Numbers by Anne Hungate
A Successful DevOps Initiative Starts with Knowning Your Numbers by Anne HungateA Successful DevOps Initiative Starts with Knowning Your Numbers by Anne Hungate
A Successful DevOps Initiative Starts with Knowning Your Numbers by Anne Hungate
 
How to Effectively Lead Focus Groups: Presented at Product School NYC
How to Effectively Lead Focus Groups: Presented at Product School NYC How to Effectively Lead Focus Groups: Presented at Product School NYC
How to Effectively Lead Focus Groups: Presented at Product School NYC
 
Building Debt Free MVP
Building Debt Free MVPBuilding Debt Free MVP
Building Debt Free MVP
 
Caroline Jarrett: Forms and their Users
Caroline Jarrett: Forms and their UsersCaroline Jarrett: Forms and their Users
Caroline Jarrett: Forms and their Users
 
On to code review lessons learned at microsoft
On to code review lessons learned at microsoftOn to code review lessons learned at microsoft
On to code review lessons learned at microsoft
 
Communication skill
Communication skillCommunication skill
Communication skill
 
Ep interview flow training
Ep interview flow trainingEp interview flow training
Ep interview flow training
 
Personal branding CVvdT_en_2014/12/8
Personal branding CVvdT_en_2014/12/8Personal branding CVvdT_en_2014/12/8
Personal branding CVvdT_en_2014/12/8
 
personal branding cvvdt_en_2014
personal branding cvvdt_en_2014personal branding cvvdt_en_2014
personal branding cvvdt_en_2014
 
Designing forms for technical specialists by @cjforms
Designing forms for technical specialists by @cjformsDesigning forms for technical specialists by @cjforms
Designing forms for technical specialists by @cjforms
 
Using Developer Conversations to Resolve Uncertainty in Software Development:...
Using Developer Conversations to Resolve Uncertainty in Software Development:...Using Developer Conversations to Resolve Uncertainty in Software Development:...
Using Developer Conversations to Resolve Uncertainty in Software Development:...
 
If Universities Won't Help Us, How do we #MakeATester?
If Universities Won't Help Us, How do we #MakeATester?If Universities Won't Help Us, How do we #MakeATester?
If Universities Won't Help Us, How do we #MakeATester?
 
Manual QA Testing Interview Questions From H2KInfosys
Manual QA Testing Interview Questions From H2KInfosysManual QA Testing Interview Questions From H2KInfosys
Manual QA Testing Interview Questions From H2KInfosys
 
How I Hire Developers
How I Hire DevelopersHow I Hire Developers
How I Hire Developers
 

Recently uploaded

Passbolt Introduction and Usage for secret managment
Passbolt Introduction and Usage for secret managmentPassbolt Introduction and Usage for secret managment
Passbolt Introduction and Usage for secret managmentThierry Gayet
 
Instrument Data Automation: The Life of a Flow
Instrument Data Automation: The Life of a FlowInstrument Data Automation: The Life of a Flow
Instrument Data Automation: The Life of a FlowGlobus
 
Cybersecurity Measures For Remote Workers.pdf
Cybersecurity Measures For Remote Workers.pdfCybersecurity Measures For Remote Workers.pdf
Cybersecurity Measures For Remote Workers.pdfCIOWomenMagazine
 
Advanced Globus System Administration Topics
Advanced Globus System Administration TopicsAdvanced Globus System Administration Topics
Advanced Globus System Administration TopicsGlobus
 
Building Research Applications with Globus PaaS
Building Research Applications with Globus PaaSBuilding Research Applications with Globus PaaS
Building Research Applications with Globus PaaSGlobus
 
Agile & Scrum, Certified Scrum Master! Crash Course
Agile & Scrum,  Certified Scrum Master! Crash CourseAgile & Scrum,  Certified Scrum Master! Crash Course
Agile & Scrum, Certified Scrum Master! Crash CourseRohan Chandane
 
انتزاع و هزینه - انتزاع و تاثیرات آن در توسعه و نگهداری نرم‌افزار
انتزاع و هزینه - انتزاع و تاثیرات آن در توسعه و نگهداری نرم‌افزارانتزاع و هزینه - انتزاع و تاثیرات آن در توسعه و نگهداری نرم‌افزار
انتزاع و هزینه - انتزاع و تاثیرات آن در توسعه و نگهداری نرم‌افزارsohilww
 
Globus for System Administrators
Globus for System AdministratorsGlobus for System Administrators
Globus for System AdministratorsGlobus
 
An Introduction to Globus for Researchers
An Introduction to Globus for ResearchersAn Introduction to Globus for Researchers
An Introduction to Globus for ResearchersGlobus
 
Workshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con FlinkWorkshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con Flinkconfluent
 
Orion Context Broker introduction 20240227
Orion Context Broker introduction 20240227Orion Context Broker introduction 20240227
Orion Context Broker introduction 20240227Fermin Galan
 
Open Source vs Closed Source LLMs. Pros and Cons
Open Source vs Closed Source LLMs. Pros and ConsOpen Source vs Closed Source LLMs. Pros and Cons
Open Source vs Closed Source LLMs. Pros and ConsSprings
 
Machine Learning Basics for Dummies (no math!)
Machine Learning Basics for Dummies (no math!)Machine Learning Basics for Dummies (no math!)
Machine Learning Basics for Dummies (no math!)Dmitry Zinoviev
 
CSS Notes in PDF, Easy to understand. For beginner to advanced. ...
CSS Notes in PDF, Easy to understand. For beginner to advanced.              ...CSS Notes in PDF, Easy to understand. For beginner to advanced.              ...
CSS Notes in PDF, Easy to understand. For beginner to advanced. ...syedfaisal759877
 
Introduction to Research Automation with Globus
Introduction to Research Automation with GlobusIntroduction to Research Automation with Globus
Introduction to Research Automation with GlobusGlobus
 
Role of DevOps in SaaS product Development.pdf.pptx
Role of DevOps in SaaS product Development.pdf.pptxRole of DevOps in SaaS product Development.pdf.pptx
Role of DevOps in SaaS product Development.pdf.pptxMindInventory
 
Best Practices for Data Sharing Using Globus
Best Practices for Data Sharing Using GlobusBest Practices for Data Sharing Using Globus
Best Practices for Data Sharing Using GlobusGlobus
 
LLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flowLLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flowNaoki (Neo) SATO
 
Joseph Yoder : Being Agile about Architecture
Joseph Yoder : Being Agile about ArchitectureJoseph Yoder : Being Agile about Architecture
Joseph Yoder : Being Agile about ArchitectureHironori Washizaki
 

Recently uploaded (20)

Passbolt Introduction and Usage for secret managment
Passbolt Introduction and Usage for secret managmentPassbolt Introduction and Usage for secret managment
Passbolt Introduction and Usage for secret managment
 
Instrument Data Automation: The Life of a Flow
Instrument Data Automation: The Life of a FlowInstrument Data Automation: The Life of a Flow
Instrument Data Automation: The Life of a Flow
 
Cybersecurity Measures For Remote Workers.pdf
Cybersecurity Measures For Remote Workers.pdfCybersecurity Measures For Remote Workers.pdf
Cybersecurity Measures For Remote Workers.pdf
 
Advanced Globus System Administration Topics
Advanced Globus System Administration TopicsAdvanced Globus System Administration Topics
Advanced Globus System Administration Topics
 
2024 Trends Transforming Enterprise Resource Planning
2024 Trends Transforming Enterprise Resource Planning2024 Trends Transforming Enterprise Resource Planning
2024 Trends Transforming Enterprise Resource Planning
 
Building Research Applications with Globus PaaS
Building Research Applications with Globus PaaSBuilding Research Applications with Globus PaaS
Building Research Applications with Globus PaaS
 
Agile & Scrum, Certified Scrum Master! Crash Course
Agile & Scrum,  Certified Scrum Master! Crash CourseAgile & Scrum,  Certified Scrum Master! Crash Course
Agile & Scrum, Certified Scrum Master! Crash Course
 
انتزاع و هزینه - انتزاع و تاثیرات آن در توسعه و نگهداری نرم‌افزار
انتزاع و هزینه - انتزاع و تاثیرات آن در توسعه و نگهداری نرم‌افزارانتزاع و هزینه - انتزاع و تاثیرات آن در توسعه و نگهداری نرم‌افزار
انتزاع و هزینه - انتزاع و تاثیرات آن در توسعه و نگهداری نرم‌افزار
 
Globus for System Administrators
Globus for System AdministratorsGlobus for System Administrators
Globus for System Administrators
 
An Introduction to Globus for Researchers
An Introduction to Globus for ResearchersAn Introduction to Globus for Researchers
An Introduction to Globus for Researchers
 
Workshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con FlinkWorkshop híbrido: Stream Processing con Flink
Workshop híbrido: Stream Processing con Flink
 
Orion Context Broker introduction 20240227
Orion Context Broker introduction 20240227Orion Context Broker introduction 20240227
Orion Context Broker introduction 20240227
 
Open Source vs Closed Source LLMs. Pros and Cons
Open Source vs Closed Source LLMs. Pros and ConsOpen Source vs Closed Source LLMs. Pros and Cons
Open Source vs Closed Source LLMs. Pros and Cons
 
Machine Learning Basics for Dummies (no math!)
Machine Learning Basics for Dummies (no math!)Machine Learning Basics for Dummies (no math!)
Machine Learning Basics for Dummies (no math!)
 
CSS Notes in PDF, Easy to understand. For beginner to advanced. ...
CSS Notes in PDF, Easy to understand. For beginner to advanced.              ...CSS Notes in PDF, Easy to understand. For beginner to advanced.              ...
CSS Notes in PDF, Easy to understand. For beginner to advanced. ...
 
Introduction to Research Automation with Globus
Introduction to Research Automation with GlobusIntroduction to Research Automation with Globus
Introduction to Research Automation with Globus
 
Role of DevOps in SaaS product Development.pdf.pptx
Role of DevOps in SaaS product Development.pdf.pptxRole of DevOps in SaaS product Development.pdf.pptx
Role of DevOps in SaaS product Development.pdf.pptx
 
Best Practices for Data Sharing Using Globus
Best Practices for Data Sharing Using GlobusBest Practices for Data Sharing Using Globus
Best Practices for Data Sharing Using Globus
 
LLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flowLLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flow
 
Joseph Yoder : Being Agile about Architecture
Joseph Yoder : Being Agile about ArchitectureJoseph Yoder : Being Agile about Architecture
Joseph Yoder : Being Agile about Architecture
 

Confusion in Code Reviews: Reasons, Impacts, and Coping Strategies