The Role of Patch Review in Software Evolution: An Analysis of the Mozilla Firefox
•Download as PPTX, PDF•
2 likes•878 views
Patch review is the basic mechanism for validating the design and implementation of patches and maintaining consistency in some commercial and Free/Libre/Open Source Software (FLOSS) projects. We examine the inner-workings of the development process of the successful and mature Mozilla foundation and highlight how different parties involved affect and steer the process. Although reviewers are the primary actors in the patch review process, success in the process can only be achieved if the community supports reviewers adequately. Peer developers play the supporting role by offering insight and ideas that help create more quality patches. Moreover, they reduce the huge patch backlog reviewers have to clear by identifying and eliminating immature patches.
Recommended
More Related Content
Viewers also liked
Viewers also liked (13)
Recently uploaded
Recently uploaded (20)
The Role of Patch Review in Software Evolution: An Analysis of the Mozilla Firefox
- 1. The Role of Patch Review in Software Evolution: An Analysis of the Mozilla Firefox IWPSE-EVOL 2009, Amsterdam, The Netherlands Mehrdad Nurolahzade SeyedMehdiNasehi Shahedul Huq Khandkar ShreyaRawal
- 2. Patch Review The process of incrementally submitting and integrating patches (fix for a bug report) into a software system is a core activity related to software evolution. It act as a basic mechanism for validating the design and implementation of patches. It happens in both, commercial and FLOSS projects. 2
- 3. Research Questions What are the roles of various people involved in the patch review process? What is the process of conducting reviews? When are reviews performed? What do reviewers look at and what do they miss? 3
- 5. Software Evolution in Mozilla 5
- 6. Mozilla Development Process review?, super-review?, ui-review? negative review backed out by QA review+, super-review+, ui-review+ 6
- 7. Overview of Research Approach 7 Online Mozilla Bugzilla Documentation Bug Reports Codes: Review Negative Review Rubber Stamp Merciful reviewer Open Coding Process Themes Tool for Qualitative Analysis Observations and Patterns Database Quantification
- 8. Data Collection 8 History (Status Changes) Attachments (Patches) Discussion and Comments Bug Report
- 9. Tool for Qualitative Analysis 9
- 10. Data Set 10 Bug Reports (112) (2002-2009) 66 Bugs 38 Enhancements 8 New Features Patches (310) Comments (1842) 318 Reviews
- 11. Findings Patterns Patchy Patcher Merciful Reviewer Doubtful reviewer Observations Module Owner Reviews Peer Reviews Types of Feedback Undiscovered Errors 11
- 15. 42 review-
- 16. Where did the 14 reviews go?After the review comments the reviewer changed the status from ‘review?’ to nothing. Hypothesis: Module Owners refrain from giving review- to Newcomers 13
- 17. Pattern: Doubtful Reviewer There has not been enough early feedback from community. A change to functionality is being made. “Let's put this in for beta, and make sure we blog about the change a little and see the reaction” (Bug # 412862) 14
- 18. Observation: Module Owner Reviews Every completed patch has to be reviewed by Module Owners unless somebody finds a flaw in it. 38 Module Owners 198 Code or UI inspections Module owners do not consider patches based on bug report priority (but developers do). 15
- 19. Observation: Peer Reviews 66 Peer Developers Most patches (76%) do not get any Peer Review 80% of the comments by peers are negative or partly negative On an average Peer conducts review before Module Owners 16
- 20. Observation: Types of Reviewer Feedback 17 Hypothesis: While peers are interested in functionality and usability, Module Owners are also interested in long-term maintainability of the product.
- 21. Observation: Undiscovered Errors 8.4% of checked-in patches were backed out. Performance and regression issues that remain unidentified during the review. It would be interesting to look if there a common pattern in occurrence of undiscovered errors? "Based on the site breakage, re-opening and suggesting we back out this change.“ (Bug # 412862) 18
- 22. Conclusion Peers play a key role in the process by providing ideas before a patch is developed and reviewing developed patches before module owners do. 19
- 23. Conclusion Module owners are more inclined toward the quality and consistency of developed patches. 20
- 24. Take Away Module owners and peer developers are complementing each other in reviewing patches. They refine the developed solution based on their own interests and concerns. 21
- 25. Debate Questions How quantitative/qualitative analysis of software engineering data can be facilitated? How empirical findings can be cross-validated with the FLOSS development communities? 22
Editor's Notes
- is a key activity in software evolution
- Conceptual model of patch evolution MozillaThe process is shaped around problem statement, i.e. bug reports.Bug Reports are stored in the Bugzilla server, they can be either a BR, ER, or a New Feature RequestThe solution of the problem is developed in the form of a patch, which is a piece of s/w.The three parties involved are:Developer: In case of Mozilla the developer is the person to whom the Bug Report is assigned.Module Owner: Who act as gatekeeper in Mozilla, they review the developed patch and if the fix meets the requirements then the MO approves the fix and the patch becomes the part of main code repository.Peers: Peers can be co-developers or any Mozilla Firefox User, they help in refining the development of patch and also make the review process more effective by giving their own comments about the fixThese three parties involved work closely to Define the problem by discussing the problem space and possible solutions. They also work together in refining the solution by discussing the possible alternatives and finally resolve the problem by checking the patch into the min code repository.
- This slide represents abstract workflow in Bugzilla.It is the life cycle of a single bug report in Bugzilla RepositoryThe development process starts with filing of the bug report by the developer or the user. The bug is filed with the description, priority and dependency i.e. if the filed bug is related to some previously filed bug reports.The filed bug report attracts the community discussion the discussion can be about the design, problem space, implementation and assignment. The discussions may happen in entire lifetime of the bug report.Then the assigned developer develops a patch for the bug report, it is the responsibility of the developer to run the Unit Tests and if the bug report addressed some User Interface then Graphical/Interactive tests are also run. The patch must also pass through the automated review, which is available online on the Bugzilla website.After the developer has developed the patch the patch is forwarded for review with a status change as (review?, super-review?, ui-review?). The patch can be reviewed by MO as well as Peer Reviewers. If a Flow is found the patch is given a negative review and then the developer need to develop an new patch. In other case the patch is checked in into the Bugzilla repository where it can be backed out by the QA and again the same procedure is repeated.
- Our two main data sources were Bug Reports and Online Mozilla Documentation We extracted the BRs from Bugzilla and used our Tool for qualitative anlysis. We performed Open Coding on the bug reports. From the codes developed we defined our themes. We also stored our codes into the database and quantified them to come up with more Observations and Patterns.Online Mozilla Documentation and Our developed codes provided the insight of the actual review process.
- We developed our own tool to combine all three components chronologically so that we can perform the qualitative analysis on the Bug Reports.
- The do so because of the following two reasons:To get early feedback: They submit incomplete patches to get early feedback form the community about the design of the patch developed.To show involvement: They want the community to know that they are working on the Bug Report.“waiting for feedback… there are some problems I need assistance on”
- We came up with this Hypothesis that Module Owners refrain from giving review- to Newcomers so as to encourage the new developers.. We have not verified this hypothesis though.
- If a patch has not attracted enough community discussion and the Reviewer is uncertain about the proposed solution.This usually happens when there is a change to functionality is being made.The Reviewer accepts the patch expressing his uncertainty and waits for the communities feedback.
- It is the MOs responsibility to review each and every patch submitted for review.Somebody: Peer or Developer himselfThis means a good amount of work for the Module Owner
- So, peer developers are more out spoken when they see something wrong with the patch otherwise they prefer to remain silent.So, if there is a MO decision already being made for the patch Peers do not provide their comments and review about the patchIMP!!!What is the number of patches that did not get any peer review? or may be the percentage of total patches
- Primary reason of rejecting a patch by both MO and Peer Reviewers is Implementation and Design issues.But we see a contrast in opinion in comments related to Functionality and Usability. Peer reviewers look more inclined towards commenting about the Functionality and Usability than MOsOn the contrary in case of Documentation and Coding Standards MOs pay more attention as compared to Peer ReviewersThis could be a possible explanation for the variation in the numbers.
- Peers also reduce a great amount of work for MOsAlthough peers and MOs both conduct review, they are conducting review in different ways but their contrast in interest works in favor of delivering a better quality product.