The Role of Patch Review in Software Evolution: An Analysis of the Mozilla Firefox
Upcoming SlideShare
Loading in...5

The Role of Patch Review in Software Evolution: An Analysis of the Mozilla Firefox



Patch review is the basic mechanism for validating the design and implementation of patches and maintaining consistency in some commercial and Free/Libre/Open Source Software (FLOSS) projects. We ...

Patch review is the basic mechanism for validating the design and implementation of patches and maintaining consistency in some commercial and Free/Libre/Open Source Software (FLOSS) projects. We examine the inner-workings of the development process of the successful and mature Mozilla foundation and highlight how different parties involved affect and steer the process. Although reviewers are the primary actors in the patch review process, success in the process can only be achieved if the community supports reviewers adequately. Peer developers play the supporting role by offering insight and ideas that help create more quality patches. Moreover, they reduce the huge patch backlog reviewers have to clear by identifying and eliminating immature patches.



Total Views
Views on SlideShare
Embed Views



3 Embeds 10 6 3 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • is a key activity in software evolution
  • Conceptual model of patch evolution MozillaThe process is shaped around problem statement, i.e. bug reports.Bug Reports are stored in the Bugzilla server, they can be either a BR, ER, or a New Feature RequestThe solution of the problem is developed in the form of a patch, which is a piece of s/w.The three parties involved are:Developer: In case of Mozilla the developer is the person to whom the Bug Report is assigned.Module Owner: Who act as gatekeeper in Mozilla, they review the developed patch and if the fix meets the requirements then the MO approves the fix and the patch becomes the part of main code repository.Peers: Peers can be co-developers or any Mozilla Firefox User, they help in refining the development of patch and also make the review process more effective by giving their own comments about the fixThese three parties involved work closely to Define the problem by discussing the problem space and possible solutions. They also work together in refining the solution by discussing the possible alternatives and finally resolve the problem by checking the patch into the min code repository.
  • This slide represents abstract workflow in Bugzilla.It is the life cycle of a single bug report in Bugzilla RepositoryThe development process starts with filing of the bug report by the developer or the user. The bug is filed with the description, priority and dependency i.e. if the filed bug is related to some previously filed bug reports.The filed bug report attracts the community discussion the discussion can be about the design, problem space, implementation and assignment. The discussions may happen in entire lifetime of the bug report.Then the assigned developer develops a patch for the bug report, it is the responsibility of the developer to run the Unit Tests and if the bug report addressed some User Interface then Graphical/Interactive tests are also run. The patch must also pass through the automated review, which is available online on the Bugzilla website.After the developer has developed the patch the patch is forwarded for review with a status change as (review?, super-review?, ui-review?). The patch can be reviewed by MO as well as Peer Reviewers. If a Flow is found the patch is given a negative review and then the developer need to develop an new patch. In other case the patch is checked in into the Bugzilla repository where it can be backed out by the QA and again the same procedure is repeated.
  • Our two main data sources were Bug Reports and Online Mozilla Documentation We extracted the BRs from Bugzilla and used our Tool for qualitative anlysis. We performed Open Coding on the bug reports. From the codes developed we defined our themes. We also stored our codes into the database and quantified them to come up with more Observations and Patterns.Online Mozilla Documentation and Our developed codes provided the insight of the actual review process.
  • We developed our own tool to combine all three components chronologically so that we can perform the qualitative analysis on the Bug Reports.
  • The do so because of the following two reasons:To get early feedback: They submit incomplete patches to get early feedback form the community about the design of the patch developed.To show involvement: They want the community to know that they are working on the Bug Report.“waiting for feedback… there are some problems I need assistance on”
  • We came up with this Hypothesis that Module Owners refrain from giving review- to Newcomers so as to encourage the new developers.. We have not verified this hypothesis though.
  • If a patch has not attracted enough community discussion and the Reviewer is uncertain about the proposed solution.This usually happens when there is a change to functionality is being made.The Reviewer accepts the patch expressing his uncertainty and waits for the communities feedback.
  • It is the MOs responsibility to review each and every patch submitted for review.Somebody: Peer or Developer himselfThis means a good amount of work for the Module Owner
  • So, peer developers are more out spoken when they see something wrong with the patch otherwise they prefer to remain silent.So, if there is a MO decision already being made for the patch Peers do not provide their comments and review about the patchIMP!!!What is the number of patches that did not get any peer review? or may be the percentage of total patches
  • Primary reason of rejecting a patch by both MO and Peer Reviewers is Implementation and Design issues.But we see a contrast in opinion in comments related to Functionality and Usability. Peer reviewers look more inclined towards commenting about the Functionality and Usability than MOsOn the contrary in case of Documentation and Coding Standards MOs pay more attention as compared to Peer ReviewersThis could be a possible explanation for the variation in the numbers.
  • Peers also reduce a great amount of work for MOsAlthough peers and MOs both conduct review, they are conducting review in different ways but their contrast in interest works in favor of delivering a better quality product.

The Role of Patch Review in Software Evolution: An Analysis of the Mozilla Firefox The Role of Patch Review in Software Evolution: An Analysis of the Mozilla Firefox Presentation Transcript

  • The Role of Patch Review in Software Evolution: An Analysis of the Mozilla Firefox
    IWPSE-EVOL 2009, Amsterdam, The Netherlands
    Mehrdad Nurolahzade
    Shahedul Huq Khandkar
  • Patch Review
    The process of incrementally submitting and integrating patches (fix for a bug report) into a software system is a core activity related to software evolution.
    It act as a basic mechanism for validating the design and implementation of patches.
    It happens in both, commercial and FLOSS projects.
  • Research Questions
    What are the roles of various people involved in the patch review process?
    What is the process of conducting reviews?
    When are reviews performed?
    What do reviewers look at and what do they miss?
  • Mozilla Firefox
    One of the successful FLOSS projects
    • Second most popular web browser
    Has both open source and commercial characteristics
    • 37% of code is contributed by the community
    40 developers in Mozilla Corporation Firefox development team
    100 daily contributors (approx)
    1000 contributors (approx)
    20-30 million users (approx)
    100,000 lines of code (approx)
    Source: Mike Beltzner, Mozilla Corporation, Web 2.0 Expo 2007
  • Software Evolution in Mozilla
  • Mozilla Development Process
    review?, super-review?, ui-review?
    negative review
    backed out by QA
    review+, super-review+, ui-review+
  • Overview of Research Approach
    Online Mozilla Bugzilla Documentation
    Bug Reports
    Negative Review
    Rubber Stamp
    Merciful reviewer
    Open Coding
    Tool for Qualitative Analysis
    Observations and Patterns
  • Data Collection
    History (Status Changes)
    Attachments (Patches)
    Discussion and Comments
    Bug Report
  • Tool for Qualitative Analysis
  • Data Set
    Bug Reports (112) (2002-2009)
    66 Bugs
    38 Enhancements
    8 New Features
    Patches (310)
    Comments (1842)
    318 Reviews
  • Findings
    Patchy Patcher
    Merciful Reviewer
    Doubtful reviewer
    Module Owner Reviews
    Peer Reviews
    Types of Feedback
    Undiscovered Errors
  • Patterns: Patchy Patcher
    • Submits work-in-progress patches
    • To get early feedback
    “incomplete, still waiting for Feedback” (Bug # 343172)
    • To show involvement
    “New patch coming soon…” (Bug # 456214)
    “wip:0.1 works, but I need to fix …” (Bug # 390614)
  • Patterns: Merciful Reviewer
    When reviewers are reluctant to give a review minus (review-).
    • 56 negative reviews
    • 42 review-
    • Where did the 14 reviews go?
    After the review comments the reviewer changed the status from ‘review?’ to nothing.
    Hypothesis: Module Owners refrain from giving review- to Newcomers
  • Pattern: Doubtful Reviewer
    There has not been enough early feedback from community.
    A change to functionality is being made.
    “Let's put this in for beta, and make sure we blog about the change a little and see the reaction” (Bug # 412862)
  • Observation: Module Owner Reviews
    Every completed patch has to be reviewed by Module Owners unless somebody finds a flaw in it.
    38 Module Owners
    198 Code or UI inspections
    Module owners do not consider patches based on bug report priority (but developers do).
  • Observation: Peer Reviews
    66 Peer Developers
    Most patches (76%) do not get any Peer Review
    80% of the comments by peers are negative or partly negative
    On an average Peer conducts review before Module Owners
  • Observation: Types of Reviewer Feedback
    Hypothesis: While peers are interested in functionality and usability, Module Owners are also interested in long-term maintainability of the product.
  • Observation: Undiscovered Errors
    8.4% of checked-in patches were backed out.
    Performance and regression issues that remain unidentified during the review.
    It would be interesting to look if there a common pattern in occurrence of undiscovered errors?
    "Based on the site breakage, re-opening and suggesting we back out this change.“ (Bug # 412862)
  • Conclusion
    Peers play a key role in the process by providing ideas before a patch is developed and reviewing developed patches before module owners do.
  • Conclusion
    Module owners are more inclined toward the quality and consistency of developed patches.
  • Take Away
    Module owners and peer developers are complementing each other in reviewing patches. They refine the developed solution based on their own interests and concerns.
  • Debate Questions
    How quantitative/qualitative analysis of software engineering data can be facilitated?
    How empirical findings can be cross-validated with the FLOSS development communities?