Five major aspects of peer review and their efficient combinations

2,722 views

Published on

This paper identifies five major aspects of peer review: timing, coverage, form, check list, metrics, and summarizes common practices of every aspect. The high efficient combinations of these five aspects, which help software developing teams to improve the effectiveness of peer-reviews, are recommended for various types of project stages and working products,. According to the actual situation, any organization that carries out peer-review activities should understand appropriate scenarios of these five major aspects, and combine them advisably in order to achieve better results.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,722
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
17
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Five major aspects of peer review and their efficient combinations

  1. 1. 5th World Congress for Software Quality – Shanghai, China – November 2011 Five Major Aspects of Peer Review and Their Efficient Combinations 1st Author 2nd Author 3rd Author Zhang Keqiang Wen Yang Huang Linfang Shanghai, China Shanghai, China Shanghai, China zhangkeqiang@gmail.com wangyang_178314@baosight.com huanglinfang@baosight.comAbstractThis paper identifies five major aspects of peer review: timing, coverage, form, check list, metrics, andsummarizes common practices of every aspect. The high efficient combinations of these five aspects,which help software developing teams to improve the effectiveness of peer-reviews, are recommended forvarious types of project stages and working products,. According to the actual situation, any organizationthat carries out peer-review activities should understand appropriate scenarios of these five major aspects,and combine them advisably in order to achieve better results.1. IntroductionPeer review is one of effective quality management methods at the product early stage. When CMM/CMMIspreads globally, peer review is carried out widely as the basic requirement and effective practice inCMM/CMMI. But after peer review lasts some period of time, there are still much of confusion. For example,How to determine the coverage of peer review? How to judge the effectiveness of peer review? How toimprove the efficiency of peer review, which peer review should be hold meeting? How to measure peerreview. According to years of peer review practice, this paper identifies five major aspects of peer reviewand their common practice. For different work products, high efficient combinations of five aspects arerecommended.2. Five Major Aspects of Peer ReviewFive Major Aspects of Peer Review are listed below. 1) Time 2) Coverage 3) Format 4) Checklist 5) Measurement2.1 TimeThe common time of peer review is listed below. Before the final version:before the final version of work product, one or several peer reviews are arranged in a short period. Commonly, the entire work product is checked with the key points. By stages:the work product is reviewed at every stage. The focus of review is the incremental parts of the work products. short cycle:regular review, the time of cycle is less than one week, even daily review. Learing style:the typical parts are selected for sharing knowledge and experience.2.2 CoverageThe common options of coverage are 100%. This is suitable for high quality requirement. About 50%, This is suitable for flexable situation. About 30%, This issuitable for low quality requirement.2.3 Format
  2. 2. 5th World Congress for Software Quality – Shanghai, China – November 2011The main formats of peer review have four. Single offline peer review. The reviewer receives the work product, and reviews it offline, then give the results to the author. This is the most cost-saving format. Pair online peer review. The reviewer and the author work together with interactive questions and answers. The another name of this format maybe "Desk check". Meeting with precheck peer review. Before the meeting, several reviewers check the work product independently, in the meeting reviewers and authors discuss the found and affirm the issues which are required to modify. The another name of this format is"Inspection". Meeting without precheck peer review. Reviewer need not check the work product before the meeting. In the meeting authors introduce the work products, reviews raise the problems and discuss them on the spot.2.4 Check ListCheck list of peer review have below styles. No checklist, reviewers judge the problems by own knowledge and experience。 Short checklist, The short checklist is maintained meticulously according to the frequent defects, the length of checklist is limited. Reviewers use the checklist as seeds of check point. Long checklist:The comprehensive long checklist is provided, reviewers check the items in the list one by one, and need no more expanding.2.5 MeasurementThere are some methods of measurements for peer review, the main is listed below: No record Record defects Record efforts, size of artifact reviewed Record the execution of checklist Carry out statistics process control (SPC)3. Recommendation of High efficient peer review combinations3.1 Necessity of high efficient combinationThe sum of five aspects combinations is a hug number, according the above aspects, the sum is4*3*4*3*5=720. For one peer review, only one combination can be selected, it is somewhat difficult to makethe right selection. so it is worth thinking of high efficient combination for better result.3.2 Work product of peer reviewPeer review can be carried out when the work product is partlaly completed. It is an effective method ofquality control at early stage. The work products in process are worth taking peer reviews. The commonwork products of peer review are requirements documents, design documents, source code, varied plans,test-case, user documents.3.3 Suitable situation of peer reviewEven for the same work product, different peer review combinations should be taken at the differentsituation, expercially in the different software develop life cycle models. There are traditional SDLC, such aswaterfall, prototype, also there are some agile software development methodology, such as eXtremePrograming, Scrum etc.3.4 Recommendation of requirement document peer review Table 1. recommendations of requirement document peer review Five Major Aspects of Peer Review and Their Efficient Combinations Page 2 of 7
  3. 3. 5th World Congress for Software Quality – Shanghai, China – November 2011 Recommendation 1 Recommendation 2 Situation Early stage, uncompleted Last stage , completed documents in documents in traditional traditional SDLC SDLC Time By stage Before the final version Coverage Several times achieve achieves 100% 100% Format Single offline Meeting with precheck Checklist Short checklist Short checklist Measurement Record defects Record defects Record the execution of Record the execution of checklist checklist3.5 Recommendation of design document peer review Table 2. recommendation of design document peer review Recommendation 1 Recommendation 2 Situation Early stage, uncompleted completed documents in traditional documents in traditional SDLC SDLC Time By stage Before the final version Coverage Several times achieve achieves 100% 100% Format Meeting without precheck Meeting with precheck Checklist Short checklist Short checklist Measurement Record defects Record defects Record the execution of checklist3.6 Recommendation of code peer review Table 3. previous 2 recommendation of code peer review Recommendation 1 Recommendation 2 Situation Quality target ofZero Agile development defect , in CMMI4or CMMI5 Time short cycle short cycle Coverage 100% 100% Format Single offline Pair on-line review , even pair programing Checklist Short checklist None Measurement Record defects None Record the execution of checklist Record efforts of review, size of the reviewed. Carry out statistical process control (SPC) The below graphic is a example of code peer review SPC. Five Major Aspects of Peer Review and Their Efficient Combinations Page 3 of 7
  4. 4. 5th World Congress for Software Quality – Shanghai, China – November 2011 缺陷密度 缺陷密度(个/千行) 50.00 CL 45.00 UCL 40.00 LCL 35.00 30.00 25.00 20.00 15.00 10.00 5.00 0.00 106 113 127 141 148 155 162 169 176 190 204 211 15 22 29 36 50 64 71 78 85 92 99 120 134 183 197 43 57 1 8 Figure1- example of code peer review defect density SPC, the title in graphic is "defect density(/kloc)" Table 4. last 2 recommendation of code peer review Recommendation 3 Recommendation 4 Situation Share experience very important code Time Learning style short cycle Coverage Select the typical code 100% Format Meeting without precheck Meeting with precheck Checklist Short checklist Short checklist Measurement Record defects Record defects3.7 Recommendation of plan documents peer review Table 5. recommendation of varied plans peer review Recommendation 1 Recommendation 2 Situation Such as project develop plan Such as testing plan Time Before the final version Before the final version Coverage 100% 100% Format Meeting with precheck Single offline Checklist Long checklist Long checklist Measurement Record defects Record defects Record the execution of Record the execution of checklist checklist Record efforts of review, Record efforts of review, Record size of the reviewed. Record size of the reviewed.The below graphic is a screen-shot of a plan checklist, it is shown that there are 74 check items with numberOf uses and number of defect found in this checklist. Five Major Aspects of Peer Review and Their Efficient Combinations Page 4 of 7
  5. 5. 5th World Congress for Software Quality – Shanghai, China – November 2011 Figure 2. development plan long checklist, the columns are "content of check item", "number of uses", "number of defect found".3.8 Recommendation of test case peer review Table6-recommendation of test case peer review Recommendation 1 Recommendation 2 Situation Quality target ofZero defect , in Agile development CMMI4or CMMI5 Time short cycle short cycle Coverage 100% 50% Format Single offline Pair on-line review , even pair programing Checklist Short checklist none Measurement Record defects none Record the execution of checklist3.9 Recommendation of user documents peer review Table 7-recommendation of user documents peer review Recommendation 1 Recommendation 2 Situation voluminous user document Short user documents, such as presentation, voluminous Time By stages Before the final version Coverage 30% 100% Format Single offline Meeting without precheck Checklist Short checklist No checklist Measurement Recoord defects Record defects Record efforts, size of the reviewed Record the execution of checklist4. Rules of recommended combinationsSome rules can be concluded From the above recommendation. Short checklist must be maintained meticulously. The typical check items which helps to find more defects should be included in checklist, and the check results should be recorded. From these records, the items which cant bring defects can be found, and be replaced by new items from new defects. Code and test case are increase fast, only short cycle peer review can cover the increase 100%. The shorter cycle brings more effectiveness, but more cost, these need balance. The extreme peer review is pair work which is reviewing all the time. For long term benefits and absolute good quality, it is worth carrying out highly frequent and full coverage peer review of code and test case. Single offline peer review is recommended at several places. It is the most cost-saving format which can deal with any work product. User documents mostly take meeting without precheck peer review, because the reviewer in meeting without precheck is similar with the real user who is listening the presentation. The learning style peer review can help the newbies to improve their skill, help work mates to share their experience. If there is no frequent and high coverage peer review, then the learning style peer review is highly recommended to ally all teams understanding. All kinds of Plan need a complete check list, because every part in plan is important which will affect the next step. So it is worth checking the plan carefully by long complete check list. Five Major Aspects of Peer Review and Their Efficient Combinations Page 5 of 7
  6. 6. 5th World Congress for Software Quality – Shanghai, China – November 20115. ExampleThe above combinations are used in a CMMI5 organization, the screen-shot is came from the real systemat that organization. Through the peer review and other quality management methods, the more defects arefound at the early stages, the testing defect density decreased obviously. Please see the below figure. 功能点缺陷密度(个/个) 0.2 0.1520.145 0.15 0.089 0.1 0.058 0.0333 0.0341 0.05 0 2008年7月 2009年7月 2010年7月 Figure 5. Density of testing defect trend, The title is "defect per function point density".The test turns to final release also decrease ,so the period of test is shorten, see below. 5 4.57 平均测试轮次 3.92 3.89 3.75 4 3.53 3 2.27 2.21 2.04 2 1 0 2008年1月 2008年12月 2009年11月 Figure 6. Turns to release trend, the Title is "Average test turns"6. ConclusionIn this paper, five major aspects of peer review are identified. The frame of peer review is establishedclearly. It it more convenient to analyse peer review. The typical practise of five aspects of peer reviews areprovided. According to different situations, high efficient combinations of five aspects are proposed.These practise in the five aspects of peer review have their strengths and weaknesses. The organizationneed combine these aspects according to its own situation advisably so that the better results can beachieved.References ®[1] Mary Beth Chrissis, Mike Konrad, Sandy Shrum , Capability Maturity Model Integration V1.2http://www.sei.cmu.edu/cmmi/[2] IEEE Std. 1028-1997, "IEEE Standard for Software Reviews" Five Major Aspects of Peer Review and Their Efficient Combinations Page 6 of 7
  7. 7. 5th World Congress for Software Quality – Shanghai, China – November 2011Contact author  zhangkeqiang@gmail.com  http://weibo.com/zhangkeqiang  http://hi.baidu.com/hespr  http://blog.csdn.net/zhangmike Five Major Aspects of Peer Review and Their Efficient Combinations Page 7 of 7

×