Your SlideShare is downloading. ×
Industry - Estimating software maintenance effort from use cases an      industrial case study
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Industry - Estimating software maintenance effort from use cases an industrial case study

746
views

Published on

Paper: Estimating Software Maintenance Effort from Use Cases: an Industrial Case Study …

Paper: Estimating Software Maintenance Effort from Use Cases: an Industrial Case Study

Authors:Yan Ku, Jing Du, Ye Yang, Qing Wang

Session: Industry Tracking 5: Metrics and
Estimation

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
746
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Estimating Software Maintenance Effort from Use Cases: an Industrial Case Study Yan Ku, Jing Du, Ye Yang, Qing Wang Institute of Software, Chinese Academy of Sciences 2011-09-29*This work is supported by the National Natural Science Foundation of China under Grant Nos. 90718042, 60873072, 60903050 and61073044; the National Hi-Tech Research and Development Plan of China under Grant Nos. 2007AA010303, 2007AA01Z186 and2007AA01Z179.
  • 2. AgendaMotivationIndustrial SettingMethodologyCase StudyDiscussionTool Implementation 2
  • 3. Motivation An anecdote in software engineering domain*: Elephant: Discipline Monkey: Agile Elephant & Monkey: Practical*Barry W. Boehm, Richard Turner, Balancing Agility and Discipline: A Guide for the Perplexed, Aug, 2003*Picture Source: www.image.baidu.com 3
  • 4. Motivation “Speaking of the problems we are facing during software lifecycle, hmmm, there is no way a short list. … I hate to, but I have to say that effort estimation especially for maintenance project, if not totally impossible, is really a big challenge -----A Project Manager Formal estimation models: time-consuming, requiring enough information and data---- the elephant Expert Judgment: expert-dependent, easily lead to thumb-up---- the monkey*Picture Source: http://www.zcool.com.cn 4
  • 5. Industrial SettingThe industrial research is inspired by the estimationdilemma a mentioned before.The problem occurred in developing the leading productof a medium-size software enterprise with CMMI ML4in China.The product, named QONE, is a commercial softwareprocess management tool.It has contributed to the process improvement for morethan 300 small and medium-sized software companiesand organizations in China. 5
  • 6. Industrial Setting (Con)Since 2004, QONE has released several major versions as wellas branches for special customized ones in succession.Several of the evolving versions are maintenance projects.Expert estimates were mainly used in the past effort estimationof QONE. Versions Begin Date End DateThe estimation results are not so stable due2004-11-15 v1 2004-10-8 to the objectivityand other issue. v2 2005-7-11 2005-11-30The actual effortv3 other data including use case documents and 2006-1-16 2007-3-30have been accumulated by QONE itself. 2007-10-31 v4 2007-5-28 v5 2007-12-10 2008-7-31 v6 2008-3-20 2008-8-21 v7 2008-9-1 2009-3-20 6
  • 7. MethodologyGoal: Achieving the balance of simplicity, early-estimating and accuracy in one effort estimate .Methodology Principles: Apply use cases as the size metric and introduce requirement elaboration factors to make the estimate inMaintenance task type is not distinguished advancedue to the difficulty for effort classification Introduce adjusted factors as few as possible in order to reduce the complexity Take advantage of the history data to help improve the estimation accuracy. 7
  • 8. Modeling Process Get the lowest level Use the same requirements data unit 8
  • 9. Count DataUse case: number of use cases newUC: new-added modUC: modified reuUC: reused without modified delUC: delete 9
  • 10. Modeling Process 10
  • 11. Count Data & Construct ModelWeight Wmod/ Wreu / Wdel :effort ratio of modified/reused/deleted use case to a new-added one Sizeadjusted = newUC + Wmod * modUC +Wreu * reuUC + Wdel * delUC Effort = A * (Sizeadjusted) B Where Effort is the maintenance effort; Sizeadjusted is the adjusted product size; A is the multiplicative calibration constant; B is the exponential calibration constant 11
  • 12. Modeling Process 12
  • 13. ValidationMetric Definition MRE = |predictive effort – actual effort| / actual effortReferred Measures MMRE: mean magnitude relative erro MdMRE: median magnitude relative error PRED25: the % of the data points with RE<=0.25 PRED30: the % of the data points with RE<=0.30 13
  • 14. Prediction Process Data collection Prediction To apply in different phases during lifecycle, elaboration factors are referred to estimate size input. Sizere-adjusted = EF * Sizeadjusted,where EF is the elaboration factor between higher and lower-level requirements 1 n NUC i whereNRi is the number of higher- EF = ∑ n i=1 NR i level requirements and NUCi is the number of lower-leverl requirements in ith data point*Picture Source: Alistair Cockburn, Writing Effective Use Cases 14
  • 15. Case StudyHistorical data are the projects mentioned inslice 6.Requirements are described in three level: capability goals (CG, least detailed) capability requirements (CR) use cases (UC, most detailed) 15
  • 16. Number Number of Number of Number of Actual EffortVersions Requirements Levels of new modified reused deleted (person/hour) Capability goals(CG) 0 5 13 0 v1 Capability requirements(CR) 0 7 35 0 2284.5 Use cases(UC) 3 10 216 0 Capability goals(CG) 0 11 7 0 v2 Capability requirements(CR) 0 16 26 0 3941 Use cases(UC) 7 22 207 0 Capability goals(CG) 4 15 3 0 v3 Capability requirements(CR) 12 30 12 0 30945 Use cases(UC) 86 94 134 8 Capability goals(CG) 1 13 6 3 v4 Capability requirements(CR) 3 33 17 4 10340.1 Use cases(UC) 50 61 229 17 Capability goals(CG) 1 12 7 1 v5 Capability requirements(CR) 1 18 33 2 7477.5 Use cases(UC) 12 31 301 15 Capability goals(CG) 1 8 12 0 v6 Capability requirements(CR) 2 20 32 0 14903.6 Use cases(UC) 37 30 311 3 Capability goals(CG) 0 3 18 0 v7 Capability requirements(CR) 3 3 51 0 7166 16 Use cases(UC) 15 8 366 4
  • 17. Case Study Weight: Wmod = 0.2, Wreu = 0.05, Wdel = 0Sizeadjusted = newUC + 0.2 * modUC +0.05 * reuUC Parameters Values A 96.9396 B 1.192651 P-value 0.000481 R2 0.928219 Adjusted R2 0.913862 Effort = 96.9396 * (Sizeadjusted) 1.192651 17
  • 18. Validation Result Leave-one-out cross validation is applied. Elaboration factors are referred from A. A. Malik’s research* since the data used there is the subset of our dataset. Metrics CG CR UC MMRE 36.87% 26.18% 26.94% UC yields the best MdMRE 27.09% 24.13% 20.01% result PRED25 0.4286 0.5714 0.7143 PRED30 0.5714 0.7143 0.8571*A. A. Malik, B. W. Boehm, Y. Ku, and Y. Yang, “Comparative Analysis of Requirements Elaboration of an Industrial Product,”Proceedings of the 2nd International Conference on Software Technology and Engineering(ICSTE 2010), Oct. 2010, pp. 46-50 18
  • 19. Validation Result (Cont)Versions Adjusted Size Actual Effort Predictive Effort MRE (%) v1 15.8 2284.5 2916.8925 27.68 v2 21.75 3941 3767.9859 4.39 v3 111.5 30945 23276.7727 24.78 v4 73.65 10340.1 19263.9780 86.30 v5 33.25 7477.5 6136.3455 17.94 v6 58.55 14903.6 11921.1786 20.01 v7 34.9 7166 6628.5125 7.50 1. COTS has been used 2. Increased productivity through requirement management 19
  • 20. Method ComparisonAnalogy Database used for analogy is from China Software Benchmarking Standard Group (CSBSG) 999 software project data from 140 organizations distributed in 15 regions across ChinaCOCOMO2000 Methods MMRE MDMRE PRED25 PRED30 Analogy 33.09% 31.59% 0.2857 0.2857 COCOMO 32.47% 15.47% 0.5714 0.5714 Use case 26.94% 20.01% 0.7143 0.8571 20
  • 21. DiscussionLessons Learned: Use Case Metrics Requirement Elaboration Factors Advantages of use-case based estimation Linear vs. Exponential relationship between effort and use caseThreats to Validity Internal threats: Weak outlier tolerance Complexity of use cases External threats: Use case weight 21
  • 22. Tool Implementation 22
  • 23. Than You !Our Lab: http://itechs.iscas.ac.cn

×