Software Estimating and Performance Measurement


Published on

The importance of Parametric estimation, performance measurement and benchmarking.

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Software Estimating and Performance Measurement

  1. 1. Software Estimation andPerformance Measurement
  2. 2. Harold van HeeringenSoftware Cost Engineer, Sogeti Nederland B.V.Sizing, Estimating & ControlISBSG presidentNESMA boardCOSMIC IAC, representing the NetherlandsSogeti StockholmApril 19 2013Software Estimation andPerformance Measurement
  3. 3. 3OverviewSoftware ProjectsSoftware Project EstimationFunctional Size MeasurementEstimating & Performance MeasurementISBSG repositoryTrends in the marketConclusions
  4. 4. 4Software projectsSoftware project industry: low maturityLow estimation maturityNo or little formal estimation processesNo or little use of historical dataLots of schedule and cost overrunsStandish Chaos reports: Most projects fail or are at leastunsuccessfulLow customer satisfaction ratesIn Europe: only slightly higher than the financial sector
  5. 5. 5Software project estimationMost of the projects are estimated by ‘experts’Bottom up, task by task effort estimationUsually very optimistic (>30%)Experts estimate, but other people (juniors) do the jobForgotten activities (e.g. testscript reviews)No feedback loop with past projects: experts don’t learn from past estimatesand actualsNo scenario’s: duration, team size, etc.Not objective, transparent, verifiable and repeatableNot defendable!‘Easy’ to push back by stakeholdersNo risk assessment (distribution of the estimate)
  6. 6. 6Software Cost engineeringNot a real profession yetConsultant software metricsEstimation officerBid specialistParametric estimatesFunctional size measurement  size of the softwareProductivity rates from historical data or industry dataParametric estimation toolsObjective, repeatable, transparent and verifiableDefendable!!‘Impossible’ to push back by stakeholdersRisk assesment (distribution of the estimate)
  7. 7. 7Functional Size MeasurementFunction Point Analysis (NESMA, IFPUG or COSMIC)Measure the functional user requirements – size in function points;ISO standards – objective, verifiable, repeatable;Strong relation between functional size and project effort needed;More function points means more functionality (e.g. more value);the user or client understands that more functionality means a higher priceWhat to do with the results?Project effort/duration/cost estimationProject benchmarking/performance measurementUse in Request for Proposal management (answer price/FP questions)What about historical data?Company data (preferably for estimation)Industry data (for benchmarking)
  8. 8. 8Software EstimationMeasure the Size of the functional user requirementsDetermine the right productivity figures, based on own history dataor industry project dataUse Parametric tools to estimate effort, duration, team size, quality- non linear relationsScenario analysis- different durations- different team size- varying influence of non-functional requirements- etcetera
  9. 9. 9Software equation (Putnam)Size/productivity= Effort 1/3* duration4/3EffortDurationPlan A: 6 months, 4.500 hoursPlan B: 7 months, 2.400 hours
  10. 10. 10DurationPlan ADuration: 6 monthsEffort: 4.500 hoursMax. team size: 5,8 fteMTTD: 1,764 daysPlan BDuration: 7 monthsEffort: 2.400 hoursMax. team size: 2,7 fteMTTD: 2,816 daysWhich duration have the experts in mind??Effort(hours)Same project, different duration
  11. 11. 11Short demo of QSM SLIM
  12. 12. 12Cost of wrong estimatesNon-linear extra costs-Planning errors-team enlargement more expensive, not faster-Extra management attention / overhead-Stress: More defects, lower maintainability !!Linear extra kostenExtra hours will be used
  13. 13. 13Sogeti SECSizing, Estimating & ControlCertified (COSMIC) Function Point AnalystsMetrics consultantsResponsible for metrics part of a quotation.Size: FPA/COSMICEstimation: SEER-SEM / QSM / Sogeti tool / ISBSGProduct: Methodical Estimation Report (scenario’s)Pricing: EUR/FPQuality: Defects/FPCenters of Excellence:MS.Net, Java, Oracle, mobile, drupal, sharepoint, BI, etc.
  14. 14. 14Estimating & Performance MeasurementPLANEstimateAdministrateEvaluateAdjust &ReportSize measurement: FPAHistorical dataEstimation toolsFinetune Estimation modelAnalyse productivity,Report productivityStart: Estimate requestStart: Project completedResults:- Parametric Estimation- Expert EstimationResult:-Management report,-Adjusted modelResult:-Growing project DB,-Performance measurement-Updated expert knowledgeACTCHECK DOStart: Project startContinuous data collection• effort hours registration• defect registration• change measurement• project characteristicsResult: Project dataData collection andadministration• Collect project data• Measure size• Benchmark the projectStart: PeriodicallyExpert Estimate
  15. 15. 15Performance MeasurementMeasure the performance of completed projects- Size in function points delivered- Defects found in various test phases- Hours spent per main activity- Compare actuals to planDerive the main metrics for the project, e.g.- Productivity (hours/FP)- Cost (EUR/FP)- Quality (Defects/FP)- Schedule / Budget slippageStore and Benchmark the data – internal and external
  16. 16. 16Historical data: ISBSG repositoriesInternational Software Benchmarking Standards GroupIndependent and not-for-profitMembers are non-profit organizations, like IFPUG and NESMAGrows and exploits two repositories of software data:New development projects and enhancements (> 6000 projects)Maintenance and support (> 1000 applications)Everybody can submit project dataDCQ on the siteAnonymousFree benchmark report in returnSpecial reports, Practical Project Estimation book, CompendiumPortal to access the project data
  17. 17. 17Trends in performancePI vs Effective FP100 150 200 250 300 350 400 450 500 550 600Effective FP510152025PIJava Totaal QSM Business 4GL ISBSG Avg. Line StyleNeeded for management to control and manageNeeded to express to the market in proposalsNeeded to make sure we keep improving
  18. 18. 18Trend information
  19. 19. 19TrendsSupplier Performance Measurement- Reality check of supplier’s quotations based on industry data- Agreement on performance improvements: hours/FP or EUR/FPRequest for Proposals- Output based pricing- Price per function point asked- Often cheapest one selected – which is usually the wrong choiceIncreasing professionalism in project estimation- Basis of Estimate – Recommended practice by AACE for software services- Tools, research and better functional sizing methods (e.g. COSMIC)- More people understand the importance of parametric estimates
  20. 20. 20ConclusionsEstimation & Performance Process is important!Increasing maturity of software realizationMore accurate estimates, based on history data  defendableestimates;‘Professional’ image communicated to client;Benchmarking  learning organization  improvement;Performance Measurement  manage by actual performance dataData is needed when answering metrics questions in RFP / RFISogeti – international collaboration is importantShared Best PracticesSogeti SEC wishes to help you when possible!
  21. 21. @haroldveendamHarold.van.heeringen@sogeti.nlHarold van HeeringenSenior Consultant Software Metrics /Software Cost EngineerSogeti Sizing, Estimating & Control (SEC)President ISBSG (International Software Benchmarking Standards Group ( member NESMA (Netherlands Software Metrics Association ( member COSMIC (