Harold van HeeringenSoftware Cost Engineer, Sogeti Nederland B.V.Sizing, Estimating & ControlISBSG presidentNESMA boardCOSMIC IAC, representing the NetherlandsSogeti StockholmApril 19 2013Software Estimation andPerformance Measurement
3OverviewSoftware ProjectsSoftware Project EstimationFunctional Size MeasurementEstimating & Performance MeasurementISBSG repositoryTrends in the marketConclusions
4Software projectsSoftware project industry: low maturityLow estimation maturityNo or little formal estimation processesNo or little use of historical dataLots of schedule and cost overrunsStandish Chaos reports: Most projects fail or are at leastunsuccessfulLow customer satisfaction ratesIn Europe: only slightly higher than the financial sector
5Software project estimationMost of the projects are estimated by ‘experts’Bottom up, task by task effort estimationUsually very optimistic (>30%)Experts estimate, but other people (juniors) do the jobForgotten activities (e.g. testscript reviews)No feedback loop with past projects: experts don’t learn from past estimatesand actualsNo scenario’s: duration, team size, etc.Not objective, transparent, verifiable and repeatableNot defendable!‘Easy’ to push back by stakeholdersNo risk assessment (distribution of the estimate)
6Software Cost engineeringNot a real profession yetConsultant software metricsEstimation officerBid specialistParametric estimatesFunctional size measurement size of the softwareProductivity rates from historical data or industry dataParametric estimation toolsObjective, repeatable, transparent and verifiableDefendable!!‘Impossible’ to push back by stakeholdersRisk assesment (distribution of the estimate)
7Functional Size MeasurementFunction Point Analysis (NESMA, IFPUG or COSMIC)Measure the functional user requirements – size in function points;ISO standards – objective, verifiable, repeatable;Strong relation between functional size and project effort needed;More function points means more functionality (e.g. more value);the user or client understands that more functionality means a higher priceWhat to do with the results?Project effort/duration/cost estimationProject benchmarking/performance measurementUse in Request for Proposal management (answer price/FP questions)What about historical data?Company data (preferably for estimation)Industry data (for benchmarking)
8Software EstimationMeasure the Size of the functional user requirementsDetermine the right productivity figures, based on own history dataor industry project dataUse Parametric tools to estimate effort, duration, team size, quality- non linear relationsScenario analysis- different durations- different team size- varying influence of non-functional requirements- etcetera
12Cost of wrong estimatesNon-linear extra costs-Planning errors-team enlargement more expensive, not faster-Extra management attention / overhead-Stress: More defects, lower maintainability !!Linear extra kostenExtra hours will be used
13Sogeti SECSizing, Estimating & ControlCertified (COSMIC) Function Point AnalystsMetrics consultantsResponsible for metrics part of a quotation.Size: FPA/COSMICEstimation: SEER-SEM / QSM / Sogeti tool / ISBSGProduct: Methodical Estimation Report (scenario’s)Pricing: EUR/FPQuality: Defects/FPCenters of Excellence:MS.Net, Java, Oracle, mobile, drupal, sharepoint, BI, etc.
15Performance MeasurementMeasure the performance of completed projects- Size in function points delivered- Defects found in various test phases- Hours spent per main activity- Compare actuals to planDerive the main metrics for the project, e.g.- Productivity (hours/FP)- Cost (EUR/FP)- Quality (Defects/FP)- Schedule / Budget slippageStore and Benchmark the data – internal and external
16Historical data: ISBSG repositoriesInternational Software Benchmarking Standards GroupIndependent and not-for-profitMembers are non-profit organizations, like IFPUG and NESMAGrows and exploits two repositories of software data:New development projects and enhancements (> 6000 projects)Maintenance and support (> 1000 applications)Everybody can submit project dataDCQ on the siteAnonymousFree benchmark report in returnSpecial reports, Practical Project Estimation book, CompendiumPortal to access the project data
17Trends in performancePI vs Effective FP100 150 200 250 300 350 400 450 500 550 600Effective FP510152025PIJava Totaal QSM Business 4GL ISBSG Avg. Line StyleNeeded for management to control and manageNeeded to express to the market in proposalsNeeded to make sure we keep improving
19TrendsSupplier Performance Measurement- Reality check of supplier’s quotations based on industry data- Agreement on performance improvements: hours/FP or EUR/FPRequest for Proposals- Output based pricing- Price per function point asked- Often cheapest one selected – which is usually the wrong choiceIncreasing professionalism in project estimation- Basis of Estimate – Recommended practice by AACE for software services- Tools, research and better functional sizing methods (e.g. COSMIC)- More people understand the importance of parametric estimates
20ConclusionsEstimation & Performance Process is important!Increasing maturity of software realizationMore accurate estimates, based on history data defendableestimates;‘Professional’ image communicated to client;Benchmarking learning organization improvement;Performance Measurement manage by actual performance dataData is needed when answering metrics questions in RFP / RFISogeti – international collaboration is importantShared Best PracticesSogeti SEC wishes to help you when possible!
@haroldveendamHarold.firstname.lastname@example.orgHarold van HeeringenSenior Consultant Software Metrics /Software Cost EngineerSogeti Sizing, Estimating & Control (SEC)President ISBSG (International Software Benchmarking Standards Group (www.isbsg.org))Board member NESMA (Netherlands Software Metrics Association (www.nesma.nl))IAC member COSMIC (www.cosmicon.com)