Metrics in RFP’sChallenges from the suppliers’ point of view and   recommendations for selecting suppliers    Harold van H...
Agenda• Project estimation based on function points• Typical questions in request for proposals (RFP’s)• Challenges from t...
Sizing projects with function points• Function Point Analysis (NESMA, IFPUG, COSMIC) − Objective (ISO/IEC) − Repeatable − ...
Project Estimation based on functional size• Size objectively measured − Size = xxx function points• Estimation of: − Effo...
Generic Estimation Model                          Effort        Metric: Effort                                        Numb...
Agenda• Project estimation based on function points• Typical questions in request for proposals (RFP’s)• Challenges from t...
Questions for the supplierWill we be able to:• Deliver the required functionality?• Comply to the technical and quality  r...
Typical RFP questions1. What is your productivity for Oracle projects?2. How long will it take for you to build a .Net   a...
Agenda• Project estimation based on function points• Typical questions in request for proposals (RFP’s)• Challenges from t...
Project          Rate Size: Cone of Uncertainty                        1                                                  ...
Size always increases!                RFPSize                                         time  Challenge: What size will we u...
The effort / duration tradeoff            Size/productivity         = Effortx x durationyEffort                    Plan A:...
Same project, different durations                                 A (minimum time)                                 Duratio...
The impact of Duration                                         Scenarios based on duration Effort hours                   ...
Challenge for supplier                           Client expectation   3. What is your price per function point for        ...
Professionalism and realism• Expertise − Use of function point analysis − Database with experience data − Repository with ...
Extra costs with incorrect estimations               Non- Lineair extra costs>100%          -Plannings errors             ...
In practice Proposal         Result                                               15.000A: Optimistic    Fails !          ...
Agenda• Project estimation based on function points• Typical questions in request for proposals (RFP’s)• Challenges from t...
Recommendations for the client• Ask the right questions − objective comparison, keeping as many relevant factors   as poss...
What is a good question?• Metric to compare, for instance: − Productivity (hours/FP, FP/month) − Cost (Price/FP) − Quality...
Example of a good question‘What is your price per function point for amoderately complex Java project of 500 functionpoint...
Reality value of the proposal• ISBSG data portal − International Software Benchmarking Standards Group − >5.800 projects ‘...
SEER-SEM• Reality assessment in SEER-SEM• Simulate the project based on the appropriate  knowledge bases in the tool SEER-...
Recommendations summarized• Ask the right questions: − Size, Cost, productivity, duration en quality are highly   interdep...
Summary• Suppliers face a number of difficulties when they have  to answer a ‘one dimensional’ question• More mature suppl...
Sogeti Sizing, Estimating & Control      Thanks for your attention !             Harold van Heeringen          Sizing, Est...
Upcoming SlideShare
Loading in …5
×

Van heeringen metrics in rf ps

1,183 views

Published on

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,183
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Van heeringen metrics in rf ps

  1. 1. Metrics in RFP’sChallenges from the suppliers’ point of view and recommendations for selecting suppliers Harold van Heeringen Sizing, Estimating & Control harold.van.heeringen@sogeti.nl @haroldveendam ISBSG president NESMA Board COSMIC IAC
  2. 2. Agenda• Project estimation based on function points• Typical questions in request for proposals (RFP’s)• Challenges from the suppliers’ point of view• Recommendations for client organizations 2
  3. 3. Sizing projects with function points• Function Point Analysis (NESMA, IFPUG, COSMIC) − Objective (ISO/IEC) − Repeatable − Verifiable• Quantifies the size of the functional user requirements − Independent of the technology used − Independent of the implementation method• A measure of the size of the product, not the project !• ‘non-functionals’ are not measured 3
  4. 4. Project Estimation based on functional size• Size objectively measured − Size = xxx function points• Estimation of: − Effort (hours) per function/role − Duration (months) and milestones − Team size (in fte) − Quality (defects during test and after delivery)• Tools − Galorath SEER-SEM − ISBSG data portal − Sogeti Estimating wizard − Other tools 4
  5. 5. Generic Estimation Model Effort Metric: Effort Number of hours Energy Manpower buildup Peak staff Productivity Size Software Size Need development Software process Defects Time Metric: Process productivity Duration Metric: Size Skills and experience team Development environmentMetric: Size Metric: QualityPoints Function Waste Metric: Duration Number of defects ComplexityFunction points Number of weeks Quality Management System Defects External influences 5
  6. 6. Agenda• Project estimation based on function points• Typical questions in request for proposals (RFP’s)• Challenges from the suppliers point of view• Recommendations for client organizations 6
  7. 7. Questions for the supplierWill we be able to:• Deliver the required functionality?• Comply to the technical and quality requirements?• Comply to the posed prerequisites?• Answer all questions in the RFP?• Estimate the costs of the project accurately?• Score the best in the decision model that the client will use?• Prove our claims? 7
  8. 8. Typical RFP questions1. What is your productivity for Oracle projects?2. How long will it take for you to build a .Net application of 500 FP?3. What is your price per function point for a 500 FP Java system?• Are these the right questions?• Is it possible for the client organization to make the right choice based on the answers to questions like these? 8
  9. 9. Agenda• Project estimation based on function points• Typical questions in request for proposals (RFP’s)• Challenges from the suppliers point of view• Recommendations for client organizations 9
  10. 10. Project Rate Size: Cone of Uncertainty 1 2 4 3 3 1Size: Function Points 4 1 4x 5 1 6 2 RFP 7 4 3x 8 4 9 5 10 5 2x Average 3 time 1x 0.8x 0.5x Idea Concept High Low level Realization Definition Level Design Design Why What How 10
  11. 11. Size always increases! RFPSize time Challenge: What size will we use in our estimate and which size Global theDetailed Idea Concept will Definition design competitor use? design Realization Why What How 11
  12. 12. The effort / duration tradeoff Size/productivity = Effortx x durationyEffort Plan A: 6 months, 4.500 hours Plan B: 7 months, 3.400 hours Duration 12
  13. 13. Same project, different durations A (minimum time) Duration: 6 months Effort: 4.500 hours Max. team size: 5,8 fte MTTD: 1,764 days Effort (hours) B (optimal effort) Duration: 7 months Effort: 3.400 hours Max. team size: 3,9 fte MTTD: 2,816 days Duration 13Size and Productivity constant
  14. 14. The impact of Duration Scenarios based on duration Effort hours Minimal Time Optimal EffortExample Scenario 1: 7: 6: 5: 4: 3: 2:Duration: 5,5 months 6,3 6,1 5,8 4,5 4,8 5,2Effort: 5.000 Mhr 4.700 4.900 5.200 6.300 5.900 5.500 Estimate / Business CaseTeam size: 6,7 fte 5,5 5,8 6,2 9,4 8,3 7,5 Duration Cost depended on Time-to- 14Cost: € 430.000 360.000 380.000 400.000 620.000 530.000 480.000 market
  15. 15. Challenge for supplier Client expectation 3. What is your price per function point for a 500 FP Java system? Answer: 452 €/FP ??Price per function point Minimum time: 767 €/FP Optimal effort: 452 €/FP Duration 15
  16. 16. Professionalism and realism• Expertise − Use of function point analysis − Database with experience data − Repository with Benchmarkdata / tooling• Realism − Opportunism: ‘Buying projects’ − Commercial interests• To make an unrealistic offer is in nobody’s interest! 16
  17. 17. Extra costs with incorrect estimations Non- Lineair extra costs>100% -Plannings errors -Larger team much more expensive, barely faster -Extra management attention / overhead -Stress: More defects, lower maintainability of the code !! Extra Costs Lineair extra costs Extra hours will be spent Underestimation Overestimation 0% Too low estimates Realistisc estimates Too high estimates 17
  18. 18. In practice Proposal Result 15.000A: Optimistic Fails ! Realisation (hours) 3.000 hours 10.000 hours 5 months 12 months 10.000B: Realistic Succesful ! 5.000 hours Efficient! 7 months 5.000 hours 7.000 7 months 5.000 3.000 hours 5.000 hours 7.000 hoursC: Pessimistic Succesful ! Not efficient ! A B C 7.000 hours 11 months 7.000 hours 11 months 18
  19. 19. Agenda• Project estimation based on function points• Typical questions in request for proposals (RFP’s)• Challenges from the suppliers point of view• Recommendations for client organizations 19
  20. 20. Recommendations for the client• Ask the right questions − objective comparison, keeping as many relevant factors as possible equal.• Perform a reality check of the proposal − Compose a range in which the proposal should be − Tools: Galorath SEER-SEM or the ISBSG database• Ask for objective proof − Experience data of the suppliers − Assess if the supplier can deliver software as productive as promised 20
  21. 21. What is a good question?• Metric to compare, for instance: − Productivity (hours/FP, FP/month) − Cost (Price/FP) − Quality (defects/FP, Mean-time-to-defect (MTTD), Maintainability index)• Technology − For instance Java, Cobol, Oracle or MS.NET• Size (in Function points or COSMIC FP)• Technical/ Functional complexity − For instance: high/average/low• Phases/Activities included − For instance Technical design, Coding, Unit test, systems test.• DURATION !! 21
  22. 22. Example of a good question‘What is your price per function point for amoderately complex Java project of 500 functionpoints and a duration of 20 weeks?Activities to include are technical design, coding,unit testing, systems testing and support of theuser organization during the user acceptance test.’The price per function point also includes alloverhead activities, like project lead and qualitymanagement. 22
  23. 23. Reality value of the proposal• ISBSG data portal − International Software Benchmarking Standards Group − >5.800 projects ‘Best in Class’ ISBSG R11 Hours/FP Duration VALUES IN INTERVAL 24 24 PERCENTILE 10% (P10) 3.5 3.3 months PERCENTILE 25% (P25) 7.2 4.5 months MEDIAN 8.4 6.0 months PERCENTILE 75% (P75) 11.6 9.5 months PERCENTILE 90% (P90) 19.6 12.2 months• Realistic range: 7.2 hours/FP – 11.6 hours/FP• Realistic range: 4.5 - 9.5 months 23
  24. 24. SEER-SEM• Reality assessment in SEER-SEM• Simulate the project based on the appropriate knowledge bases in the tool SEER-SEM Min. Time Opt. Duration PDR (Hours/FP) 8.1 13.7 Duration (months) 4.3 6.9• Realistic range: 8.1 h/FP – 13.7 h/FP• Realistic range: 4.3 months – 6.9 months 24
  25. 25. Recommendations summarized• Ask the right questions: − Size, Cost, productivity, duration en quality are highly interdependent − The goal is to try to get answers that are as comparable to each other as possible• Reality check of the proposals − Analyze Benchmark repositories or tools to come up with a realistic range. Dont accept unrealistic proposals − Always ask the supplier for evidence that they are as productive as they claim.• Choose wisely − When the cheapest proposal always wins, too few good questions have been asked! 25
  26. 26. Summary• Suppliers face a number of difficulties when they have to answer a ‘one dimensional’ question• More mature suppliers that can prove their performance based on experience data are often outbidded by suppliers that have no idea about their performance and just take the risk• However, unrealistically optimistic expectations lead to huge failures!• Clients as well as suppliers should create a common basis of understanding, so that the industry can become more mature. 26
  27. 27. Sogeti Sizing, Estimating & Control Thanks for your attention ! Harold van Heeringen Sizing, Estimating & Control harold.van.heeringen@sogeti.nl @haroldveendam Sogeti Sizing, Estimating & Control NESMA – board member NESMA – chair working group COSMIC NESMA – chair working group Benchmarking NESMA – working group Packages COSMIC – International Advisory Counsil COSMIC – Benchmarking Committee ISBSG – President 27

×