Successfully reported this slideshow.
Challenges from a supplier point of view Metrics questions  in Request for Proposals Harold van Heeringen Sogeti   Nederla...
Agenda <ul><li>Request for proposals </li></ul><ul><li>Software estimation using function points </li></ul><ul><li>Typical...
Trends in outsourcing? Harold van Heeringen - Metrics Questions in RFPs  Dutch government wastes billions of euros every y...
Request for Proposal (RFP) <ul><li>An invitation for suppliers, through a bidding process, to submit a proposal on a speci...
Demand issues (RFP sender) <ul><li>Provide the right information </li></ul><ul><ul><li>Detailed </li></ul></ul><ul><ul><li...
Supplier issues (RFP responder)  Can we: <ul><li>Deliver the required functionality ? </li></ul><ul><li>Meet the technical...
Sogeti Sizing, Estimating & Control <ul><li>Division MD – fixed price/fixed date projects </li></ul><ul><li>Sizing, Estima...
Agenda <ul><li>Request for proposals </li></ul><ul><li>Software estimation using function points </li></ul><ul><li>Typical...
Software estimation using function points <ul><li>Function point analysis (NESMA / IFPUG / COSMIC) </li></ul><ul><ul><li>O...
Project estimation <ul><li>Size objectively measured </li></ul><ul><ul><li>Size = xxx function points </li></ul></ul><ul><...
QSM SLIM Estimate
Generic Estimation Model  Harold van Heeringen - Metrics Questions in RFPs  Size Size Defects Effort Duration Defects Prod...
Agenda <ul><li>Request for proposals </li></ul><ul><li>Software estimation using function points </li></ul><ul><li>Typical...
RFP – Typical Metric Questions <ul><li>Q1  - What is your productivity for Java projects? </li></ul><ul><li>Q2  - How long...
Agenda <ul><li>Request for proposals </li></ul><ul><li>Software estimation using function points </li></ul><ul><li>Typical...
Requirements: complete and detailed? time Concept Definition High Level Design Low level   Design   Realization Idea Why  ...
Size always grows ! time Size RFP Concept Definition High   level design Low   level design Realisation Idea Challenge: Wh...
The Software equation (Putnam) <ul><li>  Size/productivity </li></ul><ul><li>  = effort  1/3  * duration  4/3 </li></ul>Ef...
Same project, different duration Effort (manhours) / Peak   size Duration Plan A Duration: 6 months Effort: 4.500 manhours...
Scenarios Estimate / Business Case Cost depending on Time-to-market Example Scenario 1: Duration: 5,5 months Effort: 5.000...
Challenge supplier Price per function point Duration Plan A: 767  €/FP Plan B: 452  €/FP Answer: 380  €/FP ?? Client expec...
Professionalism en realism <ul><li>Expertise </li></ul><ul><ul><li>Use of function point analysis </li></ul></ul><ul><ul><...
Cost of high and low estimates Non- Lineair extra costs Planning errors Enlarging team   more expensive but almost not fa...
High and low estimates in practice Harold van Heeringen - Metrics Questions in RFPs  10.000 5.000 hours 3.000 hours 7.000 ...
Agenda <ul><li>Request for proposals </li></ul><ul><li>Software estimation using function points </li></ul><ul><li>Typical...
Recommendations <ul><li>Ask the right questions </li></ul><ul><ul><li>objective comparison , keeping other factors equal. ...
RFP Question should include <ul><li>Metric to compare between competitors, for instance </li></ul><ul><ul><li>Productivity...
Example of a good RFP Question <ul><li>‘ What is your  productivity rate   (hours/FP)  for a  moderately complex Java  pro...
Assess reality with ISBSG <ul><li>ISBSG database R11 </li></ul><ul><ul><li>International Software Benchmarking Standards G...
Conclusions <ul><li>Ask the right questions: </li></ul><ul><ul><li>Size, Cost, Productivity, Duration and Quality are high...
Next steps <ul><li>IWSM, Stuttgart (Nov. 10-12, 2010) </li></ul><ul><li>Workshop on proposing a standard framework for the...
[email_address] 0031 88 6606600 +3165 metrieken.sogeti.nl @haroldveendam Harold van Heeringen - Metrics Questions in RFPs ...
Upcoming SlideShare
Loading in …5
×

Acosm 2010 Harold Van Heeringen V3

618 views

Published on

Australian Conference on Software Measurment, October 5 2010, Melbourne Australia

  • Be the first to comment

  • Be the first to like this

Acosm 2010 Harold Van Heeringen V3

  1. 1. Challenges from a supplier point of view Metrics questions in Request for Proposals Harold van Heeringen Sogeti Nederland Senior metrics consultant / Cost engineer Sizing, Estimating & Control [email_address] metrieken.sogeti.nl @haroldveendam
  2. 2. Agenda <ul><li>Request for proposals </li></ul><ul><li>Software estimation using function points </li></ul><ul><li>Typical questions in request for proposals (RFP’s) </li></ul><ul><li>Supplier challenges </li></ul><ul><li>Recommendations for the client organization. </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  3. 3. Trends in outsourcing? Harold van Heeringen - Metrics Questions in RFPs Dutch government wastes billions of euros every year on failed ICT projects
  4. 4. Request for Proposal (RFP) <ul><li>An invitation for suppliers, through a bidding process, to submit a proposal on a specific product or service (source: wikipedia) </li></ul><ul><li>Information provided: </li></ul><ul><ul><li>Corporate information </li></ul></ul><ul><ul><li>Schedule of bidding process </li></ul></ul><ul><ul><li>Project Summary </li></ul></ul><ul><ul><li>Detailed overview of the project </li></ul></ul><ul><ul><li>Decision criteria </li></ul></ul>Harold van Heeringen - Metrics Questions in RFPs
  5. 5. Demand issues (RFP sender) <ul><li>Provide the right information </li></ul><ul><ul><li>Detailed </li></ul></ul><ul><ul><li>Up-to-date </li></ul></ul><ul><li>Ask the right questions </li></ul><ul><li>Build a good decision model </li></ul><ul><li>Evaluate the proposals </li></ul><ul><li>Choose wisely </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  6. 6. Supplier issues (RFP responder) Can we: <ul><li>Deliver the required functionality ? </li></ul><ul><li>Meet the technical and quality requirements ? </li></ul><ul><li>Within the time limits required ? </li></ul><ul><li>Answer all RFP questions ? </li></ul><ul><li>Estimate the project costs accurately ? </li></ul><ul><li>Score the best on the clients decision model ? </li></ul><ul><li>Support our claims with proof ? </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  7. 7. Sogeti Sizing, Estimating & Control <ul><li>Division MD – fixed price/fixed date projects </li></ul><ul><li>Sizing, Estimating & Control </li></ul><ul><ul><li>11 (COSMIC) Function Point Analysts </li></ul></ul><ul><ul><li>2 metrics consultants </li></ul></ul><ul><li>Responsible for metrics part of a quotation. </li></ul><ul><ul><li>Size: FPA/COSMIC </li></ul></ul><ul><ul><li>Estimation: QSM / Sogeti tool / ISBSG </li></ul></ul><ul><ul><li>Scenarios ! </li></ul></ul><ul><ul><li>Product: Methodical Estimation Report </li></ul></ul>Harold van Heeringen - Metrics Questions in RFPs
  8. 8. Agenda <ul><li>Request for proposals </li></ul><ul><li>Software estimation using function points </li></ul><ul><li>Typical questions in request for proposals (RFP’s) </li></ul><ul><li>Supplier challenges </li></ul><ul><li>Recommendations for the client organization. </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  9. 9. Software estimation using function points <ul><li>Function point analysis (NESMA / IFPUG / COSMIC) </li></ul><ul><ul><li>Objective ( ISO/IEC) </li></ul></ul><ul><ul><li>Repeatable </li></ul></ul><ul><ul><li>Verifiable </li></ul></ul><ul><li>Quantifying the size om the functional user requirements </li></ul><ul><ul><li>Technology independent </li></ul></ul><ul><ul><li>Independent of implementation </li></ul></ul><ul><li>Is a product measure, not a project measure </li></ul><ul><li>Doesn’t take into account any ‘non-functionals’ </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  10. 10. Project estimation <ul><li>Size objectively measured </li></ul><ul><ul><li>Size = xxx function points </li></ul></ul><ul><li>Estimation: </li></ul><ul><ul><li>Effort (hours) per activity </li></ul></ul><ul><ul><li>Duration (months) and milestones </li></ul></ul><ul><ul><li>Team size (average staff, peak staff) </li></ul></ul><ul><li>Tools </li></ul><ul><ul><li>QSM SLIM suite </li></ul></ul><ul><ul><li>ISBSG repository release 11 </li></ul></ul><ul><ul><li>Sogeti Estimating wizard </li></ul></ul>Harold van Heeringen - Metrics Questions in RFPs
  11. 11. QSM SLIM Estimate
  12. 12. Generic Estimation Model Harold van Heeringen - Metrics Questions in RFPs Size Size Defects Effort Duration Defects Productivity Metric: Size Function points Metric: Size Function Points Metric: Effort Number of hours Manpower buildup Peak staff Metric: Duration Number of weeks Metric: Quality Number of defects Metric: Process productivity Skills and experience team Development environment Complexity Quality Management System External influences Need Software Energy Software development process Waste Time
  13. 13. Agenda <ul><li>Request for proposals </li></ul><ul><li>Software estimation using function points </li></ul><ul><li>Typical questions in request for proposals (RFP’s) </li></ul><ul><li>Supplier challenges </li></ul><ul><li>Recommendations for the client organization. </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  14. 14. RFP – Typical Metric Questions <ul><li>Q1 - What is your productivity for Java projects? </li></ul><ul><li>Q2 - How long do you need to build a .Net application of 500 FP? </li></ul><ul><li>Q3 - What is your price per function point for a 500 FP Oracle system? </li></ul><ul><li>Are these the right questions ?? </li></ul><ul><li>Is the client organization capable of making the right decision based on the answers to these questions? </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  15. 15. Agenda <ul><li>Request for proposals </li></ul><ul><li>Software estimation using function points </li></ul><ul><li>Typical questions in request for proposals (RFP’s) </li></ul><ul><li>Supplier challenges </li></ul><ul><li>Recommendations for the client organization. </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  16. 16. Requirements: complete and detailed? time Concept Definition High Level Design Low level Design Realization Idea Why What How Size: Function Points RFP 4x 3x 2x 1x 0.8x 0.5x Harold van Heeringen - Metrics Questions in RFPs Project Rate 1 4 2 3 3 1 4 1 5 1 6 2 7 4 8 4 9 5 10 5 Average 3
  17. 17. Size always grows ! time Size RFP Concept Definition High level design Low level design Realisation Idea Challenge: Which size should we use in our estimation and which size will the competitors use in their estimation? Harold van Heeringen - Metrics Questions in RFPs Why What How
  18. 18. The Software equation (Putnam) <ul><li> Size/productivity </li></ul><ul><li> = effort 1/3 * duration 4/3 </li></ul>Effort (manhours) Duration Plan A: 6 months, 4.500 manhours Plan B: 7 months, 2.400 manhours Harold van Heeringen - Metrics Questions in RFPs
  19. 19. Same project, different duration Effort (manhours) / Peak size Duration Plan A Duration: 6 months Effort: 4.500 manhours Peak staff : 5,8 fte MTTD: 1,764 days Plan B Duration: 7 months Effort: 2.400 manhours Peak staff: 2,7 fte MTTD: 2,816 days Harold van Heeringen - Metrics Questions in RFPs
  20. 20. Scenarios Estimate / Business Case Cost depending on Time-to-market Example Scenario 1: Duration: 5,5 months Effort: 5.000 Mhr Team size: 6,7 fte Cost: € 430.000 Example Scenario 2: Duration: 5,2 months Effort: 5.500 Mhr Team size: 7,5 fte Cost: € 480.000 Example Scenario 3: Duration: 4,8 months Effort: 5.900 Mhr Team size: 8,3 fte Cost: € 530.000 Example Scenario 4: Duration: 4,5 months Effort: 6.300 Mhr Team size: 9,4 fte Cost: € 620.000 Example Scenario 5: Duration: 5,8 months Effort: 5.200 Mhr Team size: 6,2 fte Cost: € 400.000 Example Scenario 6: Duration: 6,1 months Effort: 4.900 Mhr Team size: 5,8 fte Cost: € 380.000 Example Scenario 7: Duration: 6,3 months Effort: 4.700 Mhr Team size: 5,5 fte Cost: € 360.000 Effort (manhours) Duration Harold van Heeringen - Metrics Questions in RFPs
  21. 21. Challenge supplier Price per function point Duration Plan A: 767 €/FP Plan B: 452 €/FP Answer: 380 €/FP ?? Client expectation What is your price per function point for a 500 FP Oracle system? Harold van Heeringen - Metrics Questions in RFPs
  22. 22. Professionalism en realism <ul><li>Expertise </li></ul><ul><ul><li>Use of function point analysis </li></ul></ul><ul><ul><li>Database with experience data </li></ul></ul><ul><ul><li>Repository with Benchmark data / tooling </li></ul></ul><ul><li>Realisme </li></ul><ul><ul><li>Opportunism: ‘buying projects’ </li></ul></ul><ul><ul><li>Commercial interests </li></ul></ul><ul><li>A non-realistic bid is not in anyones interest !! </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  23. 23. Cost of high and low estimates Non- Lineair extra costs Planning errors Enlarging team  more expensive but almost not faster Extra management attention / overhead Stress: More defects, lower maintainability Lineair extra Costs Extra hours will be spent Harold van Heeringen - Metrics Questions in RFPs
  24. 24. High and low estimates in practice Harold van Heeringen - Metrics Questions in RFPs 10.000 5.000 hours 3.000 hours 7.000 hours 7.000 Estimation Result 5.000 hours 7 months Realistic estimation is the key to succesful projects A Realization (hours) 5.000 15.000 C B B: Realistic 5.000 hours 7 months Successful ! Efficient! Successful ! Not efficient ! 7.000 hours 11 months A: Optimistic 3.000 hours 5 months C: Pessimistic 7.000 hours 11 months FAILS !! 10.000 hours 9 months
  25. 25. Agenda <ul><li>Request for proposals </li></ul><ul><li>Software estimation using function points </li></ul><ul><li>Typical questions in request for proposals (RFP’s) </li></ul><ul><li>Supplier challenges </li></ul><ul><li>Recommendations for the RFP sender </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  26. 26. Recommendations <ul><li>Ask the right questions </li></ul><ul><ul><li>objective comparison , keeping other factors equal. </li></ul></ul><ul><li>Assess the reality value of the quotation </li></ul><ul><ul><li>Establish a range in which the answer has to fall </li></ul></ul><ul><ul><li>Tools: for instance Galorath SEER, QSM SLIM or the ISBSG database </li></ul></ul><ul><li>Demand objective evidence </li></ul><ul><ul><li>Experience data of the supplier </li></ul></ul>Harold van Heeringen - Metrics Questions in RFPs
  27. 27. RFP Question should include <ul><li>Metric to compare between competitors, for instance </li></ul><ul><ul><li>Productivity (hours/FP, Function points/hour, PI) </li></ul></ul><ul><ul><li>Costs (Price/FP) </li></ul></ul><ul><ul><li>Quality (defects per function point, Mean-time-to-defect (MTTD)) </li></ul></ul><ul><li>Technology </li></ul><ul><ul><li>for instance Java, Cobol, Oracle or MS.NET </li></ul></ul><ul><li>Size (in Function Points or COSMIC FP) </li></ul><ul><li>Technical/ Functional Complexity </li></ul><ul><ul><li>for instance high/mediate/low </li></ul></ul><ul><li>Phases/Activities to include </li></ul><ul><ul><li>for instance Technical Design, Coding, Unit testing, systems testing. </li></ul></ul><ul><li>Duration required !! </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  28. 28. Example of a good RFP Question <ul><li>‘ What is your productivity rate (hours/FP) for a moderately complex Java project of 500 function points and a duration of 20 weeks ? Phases to include are technical design, coding, unit testing, systems testing and support of the user organization during the user acceptance test.’ </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  29. 29. Assess reality with ISBSG <ul><li>ISBSG database R11 </li></ul><ul><ul><li>International Software Benchmarking Standards Group </li></ul></ul><ul><ul><li>R11: >5.200 projects ‘Best in Class’ </li></ul></ul><ul><ul><li>Realistic range: 7,2 uur/FP – 11,6 uur/FP </li></ul></ul><ul><ul><li>Realistic range: 4,5 mnd - 9,5 mnd </li></ul></ul>Harold van Heeringen - Metrics Questions in RFPs   ISBSG R11 hours/FP Duration NUMBER OF PROJECTS 24 24 PERCENTILE 10% (P10) 3,5 3,3 mnd PERCENTILE 25% (P25) 7,2 4,5 mnd MEDIAN 8,4 6,0 mnd PERCENTILE 75% (P75) 11,6 9,5 mnd PERCENTILE 90% (P90) 19,6 12,2 mnd
  30. 30. Conclusions <ul><li>Ask the right questions: </li></ul><ul><ul><li>Size, Cost, Productivity, Duration and Quality are highly inter dependent </li></ul></ul><ul><ul><li>State the questions as specific as possible! </li></ul></ul><ul><li>Evaluate bids </li></ul><ul><ul><li>Assess reality of proposals </li></ul></ul><ul><ul><li>Demand evidence of the supplier </li></ul></ul><ul><li>Choose wisely </li></ul><ul><ul><li>Don’t go automatically for the cheapest supplier! </li></ul></ul>Harold van Heeringen - Metrics Questions in RFPs
  31. 31. Next steps <ul><li>IWSM, Stuttgart (Nov. 10-12, 2010) </li></ul><ul><li>Workshop on proposing a standard framework for the use of software metrics in RFP management </li></ul>Harold van Heeringen - Metrics Questions in RFPs
  32. 32. [email_address] 0031 88 6606600 +3165 metrieken.sogeti.nl @haroldveendam Harold van Heeringen - Metrics Questions in RFPs staat voor resultaat

×