0
Software Development Cost:
                How Much? You Sure?
                  Executive Summary

         Jairus Hihn  ...
IMPORTANCE/BENEFITS

Data at JPL indicates that
   flight software planned effort grows by
     75% from Initial Confir...
RELEVANCE to NASA

NASA even at the center level has very limited
 knowledge of its actual software cost performance
 and...
TASK Problem
                              “Cost” is a quality issue
                                  If development co...
Problem: Are we using the right models?
 A major reason for poor software cost estimation:
    … NASA’s managers don’t h...
APPROACH: validated, relevant
     models that handle uncertainty
 Change the rules of the game
 Old way: reuse someone ...
Accomplishments

 2CEE (Windows application)
    21st century estimation                                        50% to 7...
Major Accomplishments

 Published 11 papers so far
    Including publications in IEEE TSE and IEEE Software
 Obtained, ...
Next Steps

Minor tool clean up and get it released.
Currently being validated in real world setting
   Checking perfor...
Come see our demo it will knock your socks off !!!




          SAS_07_Software Development_Cost_Hihn/Menzies   10
Upcoming SlideShare
Loading in...5
×

Sas 07 Costing

659

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
659
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Sas 07 Costing"

  1. 1. Software Development Cost: How Much? You Sure? Executive Summary Jairus Hihn Tim Menzies Jet Propulsion Laboratory Lane Department of Computer Science California Institute of Technology West Virginia University jhihn@jpl.nasa.gov tim@menzies.us NASA Software Assurance Symposium, Morgantown, WV September 25-27, 2007 1 SAS_07_Software Development_Cost_Hihn/Menzies
  2. 2. IMPORTANCE/BENEFITS Data at JPL indicates that  flight software planned effort grows by 75% from Initial Confirmation 55% from Confirmation Review  Schedule slips by 24% from Confirmation Review  Allocated budgets are seriously out of line with software team estimates The products of this research task will enable the ability to  improve our performance against these metrics  Develop ‘reasonable’ estimates for IV&V resource allocation to verify NASA mission software SAS_07_Software Development_Cost_Hihn/Menzies 2
  3. 3. RELEVANCE to NASA NASA even at the center level has very limited knowledge of its actual software cost performance and will be doomed to repeat the past if cannot learn the lessons from its past NASA must establish the capability to  Update estimates quickly as designs evolve  Have sufficient basis of estimate to defend reasonable software cost estimates  Understand the risk and uncertainty within our estimates and budgets SAS_07_Software Development_Cost_Hihn/Menzies 3
  4. 4. TASK Problem  “Cost” is a quality issue  If development costs is under- estimated…  … developers will be forced into many quality-threatening cost-cutting measures.  Can you think of anything you can do ...  … that harms a project more…  … than allocating insufficient resources? SAS_07_Software Development_Cost_Hihn/Menzies 4
  5. 5. Problem: Are we using the right models?  A major reason for poor software cost estimation:  … NASA’s managers don’t have information they need  Not enough relevant data  Current costing models are brittle and improperly tuned  e.g. “officially”, COCOMO’s tuning parameters vary 2.5 <= a <= 2.94 0.91 <= b < 1.01  Which is nothing like what we see with real NASA data, 3.5 <= a <= 14 0.65 <= b <= 1  Conclusion: we have to take far more care when building our cost models SAS_07_Software Development_Cost_Hihn/Menzies 5
  6. 6. APPROACH: validated, relevant models that handle uncertainty  Change the rules of the game  Old way: reuse someone else’s cost model  New way: build relevant and validated models,  Used with uncertainty “what-if” queries  Relevant: NASA historical data = table  Rows are projects (some of which aren’t relevant)  Columns are project features (some which don’t matter to you)  Row pruners: automatically find relevant projects  Column pruners: automatically dump uninformative features  Validated:  In studies with real NASA data  This automatic method out-performed existing state-of-the-art methods  Uncertainty & “what-ifs”  Planned projects aren’t “one thing”  But a range of possible “things”  Explore the “things” looking for the range of possibilities SAS_07_Software Development_Cost_Hihn/Menzies 6
  7. 7. Accomplishments  2CEE (Windows application)  21st century estimation 50% to 70%  New high-water mark in cost probability range estimation methods  Generates space of possible estimates . TRL=7  Validated on NASA data  We have tried 158 ways to build cost models  Rejected 154 methods  Find the four row/column pruners that matter  Out-performs existing cost models  Deployed at NASA  Currently, under- going a 12 month trial SAS_07_Software Development_Cost_Hihn/Menzies 7
  8. 8. Major Accomplishments  Published 11 papers so far  Including publications in IEEE TSE and IEEE Software  Obtained, cleaned, translated 124 records into a COCOMO II data set  2cee an effort estimation and model tuning tool  Estimator and Expert mode  Runs in Windows SAS_07_Software Development_Cost_Hihn/Menzies 8
  9. 9. Next Steps Minor tool clean up and get it released. Currently being validated in real world setting  Checking performance against current JPL estimation tools Update training materials Deliver two training workshops at JPL and one other center Deliver data set Final report and it is all over. Run in parallel with current models over the next year to evaluate 2cee for acceptance as JPL supported effort estimation tool Submit at least one more paper to a major journal SAS_07_Software Development_Cost_Hihn/Menzies 9
  10. 10. Come see our demo it will knock your socks off !!! SAS_07_Software Development_Cost_Hihn/Menzies 10
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×