Future of Software Analysis & Measurement_CAST


Published on

Read this informative presentation with contributions from experts on the Future of Software Analysis and Measurement: Dan Galorath, President & CEO of Galorath Inc and Bill Curtis, SVP & Chief Scientist, CAST will have an engaging discussion moderated by David Herron, VP, Knowledge Solution Services, David Consulting Group. These industry veterans will how SAM tools coupled with estimate models can impact organizational performance through increased ROI, customer satisfaction and business value.

To view the webinar, visit http://www.castsoftware.com/news-events/event/future-of-SAM?gad=ss

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Future of Software Analysis & Measurement_CAST

  1. 1. The Future ofSoftware Analysis & Measurement
  2. 2. The Future ofSoftware Analysis & Measurement
  3. 3. Daniel D. Galorath 40+ years Software & Software Management Experience Founder & CEO Galorath Incorporated, SEER By Galorath 1980 MBA Management, California State Universities 1984 Began working software estimation, planning & control 2009 Society of Cost Estimation and Analysis (SCEA) Lifetime Achievement award 2001 ISPA Freiman Award, lifetime achievement award 2006 Book: Software Sizing, Estimation, and Risk Management
  4. 4. SEER® Empowering Project Estimation, Planning & Control Since 1988 SEER parametrically estimates project cost, effort, duration, cost & risk Answer Fundamental Management Questions  How much effort is required to complete an activity?  How much calendar time is needed to complete an activity?  What is the total cost of an activity?  How reliable will it be?  What can we do better?  What If? www.galorath.com
  5. 5. David Herron Business Development Manager and VP of Knowledge Solution Services for David Consulting Group Consulting and coaching services for a variety of IT organizations throughout the US and Canada Acknowledged authority in the areas of performance measurement, process improvement and organizational change management Noted author and lecturer; co-authored several books on topics relating to IT performance measurement and Function Point Analysis
  6. 6. David Consulting Group DCG is an international IT process improvement and measurement company managing value-driven engagements with companies and government agencies around the world Software Process Improvement Utilizing CMMI, Six Sigma, Lean and Agile methods Software Sizing Using IFPUG Function point Counting and alternative sizing techniques Software Measurement Providing roadmap planning, estimation models, performance benchmarks and outsourcing SLA support IT Performance Improvement Improving IT operations through ITIL and IT Governance
  7. 7. Dr. Bill Curtis Senior Vice President and Chief Scientist Industry luminary responsible for influencing CAST’s scientific and strategic direction Best known for leading development of the Capability Maturity Model (CMM) which has become the global standard for evaluating the capability of software development organizations. Prior to joining CAST,  Co-Founder of TeraQuest, the global leader in CMM- based services,  Directed the Software Process Program at the Software Engineering Institute (SEI) at Carnegie Mellon University.
  8. 8. CAST Application Intelligence Platform Most enterprises measure everything but the product delivered to the business CAST Application Intelligence Platform (AIP) measures the product itself Process Product  Robustness  Performance Time & Quality  Security CAST AIP  Planning Duration &Size  Changeability  Estimation  Transferability  Scheduling  Size  Time Tracking  Cost Tracking  Requirements Function & Effort & Budget Scope  Earned Value  User Acceptance  Usability
  9. 9. QUESTIONS What are the single most important business drivers that lead an organization to embark on a SAM initiative? How does software analysis and measurement relate to business outcomes? What organizational behaviors are most favorable for a successful SAM program? What is the relationship between SAM and IT governance? How can measurement and analysis interact with estimating to provide improved visibility and information to make the best management decisions regarding software? How can anyone provide a viable estimate software development, software maintenance, or continuing innovation work and what use it is? How can we determine which software projects with have the highest ROI?
  10. 10. What are the single most importantbusiness drivers that lead an organization to embark on a SAM initiative?
  11. 11. Human Nature Optimism in SoftwareHBR Article explains this Phenomenon: Humans seem hardwired to be optimists We routinely exaggerate benefits and discount costsDelusions of Success: How Optimism Undermines Executives Decisions(Source: HBR Articles | Dan Lovallo, Daniel Kahneman | Jul 01, 2003) Solution: Temper with “outside view” Software Analysis & Measurement can temper Viable estimation can temper Don’t remove optimism, but balance optimism & realism 11
  12. 12. IT Failures are Pervasive: And Even Successful Projects may not be Cutter Consortium Software Project Survey:  62% overran original schedule by more than 50%  64% more than 50% over budget  70% had critical product quality defects after release Standish Group CHAOS Report  46% challenged  19% failed  35% successful~$875 billion spent on IT~$300 billion spent on IT projects~$57 billion wasted annuallyEver increasing “Technical Debt” ROI of Applied Software Analysis & Measurement is HUGE
  13. 13. Manual Estimates Human reasons for error (Software Analysis & Measurement can help) Desire for “credibility” motivates overestimate behavior (80% probability?)  So must spend all the time to be “reliable”  Better approach force 50% probability & have “buffer” for overruns Technical pride causes underestimates Buy-in causes underestimates 13
  14. 14. Important Business Drivers for Structural Measurement Poor software 1 from suppliers Embarrassing 2 disasters High cost of 3 maintenance
  15. 15. How does software analysis andmeasurement relate to business outcomes?
  16. 16. Goal Question Metric Approach Goal Organizational Goal Question Development Contractors Organizations Metric Combine goal-orientation bottoms up, decision-support & other operational management techniques  Going to weather.com and deciding to bring an umbrella is decision support
  17. 17. How Does Measurement Relate to Business Outcomes Business Tactical Objectives Application Health Value Maximize standards compliance Factors Improve software readability TRANSFERABILITYMinimize business Reduce vendor lock-in allows new teams to quickly begin working with an application risks Reduce cost of ownership Cost Reduce injected defects Reduce application mistakes CHANGEABILITY makes an application easier andMinimize IT costs Reduce application rework quicker to modify Ease team handoffs Minimize liquidated damagesMaximize business Maximize application availability ROBUSTNESS improves application stability & agility Minimize degraded service reduces injecting new defects Reduce learning curves Risk Reduce modification effort Optimize work Accelerate new function delivery PERFORMANCE productivity Reduces degraded response Maximize application scalability times and increases scalability Maximize speed of response Maximize Maximize information retrieval SECURITYcustomer loyalty Minimize unwanted breaches affects an application’s ability to Maximize customer confidence prevent unauthorized intrusions Maximize information protection
  18. 18. What organizational behaviors are mostfavorable for a successful SAM program?
  19. 19. Key Organizational Behaviors Trust but verify  Viable measurement and analysis  Answering the right questions  Having actions based on measurement & analysis Measure what can make a difference Measure to the audience needs (Goal / Question / Metrics)
  20. 20. Generating the Business Value Side of the Equation (Benefits) The business owns benefit calculations  IT should participate  Exception: projects solely improving internal IT Beware of subjectivity translating soft benefits Use probability and risk Increased Revenue Intangible Benefits Increased Profit Reduction in cost Internal benefits (people, processes, cash out)
  21. 21. Conditions that Favor Structural Measurement Low Fed up maturity managers Disciplined Mature processes High Quantitative maturity management
  22. 22. What is the relationshipbetween SAM and IT governance?
  23. 23. The Gap Between IT & Management Needs Financial Business/ ITOrganization Customer Vision & Business Process Governance Level Strategy Innovation / Growth Customer Focus CoBIT… 6σ Goals Goal Attainment Questions Answers IT GQM Metrics Measurement Governance Data Collection PSM Software Development & Maintenance (Source: Fraunhofer)
  24. 24. How Structural Measurement is Tied to IT Governance The Evolution of GovernanceGovern the Govern the Govern theDepartment Project Product Budget Schedule Structural headcount defects quality
  25. 25. How can measurement and analysis interact with estimating to provide improved visibilityand information to make the best management decisions regarding software?
  26. 26. What a Parametric Model can Tell You Firm Fixed Price? Feel lucky? What is likely to happen Understand the risk before you commit! 26
  27. 27. SAM Feeds Estimation and Provides Insight & Choice Size Effort S A SEER-SEM M Software O Estimation Schedule u Complexity Model t p u Risk t Maintainability/ s Defect Potential Reliability /Defects People, Process, Technology Maintenance Effort Required Constraints  Cost, schedule, risk  Should we update or Historical redevelop? Results (SAM Outputs)  What is the risk of continuing with software with increasing “technical debt”?
  28. 28. Looking at Maintenance and Technical Debt
  29. 29. Software Estimation Basic Model & Associated Metrics Effective Technology Cte Development Technology Legacy, Maintenance Maintenance/ Specifics & Block Change Constraints Development Effort K People Process and/or Process Block Changes As Redevelopment Staffa DefectsEffective complexity & Count (Qi Qr) D Constraints Software Size Size St Stakeholder Delivered ReuseDIT Requirements Development (Effective Se Software & Total St) Size Se Process (work units) Start Finish Calendar Time On-going Iterations of Effort (ACWP or Spent) Progress (BCWP or Earned Value) Defects (Qi Qr ) Growth (Sg)
  30. 30. Fundamental Metrics Size  Effort  AKA Volume, Mass  AKA Work, Labor  Units: Source Lines of Code  Units: Staff Months, Staff Hours (SLOC); Function Points (FP) Use Cases  Cost  New versus rework  AKA Budget, Money  COTS & Packages  Units: $, other currencies Effective Technology  Staffing  AKA Productivity Potential,  AKA Manpower Loading Efficiency  Units: FTE People  Units: none  Defects Time  AKA Reliability, Quality  AKA Duration, Schedule  Units: Defect Count  Units: Calendar Months, Calendar Weeks
  31. 31. Other Key Metrics Help Track Project Performance Track defect discovery and removal rates against expected rates Heath and Status Indicatorshows status and trends from the previous snapshotThresholds are user definable Increased defect reporting rate shows a worsening trend
  32. 32. When & Why to Collect Data When to Collect Why People Don’t Want to1. Up-front: when scoping new Provide Data project data from completed  They could be proven wrong projects  It could be used against them2. In-Process: During development for management,  Data often doesn’t exist to identify issues and progress  Even if processes dictate data3. Post Mortem: Upon requirements development completion to  If it exists it may not be clean improve corporate history  It may give away corporate repository productivity & bid strategy4. In Service: During maintenance to continue learning & improving
  33. 33. How can anyone provide a viable estimatesoftware development, software maintenance, or continuing innovation work and what use it is?
  34. 34. 10 Step System Estimation Process 20111. Establish Estimate Scope 10. Track Project Throughout Development2. Establish Technical 9. Document Estimates Baseline, Ground and Lessons Rules, Assumptions Learned 8. Generate a Project Plan 4. Refine Technical Baseline Into Estimable Components 6. Validate Business Case Costs & Benefits (go / no go) 4. Collect data / estimation inputs 6. Quantify Risks and Risk Analysis 5. Estimate Baseline Cost, Schedule, Affordability Value
  35. 35. Estimation Organizational Maturity V1.7 Manual effort Level 0 Informal or no estimating estimating without a process Level 1 Direct Task Estimation Spreadsheets Ad Hoc Process Simple model Formal (Size * Productivity) or Some Level 2 Sizing (e.g. function Direct Task Estimation informal SEER Use measurement & analysis Informal Process points) Robust Formalized Rigorous Parametric Level 3 Formal Sizing Parametric estimation (SEER) Estimate vs. actual capture Multiple Estimate Process measurement & analysis planning & Control Risk Management Repeatable process Robust Parametric Process Rigorous Level 4 Formal sizing Repeatable process parametric estimating (SEER) measurement & analysis estimation with tracking & control Risk Management improvement via lessons learned Robust Parametric Rigorous Continuous Level 5 Formal sizing Repeatable process parametric estimating (SEER) measurement & analysis estimation with tracking & control Risk Management process improvement Why should we care? Maturity is related to estimate viability…With better estimation process, projects more likely to be successful in execution
  36. 36. ESTIMATION & PLANNING: An Estimate Defined An estimate is the most knowledgeable statement you can make at a particular point in time regarding:  Effort / Cost  Schedule  Staffing  Risk  Reliability Estimates more precise with progress A WELL FORMED ESTIMATE IS A DISTRIBUTION 36
  37. 37. Avoid Surprises with Estimation Process & Tools Challenged projects  Would you still go forward if you knew  Schedule would be significantly longer?  Cost would be dramatically higher?  Probably: but perhaps more insight could identify mitigation  Plan functionality differently  Certainly you could notify stakeholders of real costs  Ensure staffing is appropriate for the constraints Failed Projects  Would you start a project you knew was unaffordable? Or if schedule was completely unrealistic?  If knowing up-front could you do something about it?  often better to kill project before it begins than waste resources & let the organization down
  38. 38. How can we determine which software projects with have the highest ROI?
  39. 39. Software Projects Must Return Business Value “Economics is primarily a science of choice… software economics should provide methods for analyzing the choices software projects must make.” Leon Levy “Base choices on those providing the maximum business value to the organization” Eli Goldratt Measurement and its uses such as estimating and defect analysis help this science of choice Some say business value is not our problem  While others generally need to perform benefit analysis  We need to build systems that optimize the business  Make IT part of the solution If IT & measurement don’t generate sufficient profit, money will go elsewhere
  40. 40. Software & IT Systems are about Business Value Cost Value
  41. 41. An ROI Analysis of an Upgrade: Software Analysis &Measurement Provides Valve Cost Can we do better? Will stakeholders tolerate a loss for 3 years? What is the risk?
  42. 42. Measurement Manifesto for Software & ITMeasurement Perspective Management & Stakeholder Perspective We measure to ultimately  Speak so I can understand produce business value to the  Give me actionable items organization  Don’t just give me problems… MEASURED Measurement Give me solutions should not ultimately be a cost  Help me make the best decisions The analysis of measurements so we can produce business produces decisions that value produce business value Measurement itself is not the answer…. Management decision making, better performance, quality improvements and better serving stakeholders is
  43. 43. L a nmo ea o t A T er r bu C S w w c ss f aec m w .a tot r.o w bo .a tot aec m lgc ss f r.o ww w fc b o .o c so q a t w . e o kc m/a tn u ly a iw w sd s aen t a tot ae w . ie h r.e/ ss f r l c w w w t ie.o O Q a t w . t r m/ n u ly wt c i