Risk And Relevance 20080414ppt

  • 493 views
Uploaded on

PROMISE Workshop 2008 - Keynote 1

PROMISE Workshop 2008 - Keynote 1

More in: Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
493
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
15
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Risk and Relevance Dr. Murray Cantor Distinguished Engineer, IBM Rational software [email_address]
  • 2. Value is being created by software/IT Transportation Equipment Hospitals Chemical Manufacturing Software/Information Food Manufacturing Telecommunications Computers/Electronics $0.00 $100.00 $200.00 $300.00 $400.00 $500.00 $600.00 $700.00 $800.00 US Industry Revenue in $B US Census Bureau
  • 3. Software provides huge value for all the other industries US Industry Revenue in $B US Census Bureau Transportation Equipment Hospitals Chemical Manufacturing Software/Information Food Manufacturing Telecommunications Computers/Electronics $0.00 $100.00 $200.00 $300.00 $400.00 $500.00 $600.00 $700.00 $800.00
  • 4. Software provides huge value for all the other industries US Industry Revenue in $B US Census Bureau Transportation Equipment Hospitals Chemical Manufacturing Software/Information Food Manufacturing Telecommunications Computers/Electronics $0.00 $100.00 $200.00 $300.00 $400.00 $500.00 $600.00 $700.00 $800.00 This does not include value created by back office/operations support
  • 5. The concern is we are not creating value efficiently
    • For example, in a European Services Strategy Unit 2007 report studied105 outsourced public sector ICT projects with significant cost overruns, delays and terminations, total value of contracts is £29.5 billion
    • Results
      • Cost overruns totaled £9.0 billion
      • 57% of contracts experienced cost overruns
      • The average percentage cost overrun is 30.5%
      • 33% of contracts suffered major delays
      • 30% of contracts were terminated
      • 12.5% of Strategic Service Delivery Partnerships have failed
    How much value was delivered by all the successful efforts?
  • 6. How should we meet the challenge of efficiently delivering value?
    • Focusing on avoiding overruns?
      • Leads to risk avoidance
      • limits opportunity for creating value, to be relevant
    • Rather, lets discuss how to apply engineering discipline to creating value
  • 7. History has shown we do not get the estimates ‘right’ From George Stark, Paul Oman, “A comparison of parametric Software Estimation Models using real project data”, in press
  • 8. History has shown we do not get the estimates ‘right’ From George Stark, Paul Oman, “A comparison of parametric Software Estimation Models using real project data”, in press, 2007 Variance dominates Maybe there is a reason
  • 9. Eternal wisdom “ There are risks and costs to a program of action. But they are far less than the long-range risks and costs of comfortable inaction.” - John F. Kennedy “ Make everything as simple as possible, but not simpler.” - Albert Einstein “ We should be guided by theory, not by numbers.” - W. Edward Deming “ To measure is to know. If you can not measure it, you can not improve it.” - Lord Kelvin
  • 10. The value conversation can vary
    • Future business efficiency
      • Investing now to receive future savings, capacity, responsiveness
    • Operational risk avoidance
      • Investing now to avoid future business/IT risks, e.g., security, privacy, continuity …
    • Business impact
      • Investing now to affect future top line
        • Treacy Framework
        • Mission capabilities
    Product Innovation Operational Efficiency Client Intimacy Your Business?
  • 11. The value conversation can vary
    • Future business efficiency
      • Investing now to receive future savings, capacity, responsiveness
    • Operational risk avoidance
      • Investing now to avoid future business/IT risks, e.g., security, privacy, continuity …
    • Business impact
      • Investing now to affect future top line
        • Treacy Framework
        • Mission capabilities
    All can be monetized Product Innovation Operational Efficiency Client Intimacy Your Business?
  • 12. Just enough theory
    • Money is how one measures value – need to monetize
    • The future cost, value are uncertain
      • Cost and value are random variables
      • Need to consider current Net Present Value of program
    • Results need only to be accurate enough to drive desired behavior: Assuming managed risk to deliver value
  • 13. How much is an incomplete development program worth?
    • Conventional wisdom is the accountants answer.
      • provides no opportunity for ongoing value management
    • Can only discuss cost, not value
    0 6 Ship date Conventional Wisdom Value Little opportunity to show relevance
  • 14. Imagine (if you will) you could sell your program
    • The buyer would spend money now to get the right to invest in completing the program – like a call option
    • How would one reason about the fair price?
      • The buyer, reasoning like an investor, would like to know
        • Expected cost to complete
        • Expected value received
        • The risk
    This is what the economists call “incomplete market reasoning”
  • 15. The Net Present Value of a development program
    • R I = revenue, benefits stream
    • E j = development expense stream
    • M k = maintenance expenses stream
    • The r R , r M , r E are discount rates accounting for the time value of money
    NPV, R i , E j , M k are all random variables
  • 16. The Net Present Value of a development program
    • R I = revenue, benefits stream
    • E j = development expense stream
    • M k = maintenance expenses stream
    NPV, R i , E j , M k are all random variables So, how can value be created?
  • 17. Expense risk: Imagine you have 12 months to deliver a business critical system
    • Your estimators tell you it will be done in 11 months
    • What do you do with the information?
      • Rest easy, believing there is no risk?
    0 6 12
  • 18. Maybe you realize that program parameters (cost, schedule, effort, quality, …) are random variables
    • Area under curve describes probability of measurement falling in range
    0.80 1.20 1.60 2.00 2.40 2.80 3.20 3.60 4.00 4.40 4.80 5.20 5.60 6.00 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0 0.00 0.40 Likelihood of actual value falling in range is area under curve
  • 19. Imagine you have 12 months to deliver a business critical systems
    • So you ask for the distribution and discover there is some uncertainty
    0 6 12
  • 20. Imagine you have 12 months to deliver a business critical systems
    • In fact there is less than 50% chance of making the date
    48% 0 6 12
  • 21. Then what?
    • Move out the date to improve likelihood of shipping?
    95% 0 6 12 15
  • 22. Then what?
    • Or move in the estimate by sacrificing quality or content?
    95% 0 6 12
  • 23. The estimate variance reflects current state of understanding
    • Source
      • Lack of knowledge
      • Lack of confidence
    • Reduction of variance reflects
      • Increased knowledge about
        • Client needs
        • Technology
        • Team capability
      • Good decisions
    0.80 1.20 1.60 2.00 2.40 2.80 3.20 3.60 4.00 4.40 4.80 5.20 5.60 6.00 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0 0.00 0.40
  • 24. Then what?
    • Determine the source of the variance
    • Over the project lifecycle, reduce the variance to improve likelihood of shipping
    90% 0 6 12
  • 25. Then what?
    • Over the lifecycle, reduce the variance further to improve likelihood of shipping
    95% 0 6 12
  • 26. To summarize so far
    • Cost, schedule are random variables
    • Variance is the measure of risk
    • Progress measured by reduction of risk
    Medium Variance Time Variance in Cost, Schedule… Project Lifecycle Phases Inception Elaboration Construction Transition/Maintenance High Variance Low Variance 80% reduction 96% reduction
  • 27. Benefits: Delivered value is also a distribution Shortfall in stakeholder needs Value is improved by increasing upside variance 0 Planned $Value Reuse opportunities
  • 28. Simple inputs
    • Expert opinion
      • Just guess using expert opinion
      • Use optimistic, pessimistic, nominal assumptions in other models
        • Function point, Use case point, Cocomo
      • Enter into triangular distributions
    • Historical data
      • Use other distributions if you have them
  • 29. Model output
    • In this case
      • Expected NPV = $4,324K
      • No chance > $14,924K, < ($4,837k)
    Mean 4,324.45 Standard Deviation 2,738.59 Variance 7,499,853.36 0% -4,836.65 10% 818.52 20% 1,959.10 30% 2,804.66 40% 3,541.42 50% 4,249.81 60% 4,972.52 70% 5,739.09 80% 6,661.06 90% 7,928.97 100% 14,923.77
  • 30. No initial risk – little opportunity to add value, stay relevant
    • NPV increases when you invest in
      • Improving likelihood of delivery (reduce variance of costs)
      • Improving range of value at delivery by reuse of architecture, services … (increase upside variance of benefits)
    0 Planned $Value 0 Planned $Value 0 6 Ship date Conventional Wisdom Value NPV/Options Theory
  • 31. So, what can you do?
    • Steer your project to success
      • Measure, analyze and embrace estimation variance
    • Tailor governance to your risk/value context
      • Determine if governance approach is paying off
        • Getting more or less value from agile, non-agile projects?
    • Invest with discipline
      • Detect early when risks are not paying off – abandon projects that are not delivering
    • Have improved stakeholder conversations
      • Risk and value with funders
        • Conveys more complete information for better collaboration
      • Inputs/assumptions with program staff
        • Asking experts for likely and ranges elicits more complete information
          • Forces needed discipline
          • Wide ranges engenders discussion on where to focus efforts
  • 32. Steer your project to success Actual Path Uncertainty in Stakeholder Satisfaction Space Initial Planned Content Initial Planned Path Uncertainty in delivery Each iteration should reduce variance and increase NPV
  • 33. Tailor the governance solution
    • Kind of value for organization processes
    • Kind of projects
      • Full variance – foster discovery and learning
      • Medium Variance – architecture alignment, lean methods
      • Low variance – focus on automation, cost
    Medium Variance Little discovery Variance in Cost, Schedule… Inception Elaboration Construction Transition/Maintenance High Variance Low Variance New platform architecture Service Delivery Code Maintenance New capability with existing architecture Some discovery Much discovery
  • 34. Tailoring governance is a process
    • Agree with stakeholders on value metrics
    • Choose internal measures (klocs, churn …) that contribute to business value and manage risk
    • Collect measures
    • Compute
      • Value
      • Productivity: Value per programmer months
    • Reassign responsibilities, authorities to improve productivity
    • Tailor organization decision rights, responsibilities to risk areas
    Assess Implement Manage Plan Governance Lifecycle Deploy Control Monitor Analyze Evaluate Design Set Objective
  • 35. Adjust Governance Solution for each risk area Medium Variance Time Variance in Cost, Schedule… Adjust Governance Solution for each risk area Inception Elaboration Construction Transition/Maintenance High Variance Low Variance New Platform New Capability on existing platform Maintenance and small change requests
    • Focusing on artifacts, not process.
    • Constrain behavior creating uncertainty
    Architectural alignment reduces uncertainty due to diseconomy of scale Automate, enforce processes to reduce uncertainty
  • 36. Disciplined Investment: Apply quantitative portfolio analysis, not only scorecards Cost Project 1 Project 2 Project 3 Project 4 Project 5 Project 7 Project 8 Project 9 Project 6 Project 10
  • 37. Pilots anyone?
    • We are creating new tools applying these ideas to
      • Instrument iterative development
      • Measure and manage value of development
      • Measure and manage architecture alignment
    • Looking for pilot programs, let us know if you are interested
      • Seating is limited!
  • 38. Some final thoughts Staying relevant requires smart risk taking Value can only be reliably delivered if it measured and managed Value delayed is value denied In the end, these techniques do allow us to “ Take risks and add value”
  • 39. For more information
    • Contact Clay Williams, manager of the Governance Science Research Group at IBM Watson Research
    • [email_address] , 914-784-7457
  • 40. © Copyright IBM Corporation 2008. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, the on-demand business logo, Rational, the Rational logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
    • Learn more at:
    • IBM Rational software
    • IBM Rational Software Delivery Platform
    • Process and portfolio management
    • Change and release management
    • Quality management
    • Architecture management
    • Rational trial downloads
    • Leading Innovation Website
    • IBM Rational TV
    • IBM Rational Business Partners