Risk And Relevance 20080414ppt

Slide 2: IBM Software Group | Rational software Value is being created by software/IT Computers/Electronics Telecommunications Food Manufacturing Software/Information Chemical Manufacturing Hospitals Transportation Equipment $0.00 $100.00 $200.00 $300.00 $400.00 $500.00 $600.00 $700.00 $800.00 US Industry Revenue in $B US Census Bureau 2

Slide 3: IBM Software Group | Rational software Software provides huge value for all the other industries Computers/Electronics Telecommunications Food Manufacturing Software/Information Chemical Manufacturing Hospitals Transportation Equipment $0.00 $100.00 $200.00 $300.00 $400.00 $500.00 $600.00 $700.00 $800.00 US Industry Revenue in $B US Census Bureau 3

Slide 4: IBM Software Group | Rational software Software provides huge value for all the other industries Computers/Electronics Telecommunications This does not Manufacturing value Food include created by Software/Information back office/operations support Chemical Manufacturing Hospitals Transportation Equipment $0.00 $100.00 $200.00 $300.00 $400.00 $500.00 $600.00 $700.00 $800.00 US Industry Revenue in $B US Census Bureau 4

Slide 5: IBM Software Group | Rational software The concern is we are not creating value efficiently  For example, in a European Services Strategy Unit 2007 report studied105 outsourced public sector ICT projects with significant cost overruns, delays and terminations, total value of contracts is £29.5 billion  Results  Cost overruns totaled £9.0 billion  57% of contracts experienced cost overruns  The average percentage cost overrun is 30.5%  33% of contracts suffered major delays  30% of contracts were terminated  12.5% of Strategic Service Delivery Partnerships have failed How much value was delivered by all the successful efforts? 5

Slide 6: IBM Software Group | Rational software How should we meet the challenge of efficiently delivering value?  Focusing on avoiding overruns?  Leads to risk avoidance  limits opportunity for creating value, to be relevant  Rather, lets discuss how to apply engineering discipline to creating value 6

Slide 7: IBM Software Group | Rational software History has shown we do not get the estimates ‘right’ Boehm COCOMO Effort Estimation Accuracy ISBSG Effort Estimation Accuracy 1000.00 1000.00 Effort (Staff-months) 100.00 100.00 Effort (LM) Rone Effort Estimation Accuracy 10.00 10.00 1000.00 1.00 1.00 100 1000 10000 100000 1.00 10.00 100.00 1000.00 10000.00 100.00 Size (SLOC) Effort (SM) Size (FP) 10.00 Jones Effort Estimation Accuracy Basili & Bailey Effort Estimation Accuracy 1000.00 1000.00 1.00 1.00 10.00 100.00 1000.00 10000.00 Size (SLOC) 100.00 100.00 Effort (LM) Effort (LM) 10.00 10.00 1.00 1.00 1.00 10.00 100.00 1000.00 10000.00 100 1000 10000 100000 1000000 Size (FP) Size (SLOC) From George Stark, Paul Oman, “A comparison of parametric Software Estimation Models using real project data”, in press 7

Slide 8: IBM Software Group | Rational software History has shown we do not get the estimates ‘right’ Boehm COCOMO Effort Estimation Accuracy ISBSG Effort Estimation Accuracy 1000.00 1000.00 Effort (Staff-months) 100.00 100.00 Effort (LM) Rone Effort Estimation Accuracy 10.00 10.00 Variance dominates 1000.00 1.00 1.00 100 1000 10000 100000 1.00 10.00 100.00 1000.00 10000.00 100.00 Size (SLOC) Effort (SM) Size (FP) 10.00 Jones Effort Estimation Accuracy Basili & Bailey Effort Estimation Accuracy 1000.00 Maybe there is a reason 1.00 1.00 10.00 100.00 1000.00 1000.00 10000.00 Size (SLOC) 100.00 100.00 Effort (LM) Effort (LM) 10.00 10.00 1.00 1.00 1.00 10.00 100.00 1000.00 10000.00 100 1000 10000 100000 1000000 Size (FP) Size (SLOC) From George Stark, Paul Oman, “A comparison of parametric Software Estimation Models using real project data”, in press, 2007 8

Slide 9: IBM Software Group | Rational software Eternal wisdom “To measure is to know. If you can not measure it, you can not improve it.” - Lord Kelvin “We should be guided by theory, not by numbers.” - W. Edward Deming “Make everything as simple as possible, but not simpler.” - Albert Einstein “There are risks and costs to a program of action. But they are far less than the long-range risks and costs of comfortable inaction.” - John F. Kennedy 9

Slide 10: IBM Software Group | Rational software The value conversation can vary  Future business efficiency  Investing now to receive future savings, capacity, responsiveness  Operational risk avoidance  Investing now to avoid future business/IT risks, e.g., security, privacy, continuity …  Business impact  Investing now to affect future top line Product Innovation  Treacy Framework Your Business?  Mission capabilities Operational Client Efficiency Intimacy 10 10

Slide 11: IBM Software Group | Rational software The value conversation can vary  Future business efficiency  Investing now to receive future savings, capacity, responsiveness  Operational risk avoidance  Investing now to avoid future business/IT risks, e.g., security, privacy, continuity …  Business impact All can be monetized Product Innovation  Investing now to affect future top line Your  Treacy Framework Business?  Mission capabilities Operational Client Efficiency Intimacy 11 11

Slide 12: IBM Software Group | Rational software Just enough theory  Money is how one measures value – need to monetize  The future cost, value are uncertain Cost and value are random variables Need to consider current Net Present Value of program  Results need only to be accurate enough to drive desired behavior: Assuming managed risk to deliver value 12 12

Slide 13: IBM Software Group | Rational software How much is an incomplete development program worth? Value Conventional Wisdom 0 6 Ship date  Conventional wisdom is the accountants answer.  provides no opportunity for ongoing value management  Can only discuss cost, not value Little opportunity to show relevance 13 13

Slide 14: IBM Software Group | Rational software Imagine (if you will) you could sell your program  The buyer would spend money now to get the right to invest in completing the program – like a call option  How would one reason about the fair price? The buyer, reasoning like an investor, would like to know  Expected cost to complete  Expected value received  The risk This is what the economists call “incomplete market reasoning” 14 14

Slide 15: IBM Software Group | Rational software The Net Present Value of a development program n Ri p Mk m Ej NPV      E0 i 1 (1  rR ) k 1 (1  rM ) j 1 (1  rE ) i k j  RI = revenue, benefits stream  Ej = development expense stream  Mk = maintenance expenses stream  The rR, rM, rE are discount rates accounting for the time value of money NPV, Ri, Ej, Mk are all random variables 15 15

Slide 16: IBM Software Group | Rational software The Net Present Value of a development program n Ri p Mk m Ej NPV      E0 i 1 (1  rR ) k 1 (1  rm ) j 1 (1  rE ) i k j  RI = revenue, benefits stream  Ej = development expense stream So, how can value be created?  Mk = maintenance expenses stream NPV, Ri, Ej, Mk are all random variables 16 16

Slide 17: IBM Software Group | Rational software Expense risk: Imagine you have 12 months to deliver a business critical system  Your estimators tell you it will be done in 11 months  What do you do with the information?  Rest easy, believing there is no risk? 0 6 12 17 17

Slide 18: IBM Software Group | Rational software Maybe you realize that program parameters (cost, schedule, effort, quality, …) are random variables  Area under curve describes probability of measurement falling in range 0.45 0.4 Likelihood of actual value 0.35 falling in range 0.3 is area under curve 0.25 0.2 0.15 0.1 0.05 0 00 0 0 20 60 0 40 80 60 00 40 80 60 0 20 20 4 8 0 0 0. 1. 1. 2. 2. 3. 4. 4. 4. 5. 0. 0. 2. 3. 5. 6. 18 18

Slide 19: IBM Software Group | Rational software Imagine you have 12 months to deliver a business critical systems  So you ask for the distribution and discover there is some uncertainty 0 6 12 19 19

Slide 20: IBM Software Group | Rational software Imagine you have 12 months to deliver a business critical systems  In fact there is less than 50% chance of making the date 48% 0 6 12 20 20

Slide 21: IBM Software Group | Rational software Then what?  Move out the date to improve likelihood of shipping? 95% 0 6 12 15 21 21

Slide 22: IBM Software Group | Rational software Then what?  Or move in the estimate by sacrificing quality or content? 95% 0 6 12 22 22

Slide 23: IBM Software Group | Rational software The estimate variance reflects current state of understanding  Source  Lack of knowledge 0.45  Lack of confidence 0.4 0.35 0.3  Reduction of variance reflects 0.25  Increased knowledge about 0.2  Client needs 0.15 0.1  Technology 0.05  Team capability 0  Good decisions 00 0 40 80 20 00 40 20 60 00 60 00 40 80 60 20 8 0. 4. 4. 2. 3. 3. 4. 5. 6. 0. 0. 1. 1. 2. 2. 5. 23 23

Slide 24: IBM Software Group | Rational software Then what?  Determine the source of the variance  Over the project lifecycle, reduce the variance to improve likelihood of shipping 90% 0 6 12 24 24

Slide 25: IBM Software Group | Rational software Then what?  Over the lifecycle, reduce the variance further to improve likelihood of shipping 95% 0 6 12 25 25

Slide 26: IBM Software Group | Rational software To summarize so far n Ri p Mk m Ej NPV      E0 i 1 (1  rR ) k 1 (1  rm ) i k j 1 (1  rE ) j  Cost, schedule are random variables  Variance is the measure of risk  Progress measured by reduction of risk Project Lifecycle Phases Variance in Cost, Schedule… Inception Elaboration Construction Transition/Maintenance 80% reduction 96% reduction High Variance Medium Low Variance Variance Time 26 26

Slide 27: IBM Software Group | Rational software Benefits: Delivered value is also a distribution n Ri p Mk m Ej NPV      E0 i 1 (1  rR ) k 1 (1  rm ) i k j 1 (1  rE ) j Reuse opportunities Shortfall in stakeholder needs 0 Planned $Value Value is improved by increasing upside variance 27 27

Slide 28: IBM Software Group | Rational software Simple inputs  Expert opinion  Just guess using expert opinion  Use optimistic, pessimistic, nominal assumptions in other models  Function point, Use case point, Cocomo  Enter into triangular distributions  Historical data  Use other distributions if you have them 28 28

Slide 29: IBM Software Group | Rational software Model output Mean 4,324.45 Standard Deviation 2,738.59 Variance 7,499,853.36 0% -4,836.65 10% 818.52 20% 1,959.10 30% 2,804.66 40% 3,541.42 50% 4,249.81 60% 4,972.52 70% 5,739.09 80% 6,661.06  In this case 90% 7,928.97  Expected NPV = $4,324K 100% 14,923.77  No chance > $14,924K, < ($4,837k) 29 29

Slide 30: IBM Software Group | Rational software No initial risk – little opportunity to add value, stay relevant  NPV increases when you invest in  Improving likelihood of delivery (reduce variance of costs)  Improving range of value at delivery by reuse of architecture, services … (increase upside variance of benefits) NPV/Options Theory Value Conventional Wisdom 0 6 Ship date Project Lifecycle Phases … Inception Elaboration Construction Transition/Maintenance 80% reduction Variance in Cost, Schedule 96% reduction 0 Planned $Value High Variance Medium Low Variance Variance Time 30 30

Slide 31: IBM Software Group | Rational software So, what can you do?  Steer your project to success  Measure, analyze and embrace estimation variance  Tailor governance to your risk/value context  Determine if governance approach is paying off  Getting more or less value from agile, non-agile projects?  Invest with discipline  Detect early when risks are not paying off – abandon projects that are not delivering  Have improved stakeholder conversations  Risk and value with funders  Conveys more complete information for better collaboration  Inputs/assumptions with program staff  Asking experts for likely and ranges elicits more complete information – Forces needed discipline – Wide ranges engenders discussion on where to focus efforts 31 31

Slide 32: IBM Software Group | Rational software Steer your project to success Each iteration should reduce variance and increase NPV Uncertainty in Stakeholder Satisfaction Space Initial Planned Path Actual Path Uncertainty in delivery Initial Planned Content 32 32

Slide 33: IBM Software Group | Rational software Tailor the governance solution  Kind of value for organization processes  Kind of projects  Full variance – foster discovery and learning  Medium Variance – architecture alignment, lean methods  Low variance – focus on automation, cost Variance in Cost, Schedule… Inception Elaboration Construction Transition/Maintenance New platform architecture New capability Service Delivery with existing architecture Code Maintenance High Medium Low Variance Variance Variance Much Some Little discovery discovery discovery 33 33

Slide 34: IBM Software Group | Rational software Tailoring governance is a process  Reassign  Agree with stakeholders on responsibilities, value metrics authorities to improve productivity Plan  Choose internal measures Set Objective (klocs, churn …) that contribute to business value and manage risk Analyze Design Implement Assess Governance Lifecycle Evaluate Deploy  Tailor organization decision rights, responsibilities to risk areas Monitor Control  Collect measures  Compute  Value Manage  Productivity: Value per programmer months 34 34

Slide 35: IBM Software Group | Rational software Adjust Governance Solution for each risk area Adjust Governance Solution for each risk area  Focusing on Inception Elaboration Construction Transition/Maintenance artifacts, not New process. Platform Architectural  Constrain behavior Newalignment Variance in Cost, Schedule… creating uncertainty Capability onreduces existing uncertainty due platform to diseconomy of scale Automate, enforce Maintenance processes to and small reduce change uncertainty requests High Variance Medium Variance Low Variance Time 35 35

Slide 36: IBM Software Group | Rational software Disciplined Investment: Apply quantitative portfolio analysis, not only scorecards 1200 Project 10 1000 800 Project 3 Project 6 Project 2 600 Project 4 Cost Value Project 5 400 Project 7 Project 9 Project 1 200 Project 8 0 -2 0 2 4 6 8 10 12 -200 Liklihood of Success 36 36

Slide 37: IBM Software Group | Rational software Pilots anyone?  We are creating new tools applying these ideas to  Instrument iterative development  Measure and manage value of development  Measure and manage architecture alignment  Looking for pilot programs, let us know if you are interested  Seating is limited! 37 37

Slide 38: IBM Software Group | Rational software Some final thoughts Staying relevant requires smart risk taking Value can only be reliably delivered if it measured and managed Value delayed is value denied In the end, these techniques do allow us to “Take risks and add value” 38 38

Slide 39: IBM Software Group | Rational software For more information  Contact Clay Williams, manager of the Governance Science Research Group at IBM Watson Research  clayw@us.ibm.com, 914-784-7457 39

Slide 40: IBM Software Group | Rational software Learn more at:  IBM Rational software  Architecture management  IBM Rational Software Delivery Platform  Rational trial downloads  Process and portfolio management  Leading Innovation Website  Change and release management  IBM Rational TV  Quality management  IBM Rational Business Partners © Copyright IBM Corporation 2008. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, the on-demand business logo, Rational, the Rational logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others. 40 40

Slideshare.net (beta)