Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

No estimates - 10 new principles for testing

8,029 views

Published on

How #NoEstimates ideas can be applied and help direct the work of testers in software projects

Published in: Leadership & Management
  • Be the first to comment

No estimates - 10 new principles for testing

  1. 1. #NoEstimates 10 New Principles for Software Development (and testing!) Vasco Duarte @duarte_vasco Tweet at: #TAHelsinki
  2. 2. Vasco Duarte @duarte_vasco http://SoftwareDevelopmentToday.com http://bit.ly/vasco_slideshare Vasco.Duarte@oikosofy.com http://NoEstimatesBook.com
  3. 3. #NoEstimates is for you if…
  4. 4. Principle #1 Trust your process, or change your process
  5. 5. NumberofBugs Timeline Bug evolution in a non-agile project Open Closed Submit Development phase Desperately testing and fixing phase Waterfall
  6. 6. Principle #2 Shorten the feedback cycle
  7. 7. Can estimates be accurate?
  8. 8. Accidental complication
  9. 9. Cost/Duration = Essential Complication x Accidental Complication
  10. 10. Yet, some people still argue we can be good at estimating… Let’s look at the evidence
  11. 11. 80% Late or Failed Source: Software Estimation by Steve McConnell Reality: 80% is late or failed
  12. 12. Lets break that down Chaos Report 1995 For the same combined challenged and impaired projects, over one-third also experienced time overruns of 200 to 300%. The average overrun is 222% of the original time estimate. For large companies, the average is 230%; for medium companies, the average is 202%; and for small companies, the average is 239%. Time Overruns % of responses Under 20% 13.9% 21 – 50% 18.3% 51 – 100% 20.0% 101 – 200% 35.5% 201 – 400% 11.2% Over 400% 1.1% ~68% of projects 51% or more late!
  13. 13. Let’s continue to break that down Chaos Report 2009 Average cost overrun: 45% Average time overrun: 63% Chaos Report 2011 Average time overrun: 63%
  14. 14. Just in case you don’t like the CHAOS report
  15. 15. Source Gartner survey of project failure, 2012 Failure, means total failure, not just late
  16. 16. Of the large systems that are comple ted, about 66% experience schedule delays & cost overrun Source: “Project Management Tools and Software Failures and Successes” by Capers Jones Crosstalk, the Journal of Defense Software Engineering
  17. 17. Traditional projects: 53% failed or challenged Source: Project success survey by Scott Ambler
  18. 18. Agile project: 40% failed of challenged Source: Project success survey by Scott Ambler
  19. 19. 17 percent of large IT projects go so badly that they can threaten the very existence of the company Source : McKinsey & Company in conjunction with the University of Oxford Type of survey : Study on large scale IT Projects Date : 2012
  20. 20. More data coming, sign-up at NoEstimatesBook.com to receive this report when available
  21. 21. Principle #3 Believe the data, not the estimates
  22. 22. In 1986, Profs. S.D. Conte, H.E. Dunsmoir, and V.Y. Shen proposed that a good estimation approach should provide estimates that are within 25% of the actual results 75% of the time --Steve McConnel, Software Estimation: Demystifying the Black Art
  23. 23. Principle #4 Use alternatives to Estimate-driven decision making
  24. 24. Comparison of 17 projects ending between 2001 and 2003. (Average: 62%)
  25. 25. Testing for value, a story…
  26. 26. Principle #5 Test for value first, then test for functionality
  27. 27. Can #NoEstimates work in Real Companies?
  28. 28. http://j.mp/NoEstimates-Book
  29. 29. Principle #6 Estimation is waste, reduce it’s impact on your business
  30. 30. Principle #7 Measure progress only with validated, running software
  31. 31. In 1986, Profs. S.D. Conte, H.E. Dunsmoir, and V.Y. Shen proposed that a good estimation approach should provide estimates that are within 25% of the actual results 75% of the time --Steve McConnel, Software Estimation: Demystifying the Black Art
  32. 32. Source: Software Estimation by Steve McConnell “Good” estimates: 25% of estimated
  33. 33. I AM GOING TO GO AHEAD AND ASK YOU TO DELIVER 10 STORIES NEXT SPRINT...
  34. 34. WTF!!!!! !#%&!
  35. 35. 0 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 velocity Average= LCL UCL Target Actual, measured throughput over 21 sprints
  36. 36. The 4 Estimates Dysfunctions 1. Estimate Bargaining 2. Internal Politics 3. Blame Shifting 4. Late Changes 5. Sunk Cost fallacy
  37. 37. Principle #8 The system where you work has predictable outputs, learn to understand the system
  38. 38. Wow! But I have a business to run! Is there a way do better than that?
  39. 39. #NoEstimates delivers!
  40. 40. Counting Stories vs. Estimated Story Points Q: Which ”metric” is more accurate when compared to what actually happened in the project?
  41. 41. A long project 24Sprints
  42. 42. Which metric predicted most accurately the output of the whole project? a) After only the first 3 Sprints b) After only the first 5 Sprints
  43. 43. Disclaimer... This is only one project! Find 21 more at: http://bit.ly/NoEstimatesProjectsDB
  44. 44. After just 3 sprints # of Stories predictive powerStory Points predictive power The true output: 349,5 SPs completed The predicted output: 418 SPs completed +20% The true output: 228 Stories The predicted output: 220 Stories -4%!
  45. 45. After just 5 sprints # of Stories predictive powerStory Points predictive power The true output: 349,5 SPs completed The predicted output: 396 SPs completed +13% The true output: 228 Stories The predicted output: 220 Stories -4%!
  46. 46. Q: Which ”metric” is more accurate when compared to what actually happened in the project?
  47. 47. But there is more...
  48. 48. What difference does a Story Point make in a project that used both Story Points and #NoEstimates?
  49. 49. Next you will see the forecasted release date when using Story Points (values 1:3:5)
  50. 50. 68 71 71 71 71 71 71 72 72 72 73 73 0 3 7 7 9 11 12 13 20 20 22 23 0 10 20 30 40 50 60 70 80 90 100 1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7 Product Backlog Cumulative Flow Diagram Remaining Done Linear (Remaining) Linear (Done) Release on 20th October 2014
  51. 51. Next you will see the forecasted release date when using Story Points (values 1:2:3)
  52. 52. 48 51 51 51 51 51 51 52 52 52 53 53 0 2 5 5 7 8 9 10 15 15 17 18 0 10 20 30 40 50 60 70 80 1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7 Product Backlog Cumulative Flow Diagram Remaining Done Linear (Remaining) Linear (Done) Release on 14th October 2014
  53. 53. Next you will see the forecasted release date when #NoEstimates (or, all stories = 1 story point)
  54. 54. 28 31 31 31 31 31 31 32 32 32 33 33 0 1 3 3 5 5 6 7 10 10 12 13 0 10 20 30 40 50 60 1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7 Product Backlog Cumulative Flow Diagram Remaining Done Linear (Remaining) Linear (Done) Release on 29th September 2014
  55. 55. Conclusion...
  56. 56. All dates within 3 weeks of each other in a 38 to 42 week project (a margin of ~8%)
  57. 57. Data used with permission from Bill Hanlon at Microsoft ”At that point, I stopped thinking that estimating was important.” Bill Hanlon: http://bit.ly/BHanlon
  58. 58. In 1986, Profs. S.D. Conte, H.E. Dunsmoir, and V.Y. Shen proposed that a good estimation approach should provide estimates that are within 25% of the actual results 75% of the time --Steve McConnel, Software Estimation: Demystifying the Black Art In this presentation you have seen examples that far outperform what estimation specialists consider a ”good estimation”. In common they have one way to look at work: #NoEstimates
  59. 59. #NoEstimates testimonial The deviation between estimated and actual velocity would have been approximately 15% lower if we would have used #NoEstimates. We have analyzed data from 50 Sprints… …at no time the story point based estimation was better than #NoEstimates.
  60. 60. Principle #9 Don’t bet your company on poor track record methods, use methods with a proven track record aka: Hope is a bad management strategy
  61. 61. Carmen faces a very difficult situation…
  62. 62. The 7 Step Journey To NoEstimates 1. Start using Story Points 2. Stop estimating tasks 3. Limit the calendar duration for Stories and Features 4. Reduce the number of allowed estimates (say, 1,2,3 and 5 only) 5. Track your data 6. Use your data 7. Simply count the number of stories
  63. 63. http://j.mp/NoEstimates-Book
  64. 64. http://j.mp/NoEstimates-Book
  65. 65. http://j.mp/NoEstimates-Book
  66. 66. Principle #10 The transformation starts with you… Vasco Duarte Vasco.Duarte@oikosofy.com

×