Building a more accurate burndown   Using Range Estimation in Scrum Agile 2010 Conference August 2010 Arin Sime 434 996 5226 [email_address]
Pitfalls of traditional estimation techniques
How long does it take you to get to work? traffic optimistic every day? method of travel
68% http://www1.standishgroup.com/newsroom/chaos_2009.php
A little about me… Senior Consultant, OpenSource Connections in Charlottesville, Virginia Masters in Management of I.T., University of Virginia, McIntire School of Commerce We tweaked our Scrum process to incorporate Range Estimation based on my studies at Uva Please take the Estimation Survey:  http://www.surveymonkey.com/s/SWNNYQJ
The root of all estimation evil:  Single point estimates Chart taken from:  Software Estimation , Steve McConnell, Figure 1-1, p6 “ A single-point estimate is usually a target masquerading as an estimate.” -Steve McConnell
A realistic estimate distribution Chart taken from:  Software Estimation , Steve McConnell, Figure 1-3, p8 “ There is a limit to how well a project can go but no limit to how many problems can occur.” -Steve McConnell Nominal Outcome (50/50 estimate)
Reasons we are wrong so often Different information Different methods Psychological Biases The Expert Problem
Bias in Estimation Imagine this scenario: “ Can you build me that CMS website in 2 weeks?” How would you respond?  What estimate would you give?
Bias in Estimation By supplying my own estimate (or desire) in my question, I have anchored your response. This is called  “The anchoring or framing trap” “ Because anchors can establish the terms on which a decision will be made, they are often used as a bargaining tactic by savvy negotiators.”  From “The Hidden Traps in Decision Making”  from Harvard Business Review, 1998, John Hammond, Ralph L. Keeney, and Howard Raiffa
You’re not as good as you think “ The Expert Problem” Experts consistently underestimate their margins of error, and discount the reasons they were wrong in the past. Excuses for past mistakes: You were playing a different game  Invoke the outlier “ Almost right” defense The Black Swan:  The impact of the Highly Improbable ,  by Nassim Nicholas Taleb, 2007, Chapter 10: The Scandal of Prediction
The best protection “ The best protection against all psychological traps – in isolation or in combination – is awareness.”  From “The Hidden Traps in Decision Making”  from Harvard Business Review, 1998, John Hammond, Ralph L. Keeney, and Howard Raiffa
Agile estimation techniques
How agile already avoids pitfalls Encourages team airing of estimates Done before assignment of tasks Scrum poker
How agile already avoids pitfalls Separates story from time units, more relative Story Points & Velocity Image from:  http://leadinganswers.typepad.com/leading_answers/2007/09/agile-exception.html
Agile and Scrum can run into other pitfalls though…
Potential pitfalls: Single-point estimates What about Risk? Implies a set delivery of features Story points are hard to explain
Better accuracy using range estimation
The Cone of Uncertainty http://www.construx.com/Page.aspx?hid=1648
Range estimation … Recognizes uncertainty Alleviates our tendency towards optimism Incorporates risk Allows for better financial projections Better informs our bosses and clients
Incorporating range estimation into Scrum
Incorporating range estimation into Scrum Team originally estimated 108 hours Range estimate went from 114-245 hours. Note the single point was a low estimate! They were able to finish original tasks a little early
Range estimation in Scrum Poker It’s very simple – just hold two cards instead of one! The same rules apply about creating discussion between low and high estimators, but you might resolve them differently...
On the high end Range estimation in Scrum Poker On the low end On the high end The likely discussion: Hey Orange, why do you say “2”?  Yellow and Blue both say “5”. Likely Outcome:  3 or 5 Middle of the road
Range estimation in Scrum Poker Still middle of the road, but Green recognizes some risk Orange sees this as really easy  Blue sees this as more complicated The likely discussion: Orange and Blue need to compare their visions for this task. Likely Outcome:  8-13? Red and Blue no longer agree:  Red is confused or sees big risks
Using ranges in your task list
Using ranges in your task list Enter Low/High =(Lo*0.33)+(Hi*0.67) Sums of Lo, Hi, 2/3; then trend them to zero update daily
Using ranges in your burndown
Ranges help to highlight obstacles and know when to cancel an iteration
We were able to improve on the next iteration, but it was still hard
Ranges help reinforce obstacles Obstacle removed
Why 2/3? Because it is both simple and pessimistic PERT does a similar thing: Expected = [BestCase + (4*MostLikely) + WorstCase] / 6 Source on PERT:  Software Estimation , Steve McConnell, p109
Using ranges to better communicate
Using range estimation to communicate risk Size of your range communicates the risk of your task May encourage you to break up tasks, or better define them. Scrum is all about better communication with the customer – so are ranges
How long? Um… 2 days 4 days Do you know your fudge factor? You Your Boss Big Boss
How long? 2-4 days 2-4 days Ranges help you control your fudge factor You Your Boss Big Boss
Another example:  Use ranges to better empower your boss or client You Your Boss Big Boss
Perfect – Do it! How long? How much for X? GRRR Umm….. You Your Boss Big Boss 2 days Actually … 4 days 4 days later… 2 days * rate Budget Left:  2 days
Instead…. You Your Boss Big Boss
No thx, do something easier How long? How much for X? YES! You Your Boss Big Boss 2-4 days Done! 2 days later… 2-4 days * rate Budget Left:  2 days
Potential pitfalls of range estimation
Potential pitfalls of range estimation Really Wide Ranges Not everything can take  2 – 200 hours or you lose all credibility
Potential pitfalls of range estimation Bosses who don’t get it You’re going to have to sell them on how your estimates will improve their decision  making ability.
Potential pitfalls of range estimation Pushed back deadlines Ranges are not an excuse to always miss deadlines.  But they do make it less of a surprise, and encourage you to be more cautious.
Potential pitfalls of range estimation Is 2/3 the new single-point? Be careful not to just start treating the 2/3 calculated estimate, use the ranges.
Further Reading
Questions? Arin Sime 434 996 5226 [email_address] Twitter.com/ArinSime

Range estimation in Scrum

  • 1.
    Building a moreaccurate burndown Using Range Estimation in Scrum Agile 2010 Conference August 2010 Arin Sime 434 996 5226 [email_address]
  • 2.
    Pitfalls of traditionalestimation techniques
  • 3.
    How long doesit take you to get to work? traffic optimistic every day? method of travel
  • 4.
  • 5.
    A little aboutme… Senior Consultant, OpenSource Connections in Charlottesville, Virginia Masters in Management of I.T., University of Virginia, McIntire School of Commerce We tweaked our Scrum process to incorporate Range Estimation based on my studies at Uva Please take the Estimation Survey: http://www.surveymonkey.com/s/SWNNYQJ
  • 6.
    The root ofall estimation evil: Single point estimates Chart taken from: Software Estimation , Steve McConnell, Figure 1-1, p6 “ A single-point estimate is usually a target masquerading as an estimate.” -Steve McConnell
  • 7.
    A realistic estimatedistribution Chart taken from: Software Estimation , Steve McConnell, Figure 1-3, p8 “ There is a limit to how well a project can go but no limit to how many problems can occur.” -Steve McConnell Nominal Outcome (50/50 estimate)
  • 8.
    Reasons we arewrong so often Different information Different methods Psychological Biases The Expert Problem
  • 9.
    Bias in EstimationImagine this scenario: “ Can you build me that CMS website in 2 weeks?” How would you respond? What estimate would you give?
  • 10.
    Bias in EstimationBy supplying my own estimate (or desire) in my question, I have anchored your response. This is called “The anchoring or framing trap” “ Because anchors can establish the terms on which a decision will be made, they are often used as a bargaining tactic by savvy negotiators.” From “The Hidden Traps in Decision Making” from Harvard Business Review, 1998, John Hammond, Ralph L. Keeney, and Howard Raiffa
  • 11.
    You’re not asgood as you think “ The Expert Problem” Experts consistently underestimate their margins of error, and discount the reasons they were wrong in the past. Excuses for past mistakes: You were playing a different game Invoke the outlier “ Almost right” defense The Black Swan: The impact of the Highly Improbable , by Nassim Nicholas Taleb, 2007, Chapter 10: The Scandal of Prediction
  • 12.
    The best protection“ The best protection against all psychological traps – in isolation or in combination – is awareness.” From “The Hidden Traps in Decision Making” from Harvard Business Review, 1998, John Hammond, Ralph L. Keeney, and Howard Raiffa
  • 13.
  • 14.
    How agile alreadyavoids pitfalls Encourages team airing of estimates Done before assignment of tasks Scrum poker
  • 15.
    How agile alreadyavoids pitfalls Separates story from time units, more relative Story Points & Velocity Image from: http://leadinganswers.typepad.com/leading_answers/2007/09/agile-exception.html
  • 16.
    Agile and Scrumcan run into other pitfalls though…
  • 17.
    Potential pitfalls: Single-pointestimates What about Risk? Implies a set delivery of features Story points are hard to explain
  • 18.
    Better accuracy usingrange estimation
  • 19.
    The Cone ofUncertainty http://www.construx.com/Page.aspx?hid=1648
  • 20.
    Range estimation …Recognizes uncertainty Alleviates our tendency towards optimism Incorporates risk Allows for better financial projections Better informs our bosses and clients
  • 21.
  • 22.
    Incorporating range estimationinto Scrum Team originally estimated 108 hours Range estimate went from 114-245 hours. Note the single point was a low estimate! They were able to finish original tasks a little early
  • 23.
    Range estimation inScrum Poker It’s very simple – just hold two cards instead of one! The same rules apply about creating discussion between low and high estimators, but you might resolve them differently...
  • 24.
    On the highend Range estimation in Scrum Poker On the low end On the high end The likely discussion: Hey Orange, why do you say “2”? Yellow and Blue both say “5”. Likely Outcome: 3 or 5 Middle of the road
  • 25.
    Range estimation inScrum Poker Still middle of the road, but Green recognizes some risk Orange sees this as really easy Blue sees this as more complicated The likely discussion: Orange and Blue need to compare their visions for this task. Likely Outcome: 8-13? Red and Blue no longer agree: Red is confused or sees big risks
  • 26.
    Using ranges inyour task list
  • 27.
    Using ranges inyour task list Enter Low/High =(Lo*0.33)+(Hi*0.67) Sums of Lo, Hi, 2/3; then trend them to zero update daily
  • 28.
    Using ranges inyour burndown
  • 29.
    Ranges help tohighlight obstacles and know when to cancel an iteration
  • 30.
    We were ableto improve on the next iteration, but it was still hard
  • 31.
    Ranges help reinforceobstacles Obstacle removed
  • 32.
    Why 2/3? Becauseit is both simple and pessimistic PERT does a similar thing: Expected = [BestCase + (4*MostLikely) + WorstCase] / 6 Source on PERT: Software Estimation , Steve McConnell, p109
  • 33.
    Using ranges tobetter communicate
  • 34.
    Using range estimationto communicate risk Size of your range communicates the risk of your task May encourage you to break up tasks, or better define them. Scrum is all about better communication with the customer – so are ranges
  • 35.
    How long? Um…2 days 4 days Do you know your fudge factor? You Your Boss Big Boss
  • 36.
    How long? 2-4days 2-4 days Ranges help you control your fudge factor You Your Boss Big Boss
  • 37.
    Another example: Use ranges to better empower your boss or client You Your Boss Big Boss
  • 38.
    Perfect – Doit! How long? How much for X? GRRR Umm….. You Your Boss Big Boss 2 days Actually … 4 days 4 days later… 2 days * rate Budget Left: 2 days
  • 39.
    Instead…. You YourBoss Big Boss
  • 40.
    No thx, dosomething easier How long? How much for X? YES! You Your Boss Big Boss 2-4 days Done! 2 days later… 2-4 days * rate Budget Left: 2 days
  • 41.
    Potential pitfalls ofrange estimation
  • 42.
    Potential pitfalls ofrange estimation Really Wide Ranges Not everything can take 2 – 200 hours or you lose all credibility
  • 43.
    Potential pitfalls ofrange estimation Bosses who don’t get it You’re going to have to sell them on how your estimates will improve their decision making ability.
  • 44.
    Potential pitfalls ofrange estimation Pushed back deadlines Ranges are not an excuse to always miss deadlines. But they do make it less of a surprise, and encourage you to be more cautious.
  • 45.
    Potential pitfalls ofrange estimation Is 2/3 the new single-point? Be careful not to just start treating the 2/3 calculated estimate, use the ranges.
  • 46.
  • 47.
    Questions? Arin Sime434 996 5226 [email_address] Twitter.com/ArinSime

Editor's Notes

  • #4 how long does it take you to drive to work? Is that everyday? Traffic? Optimistic?
  • #23 Show single point then double point
  • #26 Yellow doesn’t know what they’re talking about.