Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)

# a g i l e c a m p 2 0 1 8 | @ m a t t p h i l i p
NO ESTIMATES
Forecasting with Less Eﬀort and More Accuracy

@mattphilip #agilecamp2018
ESTIMATES, PLEASE!
▫︎ Number of triples that Babe Ruth hit in his career
▫︎ Distance in km of Chile north to south
▫︎ Year in which Newark was founded
▫︎ Windows in the Gateway Arch in St. Louis
Aim for 75%
conﬁdence

ESTIMATES, PLEASE!
▫︎ Number of triples that Babe Ruth hit in his career (136)
▫︎ Distance in km of Chile north to south (4,270 km)
▫︎ Year in which Newark was founded (1666)
▫︎ Windows in the Gateway Arch in St. Louis (32; 16 per side)

MANIFESTO RELEVANCE
Planning Poker!
Estimates
Value?
VUCA

WHAT NOESTIMATES IS NOT SAYING
▫︎ You are evil if you estimate
▫︎ All estimates are totally useless
▫︎ Stop doing your successful estimating practice
▫︎ Stop having the conversations to understand/analyze/
break down work
▫︎ Work items must be the same size
▫︎ You must place your full faith and conﬁdence in Monte
Carlo forecasts

WHAT NOESTIMATES IS SAYING
▫︎ Know why you are estimating
▫︎ Discover for yourself how good you are at estimating
(measure)
▫︎ Keep doing the things that help you understand the work
▫︎ Upfront estimates need to be held loosely
▫︎ If you focus on delivering value quickly, you obviate the
need for Iron Triangle considerations

MY NOESTIMATES MANIFESTO
… We have come to value:
Reducing sources of variation over Improving estimating
Data over Intuition*
Probabilistic over Deterministic
Minimal Viable Product over Full scope
That is, while there is value in the items on the right, we value
the items on the left more.
*Neil Killick uses “empiricism over guesswork”

HOFSTADTER’S LAW
“It always takes longer than you expect,
even when you take into account
Hofstadter’s Law.”

PARKINSON’S LAW
Work expands so as to ﬁll the time
available for its completion.

“POOR ESTIMATION SKILLS”

WHAT DO WE ESTIMATE?

WHAT WE ESTIMATE
Code

WHAT WE ESTIMATE
Analyze+Design+Code+Test+Deploy

WHAT ACTUALLY HAPPENS
Analyze Code Test DeployDesign
Wait for approval
Wait for staﬀ
availability
Rework
Wait for API
availability
Switch to  
higher-priority work
Wait for approval
Wait for  
blocker  
removal

WHAT’S GOING ON?
Value-added time
_________________________
Total elapsed delivery time
Flow eﬃciency =

WHAT’S GOING ON?
Low process eﬃciency (typically 5-15%
in software delivery) means that even if
we nailed the eﬀort estimates … we
would be accurately predicting 5-15% of
elapsed delivery time! 
— Troy Magennis

DO YOU ASSUME CORRELATION?
Is the initial sizing a good predictor for when you can get your stuﬀ?
In our case, the surprising truth was ”no.”
— Mattias Skarin, Real-World Kanban

SOURCES OF VARIATION
How many can you name?

SOURCES OF VARIATION
Waiting for  
availability
Rework
Team  
dependencies
Stages in
team
development
(Tuckman)
Steps/ 
handoﬀs
Selection  
policy
Essential  
complication
Accidental  
complication
System  
dependencies
Work In  
Progress
Technology/ 
domain/ 
product
Specialization
Team  
composition
Multitasking/ 
context  
switching
Collaboration 
policy
Blockers

WHAT YOU CAN DO ABOUT VARIATION
How many remedies 
can you name?

▫︎Lower WIP/System WIP
▫︎Five Focusing Steps
▫︎Blocker clustering
▫︎Reduce workﬂow stages
▫︎Explicit policies
▫︎Cost of Delay scheduling, sequencing and selection
▫︎Reduce “expedites”
Lean-Kanban

▫︎“Agile 101” (simple, decoupled design; thin vertical
slices; pairing)
▫︎Identify/make visible/measure dependencies
▫︎Bigger teams (to reduce dependencies)
▫︎Collaborate/Share work (Dimitar Bakardzhiev)
▫︎Spike and stabilize (Dan North)
▫︎Reduce accidental complexity (Liz Keogh)
Team
Why?

POLICIES TO REDUCE VARIATION
▫︎We will only start new work at about the same rate that
we finish old work.
▫︎We will make every reasonable effort to finish all work
that is started and minimize wasted effort due to
discarded work items
▫︎If work becomes blocked, we will do everything we can
do unblock that work as expeditiously as possible.
▫︎We will closely monitor our policies around the order in
which we pull items through our system so that some
work items do not sit and age unnecessarily.
— Daniel Vacanti, When Will It Be Done?

HUMAN BIAS
How many biases can you name?

HUMAN BIAS

MANAGEMENT PARADIGMS

FIRST, KNOW YOUR DOMAIN

KEOGH’S “SCALE OF IGNORANCE”
1
Just about
everyone in the
world has done
this.
Lots of people have
done this, including
someone on our team.2
Someone in our company
has done this, or we have
access to expertise.3Someone in the
world did this, but
not in our
organization (and
probably at a
competitor).
4
Nobody in the world
has ever done this
before.
5

CHOOSE RELEVANT APPROACH
Monte Carlo
simulations
Wisdom of
crowds
Planning
poker
Experts

DATA OVER INTUITION

LET’S PRACTICE!
1.At your table, choose someone to roll the die/dice.
2.Roll the die/dice.
3.Everyone else record each result (roll 10 times).

34
PROBABILISTIC OVER
DETERMINISTIC

PROBABILISTIC OVER DETERMINISTIC
Dec 28
— graphic courtesy of Larry Maccherone, (maccherone.com/lumenize)

VACANTI’S VERITY
When making a forecast (predicting the future),
you have to accept that there is more than one
possible outcome. Therefore… A forecast is a
calculation about the future that includes both
a range and a probability of that range
occurring.

2%
20% 21%
3%
16% 16%
9%
7%
2% 2% 2%
Dec 28 Feb 17Nov 30 Jan 20

PROBABILISTIC FORECASTING

PROBABILISTIC FORECASTING: ONE ITEM

BACK TO YOUR SCATTERPLOT

STRAIGHT-LINE (AVERAGE) FORECASTS
Team A Team B
Work item #1 5 days 3 days
Work item #2 1 day 3 days
Work item #3 3 days 3 days
Average 3 days 3 days
3-day delivery time  
expectancy
66% 100%

EXPECTATIONS, NOT COMMITMENTS

SERVICE-DELIVERY EXPECTATIONS
Expectation Performance
Expedite 90% 3 days or less
Standard Urgency 85% 10 days or less
Fixed-Date 95% On-time

TO ESTIMATE OR NOT TO ESTIMATE?
NeverEstimateAnything
AlwaysEstimateEverything
Estimate Default Behavior Don’t estimate
Eﬀort Factors All sources of variation
Intuition Approach Data
Deterministic Forecast Type Probabilistic
Tasks in hours Level of Detail Work items in days
High Eﬀort Spent Low
Estimating better Improvement Aim Reducing variation
Commitment Management Service Expectations

BETTER QUESTIONS TO ASK
@mattphilip
▫︎In what context would estimates bring value, and what
are we willing to do about it when they don’t? – Woody
Zuill
▫︎How much time do we want to invest in this? – Matt
Wynne
▫︎What can you do to maximize value and reduce risk in
planning and delivery? – Vasco Duarte
▫︎Can we build a minimal set of functionality and then
learn what else we must build?
▫︎Would we/you not invest in this work? If not, at what
order-of-magnitude estimate would we/you take that
action?
▫︎What actions would be diﬀerent based on an estimate?

HOW TO GET STARTED (PROJECT IN PROGRESS)
1. Continue to discuss the work to analyze it and break it into thin
vertical slices.
2. Start tracking two pieces of data for each work item: commit
date and delivery date.
3. After accumulating 10 data points, run a Monte Carlo
simulation to provide a probabilistic forecast.
4. Begin answering the “When” question using data, either at the
project level or individual work-item level.
5. Validate whether the probabilistic forecasts match what actually
occurs.
6. If the forecasts are helpful, wean your team oﬀ estimating.
7. Focus improvement eﬀorts on reducing the sources of variation.

HOW TO GET STARTED (NEW PROJECT)
1. Ask the “better questions” above about the use and
importance of estimates.
2. Use reference-class data (similar delivery times from other
projects in similar tech stacks with similar teams) to provide an
initial probabilistic forecast
3. As soon as you accumulate 10 data points, run a Monte Carlo
simulation to provide an updated probabilistic forecast.
4. Continually reforecast when you get more information.

REFERENCES AND FURTHER EXPLORATION
▫︎ noestimatesbook.com (Vasco Duarte)
▫︎ infoq.com/articles/noestimates-monte-carlo (Dimitar Bakardzhiev)
▫︎ priceonomics.com/why-are-projects-always-behind-schedule
▫︎ http://scrumandkanban.co.uk/estimation-meets-cyneﬁn/
▫︎ ronjeﬀries.com
▫︎ Lizkeogh.com
▫︎ https://neilkillick.wordpress.com/
▫︎ http://zuill.us/WoodyZuill/
▫︎ mattphilip.wordpress.com/noestimates-game
▫︎ When Will It Be Done? (Dan Vacanti)
▫︎ focusedobjective.com (Troy Magennis)
▫︎ actionableagile.com (Dan Vacanti)
▫︎ kanbanize.com
▫︎ scrum.org

www.noestimat.es
THANK YOU
@mattphilip@mastodon.cloud
gab.ai/mattphilip
@mattphilip

Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)

Similar to Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018) (20)

More from Matthew Philip

More from Matthew Philip (20)

Recently uploaded

Recently uploaded (9)

Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)