How do we know we're delivering value? Twin Cities Agile Meetup May 9, 2017
How do we know we’re delivering value
or having a beneficial impact?
In honor of Robert M. Pirsig,
let’s have a Chautauqua on
value, quality, and
making a difference.
May 9, 2017
Org Change Agent
email@example.com, @kevinbburns 2
History and Experience
firstname.lastname@example.org, @kevinbburns 3
email@example.com, @kevinbburns 4
Robert M. Pirsig September 6, 1928 – April 24, 2017
Before we start…
Who wants to share their favorite
firstname.lastname@example.org, @kevinbburns 5
email@example.com, @kevinbburns 6
The place to improve the world is first in one's own heart and
head and hands, and then work outward from there. - Robert M. Pirsig
The only Zen you can find on
the tops of mountains is the
Zen you bring up there.
- Robert M. Pirsig
firstname.lastname@example.org, @kevinbburns 7
WHO’s measuring value, outcomes, and/or impacts today?
HOW are you measuring them?
If you’re NOT measuring them, WHY NOT? WHAT’S STOPPING YOU?
WHO’s measuring cost?
HOW are you measuring cost?
email@example.com, @kevinbburns 8
Are you part of the GEMBA (value chain)?
Agile Principle Number One
Our highest priority is to satisfy the customer through early and
continuous delivery of valuable software.
(should we change valuable to beneficial impact?)
How do we define value (impact) [or quality] … how do we measure it?
Not all Projects (or Features) are created equal.
firstname.lastname@example.org, @kevinbburns 15
Is value determined by delivery on time, on budget, and on scope?
Are they using everything we delivered?
email@example.com, @kevinbburns 16
Is the scope delighting the customer?
And what is good,
And what is not good –
Need we ask anyone to
tell us these things?
In a survey of 4 products, 65% of the features were rarely or never used.
How much money could have been
saved if we never built them?
firstname.lastname@example.org, @kevinbburns 18
If I had an hour to solve a problem and my life
depended on the solution, I would spend the first 55
minutes determining the proper question to ask, for
once I know the proper question, I could solve the
problem in less than five minutes.
email@example.com, @kevinbburns 19
Didactic and binary challenge
firstname.lastname@example.org, @kevinbburns 23
Esthetic vs Classical
Artistic vs Technological
Subjective vs Objective
Metaphysic vs Scientific
Methos vs Logos (myth vs logic)
Mind vs Matter
Dialogue vs Rhetoric
Plutonic vs Socratic
0 vs 1
Pheadrus changed view of Reality to be Quality First
email@example.com, @kevinbburns 24
Quality Pheadrus was
should be teaching
Romantic (Emotional)Classic (Intellectual)
Objective (Physical)Subjective (Mental)
Zen and the art of motorcycle maintenance page 223.
Pirsig in pursuit of Truth
What is the truth and how do you know it when you have it?
. . .
How do we really ‘know’ anything? Is there an “I,” a “soul,”
which knows, or is this soul merely cells coordinating senses?
. . .
Is reality basically changing, or is it fixed and permanent?
. . .
When it’s said that something means something, what’s
meant by that?
firstname.lastname@example.org, @kevinbburns 25Zen and the art of motorcycle maintenance page 112.
Quality is no longer The Problem
Phaedrus became comfortable with the fact that quality isn’t definable.
It’s something intuitive. Not of the scientific world nor the esthetic
world. Neither classic nor artistic. Neither mind nor matter.
The new problem became analysis itself. He found a problem with logic
and rational thought. In order for us to be able to reason, we need to
be able to define things. Without definition, there can’t be reason. If
we can’t define Quality, we have a problem with Analysis.
email@example.com, @kevinbburns 26Zen and the art of motorcycle maintenance page 196.
Is Quality subjective? Is Truth Relative? Are Facts
firstname.lastname@example.org, @kevinbburns 27Zen and the art of motorcycle maintenance page205-223.
“Does this undefined ‘quality’ of yours exist in the things we observe?” they asked,
“Or is it subjective, existing only in the observer?”
Therefore, quality is just a fancy name for what every you like.
Is what we like truly subjective or has it been programed by society?
Quality is neither subjective nor objective. If it’s defined by the Observer, where did
the observer get his/her thoughts…from society?
It’s monism. It’s above mind and matter.
Pirsig’s Scientific Method
1. Statement of the problem,
2. Hypothesis as to the cause of the problem,
3. Experiments designed to test each hypothesis,
4. Predict results of the experiments,
5. Observed results of the experiments, and
6. Conclusions from the results of the experiments,
The real purpose of scientific method is to make sure Nature hasn’t
misled you into thinking you know something you don’t actually know.
email@example.com, @kevinbburns 28Zen and the art of motorcycle maintenance page 93.
firstname.lastname@example.org, @kevinbburns 29
It is a puzzling thing.
The truth knocks on the
door and you say, "Go
away, I'm looking for
the truth," and so it
goes away. Puzzling.
- Robert M. Pirsig
• 3 things we wish were true
• Customer knows what they want
• Developers know how to build it
• Nothing will change along the way
• 3 things we have to live with
• Impact isn’t known until software is used in production
• Developers discover how to build it
• Many things change along the way
email@example.com, @kevinbburns 30
Marty Cagan Quotes
• Customers don’t know what they want. It’s
very hard to envision the solution you want
without actually seeing it.
• Like Phaedrus’ quality…it’s not definable but
they know it when they see it.
• At least 2/3 of our ideas are never going to
work. The other 1/3 will take 3-4 iterations
to get right.
• Phaedrus too iterated, he called it
crystallization of thought and reason in the
pursuit of Truth.
• The role of the product manager is to
discover a product that is valuable, usable,
and feasible. Product, design, and
engineering work together to arrive at
firstname.lastname@example.org, @kevinbburns 31
What we measure is changing
PO, SM, BL
AD, DD, DA
UX, BA, QA, SME
email@example.com, @kevinbburns 33
• Are we asking what are Minimum
Viable (Valuable) Product and how do
we know when we’ve delivered it?
• Use a scientific method to measure,
learn and pivot or preserver.
• Use meaningful quantitative
objective measure to evaluate impact.
• Can you use A/B testing?
firstname.lastname@example.org, @kevinbburns 34
UX, BA, QA, SME
AD, DD, DA
PO, SM, BL
Use scientific method
(measurable) to learn
and discovery your
MVP innovations emerge
from Conversations email@example.com, @kevinbburns 35
firstname.lastname@example.org, @kevinbburns 36
Value and/or Impact driven culture
• Are we measuring the Cost vs Benefit at all levels of our work items?
• Are we measuring the Impact our features have on our customers?
• The act of sizing helps us define done
and what the really valuable work is
• Using story telling and test statements
create understanding of value and DoD
email@example.com, @kevinbburns 38
X-S 1 Sprint
S <1 month
M 1-3 months
L 3-9 months
X-L >9 months
Team task hours to capacity (2,4,6)
Solution Decomposition Sizing Pattern
Sizing our Cost
The act of sizing helps us
valuable to deliver.
Scope doesn’t grow, our
Deliver 100% of 10% of Project
• Can we incrementally deliver value and test it’s
• Can we create incremental release plans to
deliver 100% of 10% of project?
• What constraints do we have in working this
• Can we overcome or work within these
constraints and still deliver incrementally?
• What/who is preventing this approach?
firstname.lastname@example.org, @kevinbburns 40
Create Faster Feedback
• When queues and batch sizes are large
feedback is slow
• Slow feedback hurts quality, efficiency, and
• Feedback speed has enormous economic
leverage in product development, but it is
rarely explicitly managed
email@example.com, @kevinbburns 41
The Front-Loaded Lottery
• A lottery ticket pays $3000 to winning three digit number
• You can pick the number in two ways:
• Pay $3 to select all three digits at once
• Pay $1 for the first digit, find out if it is correct, then choose if you wish to pay
$1 for the second digit, and then choose if you wish to pay $1 for the third
firstname.lastname@example.org, @kevinbburns 42
Value of Feedback
Savings = $0.90
Savings = $0.99
0 $1 $2 $3
Cumulative Investmentkburns@sagesw.com, @kevinbburns 43
Sequence Work Correctly (Cost of Delay)
• The sequence in which work is processed is called the queuing
• By changing the queuing discipline we can reduce the cost of a queue
without decreasing the size of the queue
• Since manufacturing has homogeneous flows it always uses FIFO
• For the non-homogeneous flows of product development other
approaches have better economics
email@example.com, @kevinbburns 44
Use FIFO for Homogeneous Flow
Project Duration Cost of Delay
1 3 3
2 3 3
3 3 3
firstname.lastname@example.org, @kevinbburns 45
Weighted Shortest Job First (WSJF) for
High Weight First
Low Weight First
Project Duration Cost of
1 1 10 10
2 3 3 1
3 10 1 0.1
96 % Reduction in COD
email@example.com, @kevinbburns 46
Paul Ellarby example
1. Create Method for measuring Value
2. Understand the value and cost of each
portfolio down to the feature level
3. Allocate Value Points across
4. Track Value vs Cost for each iteration
firstname.lastname@example.org, @kevinbburns 47
How to measure anything – Douglas Hubbard
email@example.com, @kevinbburns 48
• A measurement is an observation that quantitatively reduces uncertainty.
Measurements might not yield precise, certain judgments, but they do reduce
• A good object of measurement is something that is clearly defined and it’s
• Uncertainty is the lack of certainty: the true outcome/state/value is not known.
• Risk is a state of uncertainty in which some of the possibilities involve a loss.
• Much pessimism about measurement comes from a lack of experience making
measurements. Hubbard, who is far more experienced with measurement than
his readers, says:
• Your problem is not as unique as you think.
• You have more data than you think.
• You need less data than you think.
• An adequate amount of new data is more accessible than you think.
firstname.lastname@example.org, @kevinbburns 49
Apply Information Economics for Decision-making
what you know
and don’t know
but want to
Pick a variable,
the value of
act on it.
email@example.com, @kevinbburns 50
To figure out which category of measurement
methods are appropriate for a particular case, we
must ask several questions:
1. Decomposition: Which parts of the thing
are we uncertain about?
2. Secondary research: How has the thing
(or its parts) been measured by others?
3. Observation: How do the identified
observables lend themselves to
4. Measure just enough: How much do we
need to measure it?
5. Consider the error: How might our
observations be misleading?
• Change as Rest
• Failure as Excellence
• Difference as Balance
• Start as Destination
• Phaedrus, sometimes it’s better to travel than arrive.
Carol Company philosophy
Ask yourself daily…
• Did I add value to the team?
• Did I add shareholder/stakeholder value?
• Do I believe in the mission/vision?
• Do I like the people I’m working with?
• Am I growing personally and professionally?
The more ‘no’ answers you provide to the above questions, the
faster you should head for the door.
Questions & Next Steps
• How many of us know what business/user outcomes and impacts
we’re trying to achieve on our projects?
• Do you have metrics in place to evaluation our progress/success
outcomes and impacts?
• Who want’s help creating some objective measures?
• Where do we go from here?
firstname.lastname@example.org, @kevinbburns 55