In honor of Robert M. Pirsig, author of Zen and the Art of Motorcycle Maintenance, let’s have a Chautauqua on quality and value how we’re measuring it. Piggy-backing on last year’s theme from the Twin Cities Agile Day, we’re part of an AntiFrAgile organization if we’re measuring the right stuff. Are we measuring the right stuff? How do you know? I’ll share some philosophy, thoughts, and experience to start our discussion. We’ll use a polling tool to make the discussion interactive, participatory, and allow you to share your stories and experience.

1. 4/14/2018
1
How do we know we’re delivering value,
quality, and having a beneficial impact?
Presented by Kevin Burns
@
April 14, 2018
BestBuy HQ
1
@kevinbburns, kburns@sagesw.com
2
David Hussman
Kevin Burns
Agile/Lean Product Development Coach
Org Change Agent
History and Experience
3 4
Themes to achieving
Value, Quality and
Beneficial Impact
• Transparency
• Caring
• Continuous Learning
• Measures
5
Transparency
How transparent are you?
6

2. 4/14/2018
2
7@kevinbburns, kburns@sagesw.com
3 Volunteers
What’s your favorite book on Quality?
kburns@sagesw.com, @kevinbburns 8
9
Robert M. Pirsig September 6, 1928 – April 24, 2017
10
And what is good,
Phaedrus,
And what is not good –
Need we ask anyone to
tell us these things?
11
The place to improve the world is first in one's own heart and
head and hands, and then work outward from there. - Robert M. Pirsig
Is value determined by delivery on time, on budget, and on scope?
Are they using everything we delivered?
12
Is the scope delighting the customer?

3. 4/14/2018
3
In a survey of 4 products, 65% of the features were rarely or never used.
How much money could have been
saved if we never built them?
13
Assumptions
Challenged?
3 things we wish were true
• Customer knows what they want
• Developers know how to build it
• Nothing will change along the way
14
3 things we have to live with
• Impact isn’t known until software is
used in production
• Developers discover how to build it
• Many things change along the way
How transparent is
your org today?
Transparency leads to…
• Accountability
• Psychological safety
• Creativity
• Innovation
• Opportunity
• Ideas
15
kburns@sagesw.com, @kevinbburns 16
kburns@sagesw.com, @kevinbburns 17 kburns@sagesw.com, @kevinbburns 18

4. 4/14/2018
4
Westrum Model: How orgs process info
Pathological Bureaucratic Generative
Power oriented Rule oriented Performance oriented
Low cooperation Modest cooperation High cooperation
Messengers shot Messengers neglected Messengers trained
Responsibilities shirked Narrow responsibilities Risks are shared
Bridging discouraged Bridging tolerated Bridging encouraged
Failure scapegoating Failure justice Failure inquiry
Novelty crushed Novelty problems Novelty implemented
kburns@sagesw.com, @kevinbburns19 20
Caring
What do you care about?
kburns@sagesw.com, @kevinbburns 21
People don’t care what you do, they care why you do it.
kburns@sagesw.com, @kevinbburns 22
kburns@sagesw.com, @kevinbburns 23 24

5. 4/14/2018
5
25
Are you proud of what you’ve built?
kburns@sagesw.com, @kevinbburns 26
kburns@sagesw.com, @kevinbburns 27
Business resilience isn’t only about innovation and money, it’s also
about social and environmental responsibility.
Does society see us as having a socially redeemable pursuit?
Are we behaving ethically?
28
Do you have an environment people thrive in?
kburns@sagesw.com, @kevinbburns 29 30
Continuous Learning
The Search for Truth

6. 4/14/2018
6
kburns@sagesw.com, @kevinbburns 31
Didactic and binary
challenge
32
• Esthetic vs Classical
• Artistic vs Technological
• Subjective vs Objective
• Metaphysic vs Scientific
• Methos vs Logos (myth vs logic)
• Mind vs Matter
• Dialogue vs Rhetoric
• Plutonic vs Socratic
• Ying vs yang
• 0 vs 1
What do you see?
Seven Habits of Highly Effective People - Stephen Covey
33 kburns@sagesw.com, @kevinbburns 34
Were these cake instructions implemented correctly?
Was the customer delighted?
Pirsig changed view of Reality to be Quality First
35
Reality
Quality Pheadrus was
teaching
Quality Pheadrus
should be teaching
Romantic (Emotional)Classic (Intellectual)
Objective (Physical)Subjective (Mental)
Quality (Reality)
Objective Reality
(Matter)
Subjective Reality
(Mind)
Classic Quality
(Intellectual Reality)
Romantic Quality
(Preintellectual Reality)
Zen and the art of motorcycle maintenance page 223.
Pirsig in pursuit of Truth
What is the truth and how do you know it when you have it?
. . .
How do we really ‘know’ anything? Is there an “I,” a “soul,”
which knows, or is this soul merely cells coordinating senses?
. . .
Is reality basically changing, or is it fixed and permanent?
. . .
When it’s said that something means something, what’s
meant by that?
36Zen and the art of motorcycle maintenance page 112.

7. 4/14/2018
7
Quality is no longer The Problem
Pirsig became comfortable with the fact that quality isn’t definable. It’s
something intuitive. Not of the scientific world nor the esthetic world.
Neither classic nor artistic. Neither mind nor matter.
The new problem became analysis itself. He found a problem with logic
and rational thought. In order for us to be able to reason, we need to
be able to define things. Without definition, there can’t be reason. If
we can’t define Quality, we have a problem with Analysis.
37Zen and the art of motorcycle maintenance page 196. 38
Social economics and unconventional wisdom change the game.
Is Quality subjective? Is Truth Relative? Are Facts
Alternative? Is News Fake?
39Zen and the art of motorcycle maintenance page205-223.
“Does this undefined ‘quality’ of yours exist in the things we observe?” they asked,
“Or is it subjective, existing only in the observer?”
Therefore, quality is just a fancy name for whatever you like.
Is what we like truly subjective or has it been programed by society?
Quality is neither subjective nor objective. If it’s defined by the Observer, where did
the observer get his/her thoughts…from society?
It’s monism. It’s above mind and matter.
Are you purposely
injecting chaos into
your environment
to see how resilient
it is?
40
41
Measures
What’s your single
most important
measure?
kburns@sagesw.com, @kevinbburns 42

8. 4/14/2018
8
If I had an hour to solve a problem
and my life depended on the
solution, I would spend the first
55 minutes determining the
proper question to ask, for once I
know the proper question, I could
solve the problem in less than five
minutes.
Albert Einstein
kburns@sagesw.com, @kevinbburns 44
Which measures are
meaningful & why?
45
Strategizing what to measure
KPI’s – Key Performance Indicators
OKR – Objective and Key Results
Market Differentiation Strategy – Value/Quality, Convenience, Price
Market Disruption – Changes the rules of the game
Direct
Customer/Prospect
Feedback Methods
• Interview (in-person/phone)
• Survey/Poll
• Email/Chat
• Focus Groups
• Observation
• Usability Testing
47
Indirect Feedback
Methods
• A/B Testing
• Progressive Testing / Canary Releases
• Process Behavior
• Click-through rates
• Abandonment rates
• Conversion rates
• Net Promoter
• Social media
• 3rd party data (Nielsen, Gallup, LexisNexis, etc)
48

9. 4/14/2018
9
Be careful about
what you think
you’ve learned
• What people say they would pay, isn’t
typically what they would actually pay.
• What a user says they want and what
they actually need (or want) are two
different things.
• Users and buyers aren’t always the
same person.
• Users are very different (as are buyers)
49
50
It’s the unknown unknowns and
epistemic arrogance that kills
51
Can we learn from our experience and
focus on the signal in the noise?
Bad science is 70% behaviorally based
Half of all scientific research isn’t trust worthy
Lies – Damn Lies - Statistics
Pirsig’s Scientific Method
1. State Problem,
2. Hypothesis cause of problem,
3. Design Experiments to test hypothesis,
4. Predict results,
5. Observed results, and
6. Draw Conclusions.
The real purpose of scientific method is to make sure Nature hasn’t
misled you into thinking you know something you don’t actually know.
52Zen and the art of motorcycle maintenance page 93.
Mind your hubris
53 54
It is a puzzling thing. The truth knocks on the door and you say,
"Go away, I'm looking for the truth," and so it goes away. Puzzling.
- Robert M. Pirsig

10. 4/14/2018
10
55 56
Marty Cagan
• Customers don’t know what they want. It’s very hard to envision
the solution you want without actually seeing it.
• At least 2/3 of our ideas are never going to work. The other 1/3
will take 3-4 iterations to get right.
• The role of the product manager is to discover a product that is
valuable, usable, and feasible. Product, design, and engineering
work together to arrive at optimal solution.
User, Customer, Client
Business
Valuable
Design Usable
Software EngineeringBusiness Sponsor
Technically
Feasible
Do you have the right balance to deliver Quality, Value, & Innovation?
Quality
Innovation
Value
Do you have the right balance?
60
Quality
Value

11. 4/14/2018
11
Lean Startup
• Are we asking what are Minimum
Viable (Valuable) Product and how do
we know when we’ve delivered it?
• Use a scientific method to measure,
learn and pivot or preserver.
• Use meaningful quantitative
objective measure to evaluate impact.
• Can you use A/B testing?
61
Impact-Drive Development
62
63 64
Merging Product Discovery and Product Delivery
65
http://productagility.org/
Iterative Discovery and Delivery Model
Hypothesis
Ideas
Build and Validate
Iterative Stable Team Delivery
Project/Release Planning

12. 4/14/2018
12
Value and/or Impact driven culture
• Are we measuring the Cost vs Benefit at all levels of our work items?
• Portfolio
• Program
• Project
• Feature/Capability
• Story/Requirement
• Tasks/Test
• Are we measuring the Impact our features have on our customers?
• The act of sizing helps us define done
and what the really valuable work is
• Story telling and test statements create
understanding of value and DoD
67 kburns@sagesw.com, @kevinbburns 68
How to measure anything – Douglas Hubbard
http://www.howtomeasureanything.com/
69
Measurement Basics
• A measurement is an observation that quantitatively reduces uncertainty.
• A good object of measurement is something that is clearly defined and it’s observable.
• Uncertainty is the lack of certainty: the true outcome/state/value is not known.
• Risk is a state of uncertainty in which some of the possibilities involve a loss.
• Much pessimism about measurement comes from a lack of experience
• Your problem is not as unique as you think.
• You have more data than you think.
• You need less data than you think.
• An adequate amount of new data is more accessible than you think.
Douglas Hubbard
Select a measurement
method
71
To figure out which category of measurement
methods are appropriate for a particular case, we
must ask several questions:
1. Decomposition: Which parts of the thing
are we uncertain about?
2. Secondary research: How has the thing
(or its parts) been measured by others?
3. Observation: How do the identified
observables lend themselves to
measurement?
4. Measure just enough: How much do we
need to measure it?
5. Consider the error: How might our
observations be misleading?
Conclusion
• Customers don’t always know what they want
• People don’t buy features they buy benefits
• The value you’re offering isn’t unique…it is evaluated in terms of it’s relationship
to what they already have, know or use (substitution cost)
• People will buy your product at a price commensurate with perceived benefits
received at a cost lower than your competitor or some other substitution cost
• Given this, how fast can you find-out whether you’re on the right track

13. 4/14/2018
13
Questions
&
Next Steps
• Do you know what outcomes
and impacts you’re trying to
achieve?
• Do we have the right people?
• Do you need help?
73
42
74
Life, the Universe, and Everything