Your SlideShare is downloading. ×
0
blog: jabe.co
FAILING
principles and practices of
Vanity Validation a Paradox of Passionate Commitment
WELL
@cyetain
What was the last thing you
failed at?
What did you learn?
Did you share your failure
with anyone?
Pre-Talk Quest...
blog: jabe.co
FAILING
principles and practices of
Vanity Validation a Paradox of Passionate Commitment
WELL
HELLLOOOO McFLY
TLC LABS
blog http://jabe.co
Send Anonymous Feedback
http://sayat.me/jabebloom
Joshua (Jabe) Bloom
CTO : T...
@cyetain
Fail Fast
@cyetain
@cyetain
Learning occurs when we
detect and correct error. Error is
any mismatch between what
we intend an action to produ...
@cyetain
How Do We Make Better Choices?
Why is it so hard to Fail?
Could we design a system to help?
@cyetain
One must treat his theory-in-
use as both a psychological
certainty and an intellectual
hypothesis.
-Chris Argyris
@cyetain
How wonderful that we have
met with a paradox. Now we
have some hope of making
progress.
-Niels Bohr
@cyetain
@cyetain
fake dictionary page from colbertnation.com
@cyetain
Although theory without
experiment is empty,
experiment without theory is
blind.
-Paul Thagard
@cyetain
3 Things To Leave With
• Failing Well Produces more Information than
Failing Poorly
• “Passionate Beliefs, Loosel...
@cyetain
We simply cannot rely on
randomness to correct
the problems that
randomness creates.
-Don Reinertsen
@cyetain
undifferentiated streams
of data
@cyetain
@cyetain
“Research is what
I’m doing when I
don’t know what
I’m doing.”
-Wernher von Braun
@cyetain
@cyetain
The Principle of Optimum Failure Rate
0% 100%
Probability of Failure
50%
PotentialInformation
@cyetain
0% 100%
Probability of Failure
50%
PotentialInformation
Greater Asserted
Information
Greater Asserted
Information
@cyetain
0% 100%
Probability of Failure
50%
PotentialInformation
Greater Asserted
Information
Greater Asserted
Information...
@cyetain
0% 100%
Probability of Failure
50%
PotentialInformation
Greater Asserted
Information
Greater Asserted
Information...
@cyetain
Experience of Failure
NumberofSamples The Competency Trap
@cyetain
0% 100%
Probability of Failure
50%
PotentialInformation
@cyetain
0% 100%
Probability of Failure
50%
PotentialInformation
Pretty
Sure
theory
is wrong
Pretty
Sure
theory
is right
I...
@cyetain
0% 100%
Probability of Failure
50%
PotentialInformation
Pretty
Sure
theory
is wrong
Pretty
Sure
theory
is right
I...
@cyetain
“The typical sequence of coin
tosses has high information
content but little value; an
ephemeris, giving the
posi...
@cyetain
Based on what we know
right now, what problems
do we have the least
amount of information
about that we can
reaso...
@cyetain
@cyetain
Risk vs Uncertainty
@cyetain
Alteaory vs Epistemic
Uncertainties
@cyetain
Gamble Invest
@cyetain
Justified MVP
Value of Information
Cost of Acquisition
Cost of MVP
Unjustified
MVP
Over Justified
MVP
Justified
M...
@cyetain
The first principle is that
you must not fool
yourself--and you are the
easiest person to fool.
-Richard Feynman
@cyetain
“Most people don’t know how to
learn. What’s more, those
members of the organization that
many assume to be the b...
@cyetain
“Expertise … breeds an
inability to accept new
views.”
-Laski
@cyetain
@cyetain
VanityValidation
@cyetain
I need to be right even
if I'm wrong.
@cyetain
Defensive Reasoning
@cyetain
Remain in unilateral
control
@cyetain
Maximize "winning"
Minimize "losing"
@cyetain
Suppress negative
feelings
@cyetain
Be as "rational" as possible
-- by which people mean
defining clear objectives and
evaluating their behavior in
t...
@cyetain
Mindset Actions
Results Match
Results Mismatch
Single-loop
Double-loop
@cyetain
Valid Public
Information
@cyetain
@cyetain
whenever we propose a
solution to a problem, we
ought to try as hard as we
can to overthrow our solution,
rather ...
@cyetain
• Identify Your Assumptions and Conclusions
CLEARLY AS POSSIBLE PUBLICLY
• Question Your Assumptions and Conclusi...
@cyetain
http://xkcd.com/
@cyetain
@cyetain
Abduction
not just for
Aliens
@cyetain
@cyetain
[Abduction] goes upon the hope that
there is sufficient affinity between the
reasoner's mind and nature's to rend...
@cyetain
ABDUCE
DEDUCE
INDUCE
Predictive
Probable
Plausible
The Way
Computers "Think"
The Way
Humans Think
Binary
Probabil...
@cyetain
ABDUCE
DEDUCE
INDUCE
Experiences Hypothesises
Expected
Outcomes
If Coherent
If Expected Outcomes
Match Reality
Ef...
@cyetain
ABDUCE
DEDUCEINDUCE
@cyetain
ABDUCE
DEDUCEINDUCE
SURPRISE!!!
@cyetain
Multi-Hypothesis Research
!=
@cyetain
BRAINSTORM
@cyetain
Theories
Opinions
Hypothesizes
The Facts and Just
the Facts
@cyetain
Theories
Opinions
Hypothesizes
Constraints
Criteria
@cyetain
Theories
Opinions
Hypothesizes
Question
Facts
@cyetain
Theories
Opinions
Hypothesizes Request More
Information
@cyetain
NO TALKING!
@cyetain
How Would I Validate my
understanding of this
problem?
How Would I
solve this
Problem?
•Based on your experiences...
@cyetain
I Assert that this I
know this
0% 100%
Probability of Failure
50%
PotentialInformation
Pretty
Sure
theory
is wron...
@cyetain
This is my Hypothesis,
Assumptions and
Experiments
Challenge
Assumptions &
Experiments
Rotate Pairs 2-3 Times
All...
@cyetain
This is my Hypothesis,
Assumptions and
Experiments
@cyetain
Multiple Smaller Experiments
against
Multiple Abductive Hypotheses
instead of
Single Large Experiment
against
Sin...
@cyetain
Failing Well Produces more
Information than Failing
Poorly
@cyetain
What are You Doing w All that Information?
Incremental: Confirm. Disconfirm.
Iteratively: Select Next Step. Gener...
@cyetain
Having “Passionate
Beliefs, Loosely Held”
FAILURE
MUST
BE AN OPTION
@cyetain
Reducing Variability too
soon risks suboptimal
result, too late increases
failure demand
@cyetain
@cyetain
Influences
&
Sources of More Information
@cyetain
Joshua (Jabe) Bloom
CTO : The Library Corporation
& TLC Labs
blog http://jabe.co
Send Anonymous Feedback
http://s...
Upcoming SlideShare
Loading in...5
×

Failing Well Agile 2013

819

Published on

There is a trap hidden inside Lean Startup's CustDev cycle. If entrepreneurs want to be successful, they have to be passionate about their ideas. They need to understand more about their customers than anyone else. I’ve spent the last year lecturing to and teaching entrepreneurs and I've observed that their passion for their idea and their belief that they already DO know everything about their customers can prevent them from actually LEARNING what they need to know in order to create as successful business. The same passion and positive psychology required to succeed in the face of uncertainty is hindering them from learning fast enough to survive.

How can we take our passion, our vision, a couple “wild ass guesses”, and produce meaningful, validated learning?

The question of how to learn as an organization and how to DEMONSTRATE learning has been explored by philosophers of science and by business theorists for years. What can the Lean Startup Community learn about creating scientifically valid experiments that create actionable knowledge?

Learn how to fail well and fail faster by keeping your passion focused on the vision and our dispassionate logic focused on the assumptions.

Published in: Technology, Education
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
819
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
18
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Transcript of "Failing Well Agile 2013"

  1. 1. blog: jabe.co FAILING principles and practices of Vanity Validation a Paradox of Passionate Commitment WELL
  2. 2. @cyetain What was the last thing you failed at? What did you learn? Did you share your failure with anyone? Pre-Talk Questions
  3. 3. blog: jabe.co FAILING principles and practices of Vanity Validation a Paradox of Passionate Commitment WELL
  4. 4. HELLLOOOO McFLY TLC LABS blog http://jabe.co Send Anonymous Feedback http://sayat.me/jabebloom Joshua (Jabe) Bloom CTO : The Library Corporation & Consulting Practioner TLC Labs #agile2013
  5. 5. @cyetain Fail Fast
  6. 6. @cyetain
  7. 7. @cyetain Learning occurs when we detect and correct error. Error is any mismatch between what we intend an action to produce and what actually happens when we implement that action. -Chris Argyris
  8. 8. @cyetain How Do We Make Better Choices? Why is it so hard to Fail? Could we design a system to help?
  9. 9. @cyetain One must treat his theory-in- use as both a psychological certainty and an intellectual hypothesis. -Chris Argyris
  10. 10. @cyetain How wonderful that we have met with a paradox. Now we have some hope of making progress. -Niels Bohr
  11. 11. @cyetain
  12. 12. @cyetain fake dictionary page from colbertnation.com
  13. 13. @cyetain Although theory without experiment is empty, experiment without theory is blind. -Paul Thagard
  14. 14. @cyetain 3 Things To Leave With • Failing Well Produces more Information than Failing Poorly • “Passionate Beliefs, Loosely Held” • Reducing Variability too, Soon risks suboptimal result, too Late increases Failure blindness
  15. 15. @cyetain We simply cannot rely on randomness to correct the problems that randomness creates. -Don Reinertsen
  16. 16. @cyetain undifferentiated streams of data
  17. 17. @cyetain
  18. 18. @cyetain “Research is what I’m doing when I don’t know what I’m doing.” -Wernher von Braun
  19. 19. @cyetain
  20. 20. @cyetain The Principle of Optimum Failure Rate 0% 100% Probability of Failure 50% PotentialInformation
  21. 21. @cyetain 0% 100% Probability of Failure 50% PotentialInformation Greater Asserted Information Greater Asserted Information
  22. 22. @cyetain 0% 100% Probability of Failure 50% PotentialInformation Greater Asserted Information Greater Asserted Information Pretty Sure theory is wrong Pretty Sure theory is right Interesting Ideas
  23. 23. @cyetain 0% 100% Probability of Failure 50% PotentialInformation Greater Asserted Information Greater Asserted Information Pretty Sure theory is wrong Pretty Sure theory is right Interesting Ideas Uncomfortable Confident
  24. 24. @cyetain Experience of Failure NumberofSamples The Competency Trap
  25. 25. @cyetain 0% 100% Probability of Failure 50% PotentialInformation
  26. 26. @cyetain 0% 100% Probability of Failure 50% PotentialInformation Pretty Sure theory is wrong Pretty Sure theory is right Interesting IdeasHidden Risk Hidden Value The Line of SURPRISE!
  27. 27. @cyetain 0% 100% Probability of Failure 50% PotentialInformation Pretty Sure theory is wrong Pretty Sure theory is right Interesting IdeasHidden Risk Hidden Value 1 2 3 During Customer Development Focus on Interesting Ideas Before Scaling Validate Your "We Know This Assumptions" to reduce risk of Failure Demand After Customer Validation Run experiments to Validate Assumptions of Failure 2 3 1
  28. 28. @cyetain “The typical sequence of coin tosses has high information content but little value; an ephemeris, giving the positions of the moon and planets every day for a hundred years, has no more information than the equations of motion and initial conditions from which it was calculated, but saves it’s owner the effort of recalculating these positions.” -Charles H. Bennett
  29. 29. @cyetain Based on what we know right now, what problems do we have the least amount of information about that we can reasonably expect to understand?
  30. 30. @cyetain
  31. 31. @cyetain Risk vs Uncertainty
  32. 32. @cyetain Alteaory vs Epistemic Uncertainties
  33. 33. @cyetain Gamble Invest
  34. 34. @cyetain Justified MVP Value of Information Cost of Acquisition Cost of MVP Unjustified MVP Over Justified MVP Justified MVP
  35. 35. @cyetain The first principle is that you must not fool yourself--and you are the easiest person to fool. -Richard Feynman
  36. 36. @cyetain “Most people don’t know how to learn. What’s more, those members of the organization that many assume to be the best at learning are, in fact, not very good at it. I am talking about the well-educated, high-powered, high commitment professionals” -Chris Argyris
  37. 37. @cyetain “Expertise … breeds an inability to accept new views.” -Laski
  38. 38. @cyetain
  39. 39. @cyetain VanityValidation
  40. 40. @cyetain I need to be right even if I'm wrong.
  41. 41. @cyetain Defensive Reasoning
  42. 42. @cyetain Remain in unilateral control
  43. 43. @cyetain Maximize "winning" Minimize "losing"
  44. 44. @cyetain Suppress negative feelings
  45. 45. @cyetain Be as "rational" as possible -- by which people mean defining clear objectives and evaluating their behavior in terms of whether or not they have achieved them
  46. 46. @cyetain Mindset Actions Results Match Results Mismatch Single-loop Double-loop
  47. 47. @cyetain Valid Public Information
  48. 48. @cyetain
  49. 49. @cyetain whenever we propose a solution to a problem, we ought to try as hard as we can to overthrow our solution, rather than defend it. -Karl Popper
  50. 50. @cyetain • Identify Your Assumptions and Conclusions CLEARLY AS POSSIBLE PUBLICLY • Question Your Assumptions and Conclusions • Seek Contrary Data • Learn when to correct your Actions and when to correct your Mindset
  51. 51. @cyetain http://xkcd.com/
  52. 52. @cyetain
  53. 53. @cyetain Abduction not just for Aliens @cyetain
  54. 54. @cyetain [Abduction] goes upon the hope that there is sufficient affinity between the reasoner's mind and nature's to render guessing not altogether hopeless, provided each guess is checked by comparison with observation... The effort should therefore be to make each hypothesis... as near an even bet as possible. -Charles Peirce
  55. 55. @cyetain ABDUCE DEDUCE INDUCE Predictive Probable Plausible The Way Computers "Think" The Way Humans Think Binary Probability Analogue Justifiable
  56. 56. @cyetain ABDUCE DEDUCE INDUCE Experiences Hypothesises Expected Outcomes If Coherent If Expected Outcomes Match Reality Effective Match
  57. 57. @cyetain ABDUCE DEDUCEINDUCE
  58. 58. @cyetain ABDUCE DEDUCEINDUCE SURPRISE!!!
  59. 59. @cyetain Multi-Hypothesis Research !=
  60. 60. @cyetain BRAINSTORM
  61. 61. @cyetain Theories Opinions Hypothesizes The Facts and Just the Facts
  62. 62. @cyetain Theories Opinions Hypothesizes Constraints Criteria
  63. 63. @cyetain Theories Opinions Hypothesizes Question Facts
  64. 64. @cyetain Theories Opinions Hypothesizes Request More Information
  65. 65. @cyetain NO TALKING!
  66. 66. @cyetain How Would I Validate my understanding of this problem? How Would I solve this Problem? •Based on your experiences, what would you do to solve this problem? This is your Hypothesis. •Identify What Needs to Be True if your Hypothesis is true. •Assert, Presume, Assume Truth •Imagine Experiments that would justify the Assumptions
  67. 67. @cyetain I Assert that this I know this 0% 100% Probability of Failure 50% PotentialInformation Pretty Sure theory is wrong Pretty Sure theory is right Interesting Ideas I Presume Somebody knows this I am going to Assume this is true for my Hypothesis to be true 0% 100% Probability of Failure 50% PotentialInformation Pretty Sure theory is wrong Pretty Sure theory is right Interesting Ideas 0% 100% Probability of Failure 50% PotentialInformation Pretty Sure theory is wrong Pretty Sure theory is right Interesting Ideas
  68. 68. @cyetain This is my Hypothesis, Assumptions and Experiments Challenge Assumptions & Experiments Rotate Pairs 2-3 Times Allow Time for Revision Between Rounds
  69. 69. @cyetain This is my Hypothesis, Assumptions and Experiments
  70. 70. @cyetain Multiple Smaller Experiments against Multiple Abductive Hypotheses instead of Single Large Experiment against Single Hypotheses
  71. 71. @cyetain Failing Well Produces more Information than Failing Poorly
  72. 72. @cyetain What are You Doing w All that Information? Incremental: Confirm. Disconfirm. Iteratively: Select Next Step. Generate More Options
  73. 73. @cyetain Having “Passionate Beliefs, Loosely Held” FAILURE MUST BE AN OPTION
  74. 74. @cyetain Reducing Variability too soon risks suboptimal result, too late increases failure demand
  75. 75. @cyetain
  76. 76. @cyetain Influences & Sources of More Information
  77. 77. @cyetain Joshua (Jabe) Bloom CTO : The Library Corporation & TLC Labs blog http://jabe.co Send Anonymous Feedback http://sayat.me/jabebloom
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×