Failing Well Agile 2013

1,682 views

Published on

There is a trap hidden inside Lean Startup's CustDev cycle. If entrepreneurs want to be successful, they have to be passionate about their ideas. They need to understand more about their customers than anyone else. I’ve spent the last year lecturing to and teaching entrepreneurs and I've observed that their passion for their idea and their belief that they already DO know everything about their customers can prevent them from actually LEARNING what they need to know in order to create as successful business. The same passion and positive psychology required to succeed in the face of uncertainty is hindering them from learning fast enough to survive.

How can we take our passion, our vision, a couple “wild ass guesses”, and produce meaningful, validated learning?

The question of how to learn as an organization and how to DEMONSTRATE learning has been explored by philosophers of science and by business theorists for years. What can the Lean Startup Community learn about creating scientifically valid experiments that create actionable knowledge?

Learn how to fail well and fail faster by keeping your passion focused on the vision and our dispassionate logic focused on the assumptions.

Published in: Technology, Education

Failing Well Agile 2013

  1. 1. blog: jabe.co FAILING principles and practices of Vanity Validation a Paradox of Passionate Commitment WELL
  2. 2. @cyetain What was the last thing you failed at? What did you learn? Did you share your failure with anyone? Pre-Talk Questions
  3. 3. blog: jabe.co FAILING principles and practices of Vanity Validation a Paradox of Passionate Commitment WELL
  4. 4. HELLLOOOO McFLY TLC LABS blog http://jabe.co Send Anonymous Feedback http://sayat.me/jabebloom Joshua (Jabe) Bloom CTO : The Library Corporation & Consulting Practioner TLC Labs #agile2013
  5. 5. @cyetain Fail Fast
  6. 6. @cyetain
  7. 7. @cyetain Learning occurs when we detect and correct error. Error is any mismatch between what we intend an action to produce and what actually happens when we implement that action. -Chris Argyris
  8. 8. @cyetain How Do We Make Better Choices? Why is it so hard to Fail? Could we design a system to help?
  9. 9. @cyetain One must treat his theory-in- use as both a psychological certainty and an intellectual hypothesis. -Chris Argyris
  10. 10. @cyetain How wonderful that we have met with a paradox. Now we have some hope of making progress. -Niels Bohr
  11. 11. @cyetain
  12. 12. @cyetain fake dictionary page from colbertnation.com
  13. 13. @cyetain Although theory without experiment is empty, experiment without theory is blind. -Paul Thagard
  14. 14. @cyetain 3 Things To Leave With • Failing Well Produces more Information than Failing Poorly • “Passionate Beliefs, Loosely Held” • Reducing Variability too, Soon risks suboptimal result, too Late increases Failure blindness
  15. 15. @cyetain We simply cannot rely on randomness to correct the problems that randomness creates. -Don Reinertsen
  16. 16. @cyetain undifferentiated streams of data
  17. 17. @cyetain
  18. 18. @cyetain “Research is what I’m doing when I don’t know what I’m doing.” -Wernher von Braun
  19. 19. @cyetain
  20. 20. @cyetain The Principle of Optimum Failure Rate 0% 100% Probability of Failure 50% PotentialInformation
  21. 21. @cyetain 0% 100% Probability of Failure 50% PotentialInformation Greater Asserted Information Greater Asserted Information
  22. 22. @cyetain 0% 100% Probability of Failure 50% PotentialInformation Greater Asserted Information Greater Asserted Information Pretty Sure theory is wrong Pretty Sure theory is right Interesting Ideas
  23. 23. @cyetain 0% 100% Probability of Failure 50% PotentialInformation Greater Asserted Information Greater Asserted Information Pretty Sure theory is wrong Pretty Sure theory is right Interesting Ideas Uncomfortable Confident
  24. 24. @cyetain Experience of Failure NumberofSamples The Competency Trap
  25. 25. @cyetain 0% 100% Probability of Failure 50% PotentialInformation
  26. 26. @cyetain 0% 100% Probability of Failure 50% PotentialInformation Pretty Sure theory is wrong Pretty Sure theory is right Interesting IdeasHidden Risk Hidden Value The Line of SURPRISE!
  27. 27. @cyetain 0% 100% Probability of Failure 50% PotentialInformation Pretty Sure theory is wrong Pretty Sure theory is right Interesting IdeasHidden Risk Hidden Value 1 2 3 During Customer Development Focus on Interesting Ideas Before Scaling Validate Your "We Know This Assumptions" to reduce risk of Failure Demand After Customer Validation Run experiments to Validate Assumptions of Failure 2 3 1
  28. 28. @cyetain “The typical sequence of coin tosses has high information content but little value; an ephemeris, giving the positions of the moon and planets every day for a hundred years, has no more information than the equations of motion and initial conditions from which it was calculated, but saves it’s owner the effort of recalculating these positions.” -Charles H. Bennett
  29. 29. @cyetain Based on what we know right now, what problems do we have the least amount of information about that we can reasonably expect to understand?
  30. 30. @cyetain
  31. 31. @cyetain Risk vs Uncertainty
  32. 32. @cyetain Alteaory vs Epistemic Uncertainties
  33. 33. @cyetain Gamble Invest
  34. 34. @cyetain Justified MVP Value of Information Cost of Acquisition Cost of MVP Unjustified MVP Over Justified MVP Justified MVP
  35. 35. @cyetain The first principle is that you must not fool yourself--and you are the easiest person to fool. -Richard Feynman
  36. 36. @cyetain “Most people don’t know how to learn. What’s more, those members of the organization that many assume to be the best at learning are, in fact, not very good at it. I am talking about the well-educated, high-powered, high commitment professionals” -Chris Argyris
  37. 37. @cyetain “Expertise … breeds an inability to accept new views.” -Laski
  38. 38. @cyetain
  39. 39. @cyetain VanityValidation
  40. 40. @cyetain I need to be right even if I'm wrong.
  41. 41. @cyetain Defensive Reasoning
  42. 42. @cyetain Remain in unilateral control
  43. 43. @cyetain Maximize "winning" Minimize "losing"
  44. 44. @cyetain Suppress negative feelings
  45. 45. @cyetain Be as "rational" as possible -- by which people mean defining clear objectives and evaluating their behavior in terms of whether or not they have achieved them
  46. 46. @cyetain Mindset Actions Results Match Results Mismatch Single-loop Double-loop
  47. 47. @cyetain Valid Public Information
  48. 48. @cyetain
  49. 49. @cyetain whenever we propose a solution to a problem, we ought to try as hard as we can to overthrow our solution, rather than defend it. -Karl Popper
  50. 50. @cyetain • Identify Your Assumptions and Conclusions CLEARLY AS POSSIBLE PUBLICLY • Question Your Assumptions and Conclusions • Seek Contrary Data • Learn when to correct your Actions and when to correct your Mindset
  51. 51. @cyetain http://xkcd.com/
  52. 52. @cyetain
  53. 53. @cyetain Abduction not just for Aliens @cyetain
  54. 54. @cyetain [Abduction] goes upon the hope that there is sufficient affinity between the reasoner's mind and nature's to render guessing not altogether hopeless, provided each guess is checked by comparison with observation... The effort should therefore be to make each hypothesis... as near an even bet as possible. -Charles Peirce
  55. 55. @cyetain ABDUCE DEDUCE INDUCE Predictive Probable Plausible The Way Computers "Think" The Way Humans Think Binary Probability Analogue Justifiable
  56. 56. @cyetain ABDUCE DEDUCE INDUCE Experiences Hypothesises Expected Outcomes If Coherent If Expected Outcomes Match Reality Effective Match
  57. 57. @cyetain ABDUCE DEDUCEINDUCE
  58. 58. @cyetain ABDUCE DEDUCEINDUCE SURPRISE!!!
  59. 59. @cyetain Multi-Hypothesis Research !=
  60. 60. @cyetain BRAINSTORM
  61. 61. @cyetain Theories Opinions Hypothesizes The Facts and Just the Facts
  62. 62. @cyetain Theories Opinions Hypothesizes Constraints Criteria
  63. 63. @cyetain Theories Opinions Hypothesizes Question Facts
  64. 64. @cyetain Theories Opinions Hypothesizes Request More Information
  65. 65. @cyetain NO TALKING!
  66. 66. @cyetain How Would I Validate my understanding of this problem? How Would I solve this Problem? •Based on your experiences, what would you do to solve this problem? This is your Hypothesis. •Identify What Needs to Be True if your Hypothesis is true. •Assert, Presume, Assume Truth •Imagine Experiments that would justify the Assumptions
  67. 67. @cyetain I Assert that this I know this 0% 100% Probability of Failure 50% PotentialInformation Pretty Sure theory is wrong Pretty Sure theory is right Interesting Ideas I Presume Somebody knows this I am going to Assume this is true for my Hypothesis to be true 0% 100% Probability of Failure 50% PotentialInformation Pretty Sure theory is wrong Pretty Sure theory is right Interesting Ideas 0% 100% Probability of Failure 50% PotentialInformation Pretty Sure theory is wrong Pretty Sure theory is right Interesting Ideas
  68. 68. @cyetain This is my Hypothesis, Assumptions and Experiments Challenge Assumptions & Experiments Rotate Pairs 2-3 Times Allow Time for Revision Between Rounds
  69. 69. @cyetain This is my Hypothesis, Assumptions and Experiments
  70. 70. @cyetain Multiple Smaller Experiments against Multiple Abductive Hypotheses instead of Single Large Experiment against Single Hypotheses
  71. 71. @cyetain Failing Well Produces more Information than Failing Poorly
  72. 72. @cyetain What are You Doing w All that Information? Incremental: Confirm. Disconfirm. Iteratively: Select Next Step. Generate More Options
  73. 73. @cyetain Having “Passionate Beliefs, Loosely Held” FAILURE MUST BE AN OPTION
  74. 74. @cyetain Reducing Variability too soon risks suboptimal result, too late increases failure demand
  75. 75. @cyetain
  76. 76. @cyetain Influences & Sources of More Information
  77. 77. @cyetain Joshua (Jabe) Bloom CTO : The Library Corporation & TLC Labs blog http://jabe.co Send Anonymous Feedback http://sayat.me/jabebloom

×