Experimenting in the Enterprise Agile 2013 Conference

4,226 views

Published on

Published in: Technology, Business
0 Comments
23 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,226
On SlideShare
0
From Embeds
0
Number of Embeds
814
Actions
Shares
0
Downloads
111
Comments
0
Likes
23
Embeds 0
No embeds

No notes for slide
  • Overall, pedometer users increased their physical activity by 26.9% over baseline. An important predictor of increased physical activity was having a step goal such as 10,000 steps per day (P = .001). When data from all studies were combined, pedometer users significantly decreased their body mass index by 0.38 (95% CI, 0.05-0.72; P = .03)." Promoting Lifestyle Physical Activity: Experiences with the First Step Program.Am J LifestyleMed. 2009 Jul 1;3(1 Suppl):508-548.
  • Overall, pedometer users increased their physical activity by 26.9% over baseline. An important predictor of increased physical activity was having a step goal such as 10,000 steps per day (P = .001). When data from all studies were combined, pedometer users significantly decreased their body mass index by 0.38 (95% CI, 0.05-0.72; P = .03)." Promoting Lifestyle Physical Activity: Experiences with the First Step Program.Am J LifestyleMed. 2009 Jul 1;3(1 Suppl):508-548.
  • Experimenting in the Enterprise Agile 2013 Conference

    1. 1. Experimenting Enterprise © 2013 BigVisible Solutions in the
    2. 2. 2© 2012 BigVisible Solutions David J Bland Startups consulted: 18 Twitter followers: 2179 Iterations of Coaching: 215 Speaker Vanity Metrics Brian Bozzuto Blogs & Articles Written: 77 Presentations Delivered: 46 Teams coached: 54
    3. 3. © 2013 BigVisible Solutions 3 This session doesn’t guarantee a 100 million dollar success, but it should help prevent a 100 million dollar failure.
    4. 4. © 2012 BigVisible Solutions 4 A business model used to be static and survived many years. 38 13 4 3 2 0 10 20 30 40 Radio TV Internet iPod Facebook Years to Reach a Market of 50 Million
    5. 5. © 2012 BigVisible Solutions 5
    6. 6. © 2012 BigVisible Solutions 6 The Lean Startup provides a scientific approach to creating and managing startups and get a desired product to customers’ hands faster. -Eric Ries
    7. 7. Entrepreneur Intrapreneur © 2013 BigVisible Solutions 7
    8. 8. Entrepreneur Intrapreneur Skills © 2013 BigVisible Solutions 8
    9. 9. © 2013 BigVisible Solutions 9 This is great in theory, but it’s not for B2B. This is great in theory, but I’m a visionary. EntrepreneurIntrapreneur
    10. 10. © 2013 BigVisible Solutions 10 Define the value proposition for: Source: Adapted from Steve Blank Buyer User Decider Buyer User Decider B2C B2B
    11. 11. © 2013 BigVisible Solutions 11 Are your users/customers: Source: Adapted from Steve Blank Aware of problem? Actively seeking a solution? Hacking together a solution? Have $$$ for your solution?
    12. 12. © 2013 BigVisible Solutions 12 Can We Build It? Should We Build It?
    13. 13. © 2013 BigVisible Solutions 13 Requirements = Hypotheses
    14. 14. © 2013 BigVisible Solutions 14 Source: Adapted from Cynefin Simple “Everyone Knows That” Complicated “Let’s consult the expert” Complex “Whoa, it all makes sense now…” Chaos “That still doesn’t make sense” Work with experts to inform our hypothesis Run experiments to gain new knowledge
    15. 15. © 2013 BigVisible Solutions 15 Safe Fail != Fail Safe
    16. 16. © 2012 BigVisible Solutions 16 Experiment with product & process
    17. 17. © 2013 BigVisible Solutions 17 If you experiment just to see what happens, then you’ll succeed at seeing what happens.
    18. 18. © 2013 BigVisible Solutions 18 Be careful how you structure your experiments
    19. 19. © 2013 BigVisible Solutions 19 Image Source: Eric Ries The Lean Startup Then here Start here Last here
    20. 20. © 2013 BigVisible Solutions 20 Image Source: Eric Ries The Lean Startup Now Execute Quickly
    21. 21. © 2012 BigVisible Solutions 21 Experiment Template For <customer segment> I predict <outcome> when I run <experiment>
    22. 22. © 2012 BigVisible Solutions 22 Experiment Example For new customers, I predict 10% increase in conversion when I offer a “no questions asked” cancellation policy
    23. 23. © 2012 BigVisible Solutions 23 Experiment Guidelines • Customer Segment (who) • Up to / Limit Of (how many) • Length of Time (how long) • Financial Exposure (how much) • Amplification & Dampening Measures
    24. 24. © 2013 BigVisible Solutions 24 Experiment Dashboard ID Hypothesis Description Status Next Steps 01a We predict that adding a refund feature will increase conversions Story 13c adds refund label and call to action button that puts customer in contact with call center agent. Complete 0.2% increase in conversion on cohort. Meeting with VP of Ops to discuss strategy.
    25. 25. © 2013 BigVisible Solutions 25 “This is all great, but how do we know what experiments to run?”
    26. 26. © 2012 BigVisible Solutions 26 Iterative Product + Static Business Model Failed Organization
    27. 27. © 2012 BigVisible Solutions 27Source: Alex Osterwalder
    28. 28. © 2012 BigVisible Solutions 28 ? ? ??? ? ? ? ? ? ? ? “Let’s map out our existing business model” Source: Adapted from Alex Osterwalder tip: tell a story
    29. 29. © 2012 BigVisible Solutions 29 ? ? ??? ? ? ? ? ? ? ? “Let’s validate our riskiest assumptions” Source: Adapted from Alex Osterwalder
    30. 30. © 2012 BigVisible Solutions 30 This is most likely your Riskiest Assumption Source: Adapted from Alex Osterwalder
    31. 31. © 2012 BigVisible Solutions 31 This is most likely your other Riskiest Assumption Source: Adapted from Alex Osterwalder
    32. 32. © 2012 BigVisible Solutions 32 What happens when we subtract costs from revenue? Source: Adapted from Alex Osterwalder
    33. 33. © 2013 BigVisible Solutions 33 “Okay, now we have a big list of assumptions, how do we decide where to start?”
    34. 34. © 2013 BigVisible Solutions 34 experiment experiment experiment experiment experiment experiment experiment “Let’s dot vote!” (you could, but that’s pretty subjective)
    35. 35. © 2013 BigVisible Solutions 35 time learning experiment experiment experiment experiment experiment experiment experiment experiment experiment experiment experiment experiment experiment (ok better, but what are they tied to?)
    36. 36. © 2013 BigVisible Solutions 36 Now we have a focusing mechanism focustime time learninglearning experiments experiments experiments experiments
    37. 37. © 2013 BigVisible Solutions 37 And assumptions that need validation focus experiments experiments experiments experiments time time learninglearning
    38. 38. © 2013 BigVisible Solutions 38 This is Experiment Mapping focus experiments experiments experiments experiments time time learninglearning
    39. 39. © 2013 BigVisible Solutions 39 Customer Segment is our focus focus experiments experiments experiments experiments time time learninglearning Customer Segment
    40. 40. © 2013 BigVisible Solutions 40 Will they pay for our product? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product
    41. 41. © 2013 BigVisible Solutions 41 Can we scale to meet demand? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product We can scale to meet demand
    42. 42. © 2013 BigVisible Solutions 42 Can we use our distribution channels? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product We can use existing distribution channels We can scale to meet demand
    43. 43. © 2013 BigVisible Solutions 43 Are we solving a real need? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand
    44. 44. © 2013 BigVisible Solutions 44 Which of these is the riskiest? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand
    45. 45. © 2013 BigVisible Solutions 45 Can we do a concierge test? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand Concierge test with 10 customers
    46. 46. © 2013 BigVisible Solutions 46 We could build a prototype focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand Concierge test with 10 customers Build a prototype
    47. 47. © 2013 BigVisible Solutions 47 Maybe run a focus group? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand Concierge test with 10 customers Build a prototype Run a focus group with 80 people
    48. 48. © 2013 BigVisible Solutions 48 Prototype will take time focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand Concierge test with 10 customers Build a prototype Run a focus group with 80 people
    49. 49. © 2013 BigVisible Solutions 49 Focus groups take a long time too focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand Concierge test with 10 customers Build a prototype Run a focus group with 80 people
    50. 50. © 2013 BigVisible Solutions 50 Concierge test is quick & we’ll learn focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand Concierge test with 10 customers Build a prototype Run a focus group with 80 people
    51. 51. © 2013 BigVisible Solutions 51 “Let’s swarm on this experiment” high learning short timeframe
    52. 52. © 2013 BigVisible Solutions 52 Just because you can measure it, does not mean you should measure it.
    53. 53. © 2013 BigVisible Solutions 53 Auditable Actionable Accessible Source: Eric Ries The Lean Startup
    54. 54. © 2013 BigVisible Solutions 54 Auditable “Be able to defend your metrics with confidence”
    55. 55. © 2013 BigVisible Solutions 55 Auditable? zombie project metrics
    56. 56. © 2013 BigVisible Solutions 56 Actionable “Does this metric incentivize a change in behavior?”
    57. 57. © 2013 BigVisible Solutions 57 Actionable? Facebook likes won’t influence your dog
    58. 58. © 2013 BigVisible Solutions 58 Actionable Number of steps should influence your lifestyle
    59. 59. 0 50 100 150 200 250 1 2 3 4 5 6 7 8 9 10 11 Velocity(Points) Iteration Team Velocity per Iteration Delta Planned Actual Avg Last 3 © 2013 BigVisible Solutions 59 Actionable? What does this tell you about your product?
    60. 60. © 2013 BigVisible Solutions 60 Accessible “Can you easily access and share this data?”
    61. 61. © 2013 BigVisible Solutions 61 Accessible? Not accessible
    62. 62. © 2013 BigVisible Solutions 62 Accessible Accessible Image Source: Lean.st
    63. 63. © 2013 BigVisible Solutions 63 Source: Dave McClure Pirate Metrics Acquisition Activation Retention Referral Revenue
    64. 64. © 2013 BigVisible Solutions 64 Source: Dave McClure Pirate Metrics Acquisition Activation Retention Referral Revenue
    65. 65. © 2013 BigVisible Solutions 65 “Metrics are people, too” Source: Eric Ries The Lean Startup
    66. 66. © 2013 BigVisible Solutions 66 Aware Hopeful Satisfied Passionate Source: Adapted from Cooper & Vlaskovits
    67. 67. © 2013 BigVisible Solutions 67 Aware Hopeful Satisfied Passionate Acquisition Activation Retention Referral
    68. 68. © 2012 BigVisible Solutions 68 Start small.
    69. 69. © 2012 BigVisible Solutions 69 When you have stories around an uncertain feature, add a hypothesis that states the expected outcome on a specific metric.
    70. 70. © 2012 BigVisible Solutions 70 If you have a Kanban Board, add a Validate Column to the very right of it.
    71. 71. © 2012 BigVisible Solutions 71 Problem Solution Process Known Known Waterfall or Agile Known Unknown Agile Unknown Unknown Lean Startup
    72. 72. © 2012 BigVisible Solutions 72 Where are the unknown / unknown’s in your enterprise?
    73. 73. David J Bland Experimentor Brian Bozzuto Principal 73© 2013 BigVisible Solutions Thank You!

    ×