Lean Product Management The Value of Experiments                                                   @DaveSharrock          ...
Assembly Line Manufacturing    has a high cost of change                   http://www.archives.gov.on.ca/english/on-line-e...
Inclusive thinking                                                                                       Feature usage in•...
Cost of                                                                          HypothesizedChange                       ...
Software Development has                                                                   commoditized cost of change    ...
Cost of                                                                                         HypothesizedChange        ...
Changing paradigms• If change is cheap, requirements  can change continuously• We can evolve our thinking as  we learn mor...
Cost of                                                                                         HypothesizedChange        ...
Shifting Paradigms                                                           B.G.|A.G.“I know that half of   my advertisin...
Validated Learning                                                                   • Understand the problem a           ...
The Scientific Method                                                                    Theory             Use the theory ...
The Theory of General RelativityGradual acceptance ofGeneral Relativity over                                              ...
Applied to an Online Experience                                                                   User Model         Use t...
Designed by KISSmetricsagile42 | We advise, train and coach companies building software   www.agile42.com |   All rights r...
User Story (Journey) Mapping                                                                      Elements of Customer Jou...
Vanity vs. Actionable MetricsVanity metrics (like web hits or number ofdownloads) which only serve todocument the current ...
http://500hats.typepad.com/500blogs/2007/09/startup-metrics.htmlagile42 | We advise, train and coach companies building so...
DepersonalizeDecision Making• “HiPPO’s” rule the business world• It is never about you, it can’t be  about you    • Benchm...
“I can’t say it any better, HiPPO’s rule the world, they over ruleyour data, they impose their opinions on you and yourcom...
Interaction Model                          Backlog             Validation                              Execution          ...
How Much Effort on Validation                                                                                    Feature u...
Feedback loops makeall the difference In 1940, the chance of anAmerican being killed bylightning was about 1 in400,000.Tod...
Lean Product Management The Value of Experiments • Validate requirements trump everything • Incremental cost of validation...
“Coming together is a beginning.  Keeping together is progress.  Working together is success.”          Henry Ford        ...
Upcoming SlideShare
Loading in …5
×

ProductCamp Vancouver 2013

672 views
605 views

Published on

Product management as a discipline is undergoing a transition towards data-driven decision making. As the cost of testing product feature ideas falls, the role of a product manager becomes one based on testing many feature ideas, and rolling out proven features. In this presentation we touch on some of the trends behind this paradigm shift, and consider some tools the product manager can use.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
672
On SlideShare
0
From Embeds
0
Number of Embeds
87
Actions
Shares
0
Downloads
9
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

ProductCamp Vancouver 2013

  1. 1. Lean Product Management The Value of Experiments @DaveSharrock ProductCampVancouver 2013Photo: ESAhttp://www.nbcnews.com/id/50677062/ns/technology_and_science-space reserved. Copyright ©2007-2013 agile42 | We advise, train and coach companies building software www.agile42.com | All rights
  2. 2. Assembly Line Manufacturing has a high cost of change http://www.archives.gov.on.ca/english/on-line-exhibits/d-day/big/big_03_airline_assembly.aspx Archives of Ontario, Reference Code: C 190-5-0-0-21agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  3. 3. Inclusive thinking Feature usage in• In software development high cost of change leads to inclusive enterprise software thinking Never Rarely Sometimes Often Always• Any and every idea has to be 7% captured in the first version of a 13% requirements specification• Creates waste - bloated 45% documents, unwanted features 16% and entitlement thinking 19% Standish Group - CHAOS Report, 1995, 2002agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  4. 4. Cost of HypothesizedChange Requirements Capture all possible needs Detail of Requirementsagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  5. 5. Software Development has commoditized cost of change Object-oriented languages jquery Ruby on Rails Coffeescript Continuous Deliveryagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  6. 6. Cost of HypothesizedChange Requirements Evolving Requirements Capture all possible needs Emerging needs as development progresses Detail of Requirementsagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  7. 7. Changing paradigms• If change is cheap, requirements can change continuously• We can evolve our thinking as we learn more about the product we are building• Maximizing Return-on- Investment means validating new features early, before the total investment cost has been spentagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  8. 8. Cost of HypothesizedChange Requirements Evolving Requirements Validated Requirements Capture all possible needs Emerging needs as development progresses Lean Startup experiments Detail of Requirementsagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  9. 9. Shifting Paradigms B.G.|A.G.“I know that half of my advertising budget is wasted, but I’m not sure which half” Lord Leverhulme http://www.marketingprofs.com/charts/2009/3161/more-smbs-use-online-media-than-traditional http://marketing.blogs.ie.edu/archives/2011/01/trends-in-online-advertising-in-2011-in-the-united-states.phpagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  10. 10. Validated Learning • Understand the problem a feature will solve • Decide how to ask if your customer cares • Validate before investing complete costagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  11. 11. The Scientific Method Theory Use the theory to make a prediction Modify or change your theory Observe Prediction Design an experiment to test the Run the prediction experiment Experimentagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  12. 12. The Theory of General RelativityGradual acceptance ofGeneral Relativity over Einstein’s Theory of General Relativity Newtonian Mechanics Einstein’s prediction (1907) Observations validated Light bends in a General Relativity gravitational field Wait for 1919 solar Arthur Eddington eclipse to observeobserved that Light did whether or not light bend around the sun bends around the sun Solar Eclipse of 1919 http://thethoughtstash.wordpress.com/2011/01/03/how-eddington-demonstrated-that-einstein-was-right/agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  13. 13. Applied to an Online Experience User Model Use the model to make a prediction Modify or change your user model Validate Prediction Design an experiment to test the Run the prediction experiment Experimentagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  14. 14. Designed by KISSmetricsagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  15. 15. User Story (Journey) Mapping Elements of Customer Journey Different Customer Your model for Experiences customer interaction covers each step of the customer Each row matches a different journey customer experienceStory Mapping © Jeff Patton, all rights reserved, www.AgileProductDesign.com agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  16. 16. Vanity vs. Actionable MetricsVanity metrics (like web hits or number ofdownloads) which only serve todocument the current state of theproduct but offer no insight into how wegot here or what to do next. An actionable metric is one that ties specific and repeatable actions to observed results.agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  17. 17. http://500hats.typepad.com/500blogs/2007/09/startup-metrics.htmlagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  18. 18. DepersonalizeDecision Making• “HiPPO’s” rule the business world• It is never about you, it can’t be about you • Benchmarking is awesome  • Leverage competitive analysis  • Experimentation / testing rock• Execution strategy:  • Transparency, standardization, looking outside in  • Be a slave to customer centricity (its about your customers (internal & external))agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  19. 19. “I can’t say it any better, HiPPO’s rule the world, they over ruleyour data, they impose their opinions on you and yourcompany customers, they think they know best (sometimes theydo), their mere presence in a meeting prevents ideas fromcoming up. The solution to this problem is to depersonalize decisionmaking, simply don’t make it about you or what you think. Gooutside, get context from other places. Include external orinternal benchmarks in your analysis. Get competitive data (weare at x% of zz metric and our competition is at x+9% of zzmetric).Be incessantly focussed on your company customers anddragging their voice to the table (for example viaexperimentation and testing or via open ended surveyquestions). Very few people, HiPPO’s included, can argue with acustomer’s voice, the customer afterall is the queen / king! : )” Avinesh Kaushik, kaushik.netagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  20. 20. Interaction Model Backlog Validation Execution To Do Doing Observation Validated? Doing Deployed For <target group> Who <has a specific need> Idea #1.1 The <product name> Is A <product category> That <key benefit> Idea Unlike <primary competitors> #1.2 Our Product <additional advantages> Idea Idea #1.3 Idea #1.5 Pathway #1.4 #1 Idea Idea #1.6 #2.1 Idea Pathway #2.4 Idea Idea #2 #2.3 #2.2 Level of Detail / Close to done•Visualize your User Interaction Model •Always revisit validation tests for size •The execution board is equivalent toand adjust it as you learn the team’s task board, but at the Idea•Come up with ideas for each pathway Card level•Prioritize experimental testing•Spend time on measurementinfrastructure (a test without ability tomeasure = an unvalidated feature)
  21. 21. How Much Effort on Validation Feature usage in• In 2009, Google ran approximately 12,000 enterprise software experiments, of which about Never Rarely 10% led to business changes Sometimes Often Always 7%• Linus Pauling famously said, 13% "The best way to have a good idea is to have a lot of ideas." 45%• Focus on minimizing the 16% incremental cost of testing ideas with your customers/users 19% Standish Group - CHAOS Report, 1995, 2002agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  22. 22. Feedback loops makeall the difference In 1940, the chance of anAmerican being killed bylightning was about 1 in400,000.Today it’s 1 in 11 million. In November 2007, economists in the Survey of Professional Forecasters — examining some 45,000 economic-data series — foresaw less than a 1-in-500 chance of an economic meltdown as severe as the http://www.nytimes.com/2012/09/09/ magazine/the-weatherman-is-not-a- one that would begin one moron.html month later.agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
  23. 23. Lean Product Management The Value of Experiments • Validate requirements trump everything • Incremental cost of validation is falling • Feedback means nothing without a model The race doesn’t go to the one with the best idea, but to the one with the most tested ideasPhoto: ESAhttp://www.nbcnews.com/id/50677062/ns/technology_and_science-space reserved. Copyright ©2007-2013 agile42 | We advise, train and coach companies building software www.agile42.com | All rights
  24. 24. “Coming together is a beginning. Keeping together is progress. Working together is success.” Henry Ford thank you dave.sharrock@agile42.com skype: dave.sharrock twitter: @davesharrock slides: slideshare.net/davesharrockagile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright © 2007 - 2009.

×