Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Experimental Product Development

998 views

Published on

This class will present hypothesis-driven development, the cutting-edge paradigm for evolving validated products. We’ll dive into how to frame hypotheses, design experiments, and use A/B testing to gather data to prove or disprove our ideas.

Published in: Leadership & Management

Experimental Product Development

  1. 1. i290 lean/agile product management unit 6: experimental product development @jezhumble https://lapm.continuousdelivery.com/ humble@berkeley.edu This work © 2015-2016 Jez Humble Licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
  2. 2. identify experiments to test hypotheses understand how to do outcome-based planning describe hypothesis-driven development understand why small batches are important define A/B testing and the culture it enables learning outcomes
  3. 3. Epic Theme Story
  4. 4. impact mapping Gojko Adzic, Impact Mapping
  5. 5. working backwards http://www.allthingsdistributed.com/2006/11/working_backwards.html
  6. 6. @jezhumbleJeff Gothelf “Better product definition with Lean UX and Design” http://bit.ly/TylT6A hypothesis-driven delivery We believe that [building this feature] [for these people] will achieve [this outcome]. We will know we are successful when we see [this signal from the market].
  7. 7. COST OF EXPERIMENTS 8 Production Software SPEED COST new services feasibility spike service substitution integration Quantitative forecasting real-time price experiment Data sampling and modeling tests Sketches & Paper Prototypes Interactive Prototype Software demo Interviews & surveys micro-niche Wizard of Oz VIABILITY (BUSINESS) | DESIRABILITY (CUSTOMER) | FEASIBILITY (TECH)
  8. 8. exercise • choose a hypothesis from your assignment • design an experiment to test your hypothesis • what do you expect the results to be? • what result will confirm your hypothesis? • what result will disprove your hypothesis? • how soon can we get the result?
  9. 9. “Etsy’s Product Development with Continuous Experimentation” Frank Harris and Nellwyn Thomas | http://bit.ly/19Z5izI
  10. 10. “Etsy’s Product Development with Continuous Experimentation” Frank Harris and Nellwyn Thomas | http://bit.ly/19Z5izI
  11. 11. “Etsy’s Product Development with Continuous Experimentation” Frank Harris and Nellwyn Thomas | http://bit.ly/19Z5izI
  12. 12. Jon Jenkins, “Velocity Culture, The Unmet Challenge in Ops” 2011 | http://bit.ly/1vJo1Ya
  13. 13. do less “Evaluating well-designed and executed experiments that were designed to improve a key metric, only about 1/3 were successful at improving the key metric!” “Online Experimentation at Microsoft” | Kohavi et al | http://stanford.io/130uW6X
  14. 14. “I think building this culture is the key to innovation. Creativity must flow from everywhere. Whether you are a summer intern or the CTO, any good idea must be able to seek an objective test, preferably a test that exposes the idea to real customers. Everyone must be able to experiment, learn, and iterate.” http://glinden.blogspot.com/2006/04/early-amazon-shopping-cart.html
  15. 15. @jezhumble WORK IN SMALL BATCHES WORK IN SMALL BATCHES WORK IN SMALL BATCHES WORK IN SMALL BATCHES WORK IN SMALL BATCHES WORK IN SMALL BATCHES WORK IN SMALL BATCHES WORK IN SMALL BATCHES WORK IN SMALL BATCHES
  16. 16. higher quality you can stop at any time with a working system faster feedback (assuming people pay attention) higher motivation less rework working in small batches Don Reinertsen, Principles of Product Development Flow, ch5.
  17. 17. further reading https://www.infoq.com/presentations/controlled-experiments http://xp123.com/articles/resources-on-set-based-design/ Tom DeMarco & Tim Lister, Waltzing with Bears Humble et al, Lean Enterprise ch 9 Gojko Adzic, Impact Mapping

×