Experimenting Your Way to Success: Applying Lean Startup Principles to Product Development at IMVU
•
18 likes•4,983 views
How IMVU uses experiments to ship products that customers love, and to improve the way we build those products. Key points include company culture and methods of learning, including 5 Whys, project postmortems, and sprint retrospectives. Discussion is framed within the Lean Startup Build-Measure-Learn loop. Roots of the Lean Startup method in the scientific method are discussed.
Recommended
More Related Content
Viewers also liked
Viewers also liked (12)
Similar to Experimenting Your Way to Success: Applying Lean Startup Principles to Product Development at IMVU
Similar to Experimenting Your Way to Success: Applying Lean Startup Principles to Product Development at IMVU (20)
Recently uploaded
Recently uploaded (20)
Experimenting Your Way to Success: Applying Lean Startup Principles to Product Development at IMVU
- 1. James Birchler Engineering Director, IMVU GDC, San Francisco, March 3, 2011
- 3. 1. Experiments 2. Product Features 3. product development
- 4. The scientific method is based on
- 8. @copernicus
- 9. #inquisition
- 10. @copernicus
- 11. @Giordano_Bruno
- 12. @Galileo
- 14. @Jamesbirchler
- 15. $ IMVU Revenue Growth Qtr Q1'06 Q2'06 Q3'06 Q4'06 Q1'07 Q2'07 Q3'07 Q4'07 Q1'08 Q2'08 Q3'08 Q4'08 Q1'09 Q2'09 Q3'09 Q4'09
- 17. Ask Question Got Do Research Hypothesis Science? Test Analyze Data Conclusion Report Results
- 18. Question Results Research Scientific Conclusion Method Hypothesis Analyze Data Test
- 20. Question Results Research Quack. Scientific #leanstartup Conclusion Method Hypothesis Analyze Data Test An Alleged “duck”
- 22. Okay, I’ll run an Okay, I’ll run an Okay, I’ll run an experiment! Okay, I’ll run experiment! Okay, I’ll run an experiment! experiment experiment! Okay, I’ll run an experiment! Let’s keep this Okay, I’ll run an simple! Okay, I’ll run an experiment! Prove your idea experiment! is the best!
- 23. Running experiments is fun!
- 24. Ask Question Do Research Hypothesis Test Analyze Data Conclusion Report Results Looks a lot like the scientific method (it is!)…
- 25. Keep it simple
- 26. if( setup_experiment(...) == "control" ) { // do it the old way } else { // do it the new way Simple To code }
- 27. Simple To manage
- 28. Simple To share
- 29. embrace failure
- 37. Measure
- 38. Measure
- 47. 5 whys
- 48. 5 whys Root Cause: Fix this!
- 52. Dogma
- 60. Ideal Days
- 61. Ideal Days
- 66. Hack Week @IMVU…
- 68. Recap…
- 70. credits Thanks
Editor's Notes
- I’m James Birchler, Engineering Director at IMVU, and today we’re going to talk about how IMVU uses experiments and Lean Startup methods to build products our customers love, and to innovate and improve our product development process.
- What are experiments, anyway? Experiments are one way to learn—about the world, or about your business. What makes scientific experimentation different is that you pay attention to a lot of details—like how you set up your experiment to ensure your testing the right things, or whether your results just look good, or if they statistically significant.
- IMVU uses experiments to help inform product feature development (e.g., what type of user interface works better for managing a large inventory.
- We also use process experiments to inform how we build those features (e.g., does it work better to have a small team of 4 engineers or a larger team of 12?).
- Sadly, Copernicus inspired the wrath of the “folks in charge”, and was persecuted by the Roman Catholic Inquisition.
- Giordano Bruno suffered a worse fate: he was burned at the stake.
- Galileocrystallized the concept that experimentation is at the very heart of the scientific method.
- Their findings were good, but their news was bad. What is it like to share bad news in your company?
- Happily, this whole “science” and “experimentation” thing has caughton, certainly at IMVU.
- Here is why you should really care, though: experimentation may help you understand your customers better, and help your business succeed.
- Obviously, at IMVU we feel that experimentation is good.
- But how does it really work? Let’s start with a quick refresher on The scientific method – from 7th grade science class!This is structured learning at its finest. The scientific method helps you remember the context surrounding your experiment so you can repeat your success—and control the variables that can help you avoid failure.Now imagine that instead of a linear process, the scientific could be applied as a method of continuous learning and improvement.
- At IMVU, we approach the scientific method as an ongoing process.
- The Lean Startup Build-Measure-Learn loop is based on the scientific method.
- You can apply the duck test if you’re not sure… if it looks like a duck, swims like a duck, and sounds like a duck, it probably is a duck!
- If your company does this already, you likely have a good Culture of Experimentation, too.
- We like to keep things simple, because running experiments should be fun and easy.
- Make sure it is simple, even trivial, for your team to run experiments. The easier it is, the more likely you’ll be running lots of experiments and learning more quickly.
- Make it easy to manage your experiments. If it’s easy to manage which experiments are on and off, and which customers should be participating, your product team will run more experiments and you’ll learn more faster.
- Viewing and sharing data should also be simple. You might notice that some of these metrics are red. Not all experiments succeed, but happily at IMVU, our culture supports both success and failure…
- Everyone in the company knows that sharing data and findings is important, whether the results are expected or not, good or “bad”.
- Another tipwesubscribe to at IMVU…
- Now lets talk about experimenting with the way we get things done.
- I’ll frame this within the Lean Startup Build-Measure-Learn loop.
- Think delivering value to customers in your current sprint is most important?Learning is actually the more important.
- Think delivering value to customers in your current sprint is most important?Learning is actually the more important.
- Measure and Learn: 5 Whys Root Cause AnalysisTaiichiOhno, the architect of the Toyota Production System (which includes both management philosophy and production practices) described the 5 whys method as "the basis of Toyota's scientific approach . . . by repeating why five times, the nature of the problem as well as its solution becomes clear." IMVU uses this technique to understand root causes of failures, not just symptoms. Here is how we do it:1. Ask “why” 5 levels deep for each issue2. Identify corrective measures for each cause3. Respond with appropriate level of investment
- Remember a few thingsabout these approaches: What works now may not work in the future.So be cautious about the kinds of conclusions that you draw from our results.We do the same ourselves, and consciously work to avoid being dogmatic about approaches.
- Story points: our teams argued for hours about story points, so proceed with caution.