Recommended
More Related Content
Similar to Embracing Failure: How Netflix Builds better global products through A/B testing, research & empathy.
Similar to Embracing Failure: How Netflix Builds better global products through A/B testing, research & empathy. (20)
Recently uploaded
Recently uploaded (20)
Embracing Failure: How Netflix Builds better global products through A/B testing, research & empathy.
- 1. 3Slide / Andrew Law Design Director @Netflix Embracing Failure: How Netflix builds better global products through A/B testing, research & empathy.
- 2. 4Slide / Andrew Law Design Director @Netflix Embracing Failure: How Netflix builds better global products through A/B testing.
- 3. 5Slide / Find if there is a there, there. It’s your job. Product Designer Andrew Law As a product designer it’s your job to identify opportunities that grow the business.
- 4. 6 Validating Ideas Slide / Innovation Opportunity Andrew Law Business Objectives User Needs Need Size Opp Cost Do it!
- 5. 7 PIF Slide / Start with what you know. 32 Product innovation framework 1 What data do you have access to? Why hypothesis can you draw from the information? What information do you still need? Ask yourself is there a there, there? Foundational Research How important is personalization & the recruit? Native prototypes are not necessary. Use whatever tool allows you to get a read on the idea. Paper is always an option. Prototype & Learn How do you know if the idea is something that would benefit the company? Identify argumentative support of your hypothesis. It’s often easy to find data that justifies your hypothesis, I urge you to spend time focusing on data that disproves it as well. Ideate Andrew Law
- 6. 8Slide / Product innovation framework 6 Start all over again! 4 5 80% of the work on this feature is complete, it’s now time to smooth over any rough edges and ship. What needs to launch versus fast follow items? Concessions almost always have to be made during the process, pick your battles. Productize (ship it!) It’s time to turn those learnings into actionable insights. What worked well? What didn’t? How much of what you heard was noise? Synthesize Learnings There are a few different types of tests that we can utilize to get a signal on a new feature or idea. This is not about throwing stuff against a wall to see if it sticks. A/B tests should be calculated and focused to insure success; learning. Identify Variables & Test PIF Continued Andrew Law
- 7. 9Slide / Foster a culture of iterative testing. A Culture Code The Netflix testing culture is unique and supports the experimentation/iteration of product features and ideas. Andrew Law
- 8. 10Slide / A/B testing helps us rule out bad ideas and focus on larger more risky product bets. Testing Benefits Scientific approach to making thoughtful and measured product decisions. At Netflix we test large ideas, not something small like the color of a button. Understand what success looks like before you run the test. Andrew Law
- 9. 11Slide / How do you build a global product for 190 countries in 27 languages with over 130M members? 3 Global Product Design Andrew Law
- 10. 12Slide / How do you build a global product for 190 countries in 27 languages with over 130M members? 3 Rest of world2 LatAM1 Western Countries Global Product Design Andrew Law
- 11. 13Slide / Empathy Empathy is a muscle, and just like any other, it can be built stronger and shaped over time. Andrew Law
- 12. 15Slide / Empathy Understanding someones frame of reference is powerful and can lead to meaningful innovation. Andrew Law When you empathize you tap into emotions and bond over shared feelings and experiences. Identify your biases, awareness leads to a more measured and thoughtful product designer. You don’t “do” empathy you “have” empathy. You’re going to mess it up - and that’s okay.
- 13. 16Slide / Foundational ethnographic research helps to paint a broad understanding of potential opportunities. Man on the Street A low cost way to get quick/dirty and cheap usability feedback. Video Diaries Interviews where users conduct in situ observations on prototypes or their own behavior with other competitive ideas or services. Lab Sessions Exploratory or usability lab sessions help to identify Noise is common in research, knowing will help you be able to interpret good feedback from fluff. Research Types Andrew Law Exploratory & Ethnographic
- 14. 17Slide / Andrew Law andy@ideation.pro Design Director @Netflix I do what makes me happy and that is the single reason for any of my success. Creative mobile product designer leader with a passion to work with the most dynamic, innovative and driven people. Contact me for design help, speaking engagements or mentorship. BIO Questions?
- 15. Thank you!