Call Girls Aslali 7397865700 Ridhima Hire Me Full Night
Embracing Failure: How Netflix Builds better global products through A/B testing, research & empathy.
1. 3Slide /
Andrew Law
Design Director @Netflix
Embracing Failure: How
Netflix builds better
global products through
A/B testing, research &
empathy.
2. 4Slide /
Andrew Law
Design Director @Netflix
Embracing Failure: How
Netflix builds better
global products through
A/B testing.
3. 5Slide /
Find if there is
a there, there.
It’s your job.
Product Designer
Andrew Law
As a product designer it’s your job
to identify opportunities that
grow the business.
5. 7
PIF
Slide /
Start with
what you
know.
32
Product innovation framework
1
What data do you have access to? Why
hypothesis can you draw from the
information? What information do you still
need? Ask yourself is there a there, there?
Foundational Research
How important is personalization & the
recruit? Native prototypes are not necessary.
Use whatever tool allows you to get a read on
the idea. Paper is always an option.
Prototype & Learn
How do you know if the idea is something that
would benefit the company? Identify
argumentative support of your hypothesis. It’s
often easy to find data that justifies your
hypothesis, I urge you to spend time focusing
on data that disproves it as well.
Ideate
Andrew Law
6. 8Slide /
Product innovation framework
6
Start all
over
again!
4 5
80% of the work on this feature is complete,
it’s now time to smooth over any rough edges
and ship. What needs to launch versus fast
follow items? Concessions almost always
have to be made during the process, pick
your battles.
Productize (ship it!)
It’s time to turn those learnings into
actionable insights. What worked well? What
didn’t? How much of what you heard was
noise?
Synthesize Learnings
There are a few different types of tests that
we can utilize to get a signal on a new feature
or idea. This is not about throwing stuff
against a wall to see if it sticks. A/B tests
should be calculated and focused to insure
success; learning.
Identify Variables & Test
PIF Continued
Andrew Law
7. 9Slide /
Foster a
culture of
iterative
testing.
A Culture Code
The Netflix testing culture is
unique and supports the
experimentation/iteration of
product features and ideas.
Andrew Law
8. 10Slide /
A/B testing
helps us rule
out bad ideas
and focus on
larger more
risky product
bets.
Testing Benefits
Scientific
approach to
making
thoughtful
and
measured
product
decisions.
At Netflix we
test large
ideas, not
something
small like the
color of a
button.
Understand
what success
looks like
before you
run the test.
Andrew Law
9. 11Slide /
How do you
build a global
product for 190
countries in 27
languages with
over 130M
members?
3
Global Product Design
Andrew Law
10. 12Slide /
How do you
build a global
product for 190
countries in 27
languages with
over 130M
members?
3 Rest of world2 LatAM1 Western Countries
Global Product Design
Andrew Law
11. 13Slide /
Empathy
Empathy is a
muscle, and
just like any
other, it can be
built stronger
and shaped
over time.
Andrew Law
12. 15Slide /
Empathy
Understanding
someones
frame of
reference is
powerful and
can lead to
meaningful
innovation.
Andrew Law
When you
empathize you
tap into
emotions and
bond over
shared
feelings and
experiences.
Identify your
biases,
awareness
leads to a
more
measured
and
thoughtful
product
designer.
You don’t
“do”
empathy you
“have”
empathy.
You’re going
to mess it up
- and that’s
okay.
13. 16Slide /
Foundational ethnographic research helps to paint a broad understanding of
potential opportunities.
Man on the Street
A low cost way to get quick/dirty and cheap usability feedback.
Video Diaries
Interviews where users conduct in situ observations on prototypes or their own
behavior with other competitive ideas or services.
Lab Sessions
Exploratory or usability lab sessions help to identify
Noise is
common in
research,
knowing will
help you be
able to interpret
good feedback
from fluff.
Research Types
Andrew Law
Exploratory & Ethnographic
14. 17Slide /
Andrew Law
andy@ideation.pro
Design Director @Netflix
I do what
makes me
happy and that
is the single
reason for any
of my
success.
Creative mobile product designer leader
with a passion to work with the most
dynamic, innovative and driven people.
Contact me for design help, speaking
engagements or mentorship.
BIO
Questions?