Find out how to move beyond customer focus to "customer obsession" via consumer science to discover what delights customers in hard-to-copy, margin-enhancing ways. Learn how to get insight from four sources of consumer insight, then evaluate these ideas through various research techniques. Gib will illustrate Netflix’s customer obsession through a series of experiments the company executed to help connect its customer with movies they’ll love through its personalization technology. Last, Gib will put the techniques into practice with a highly interactive, modern-day Netflix case. Gibson Biddle was VP of Product at Netflix starting in 2005, then in 2010 became the Chief Product Officer of Chegg, a textbook rental company that went public in 2014. Today, he's an advisor to several tech companies and is a guest lecturer at Stanford.

1. Traction: August 7, 2019
N E T F L I X ’ S
CUSTOMER
OBSESSION
Gibson Biddle // @gibsonbiddle

2. @gibsonbiddle
Are you a
Netflix member?

3. Net Promoter Score
?
Detractors (0-6)Passives (7-8)Promoters (9-10) Net Promoter Score

4. Net Promoter Score
10%10%80% 70%
70
Detractors (0-6)Passives (7-8)Promoters (9-10) Net Promoter Score

5. Jeff Bezos
CEO, Amazon
@gibsonbiddle
Customers are always beautifully, wonderfully
dissatisfied, even when they report being happy
and business is great. Even when they don’t yet
know it, customers want something better, and
your desire to delight customers will drive
you to invent on their behalf.”

6. @gibsonbiddle
Electronic Arts

7. @gibsonbiddle
Elmo’s Preschool

8. Creative Wonders
@gibsonbiddle

9. @gibsonbiddle
The Learning Company

10. Mattel

11. @gibsonbiddle
Netflix

12. Customer focus
Satisfy
Listen
Compete.
@gibsonbiddle

13. Customer obsession
Delight
Experiment
Invent.
@gibsonbiddle

14. 1999

15. Today

16. Speaker, Adviser, Teacher (2015)
CPO, Chegg (2010)
VP Product, Netflix (2005)
SVP, Creative Wonders,The Learning
Company, Mattel (1995)
Director, Product, Electronic Arts (1991)
Gibson Biddle
@gibsonbiddle

17. Customer
Obsession
The DHM Model
Consumer Science
Free Trial Reminder
Agenda
@gibsonbiddle

18. @gibsonbiddle
Product leader’s job
Delight customers in
Hard to copy
Margin-enhancing ways

19. Balancing act
Delight v. margin
New Release DVD test.
@gibsonbiddle

20. NEW on DVD
Consumer insight
Surveys, focus groups &
CS data
The A/B test
Perfect New Release
Delivery
The result
Cancels: from 4.50% to 4.45%
“Save” 5,000 customers
“Worth” $1M= 5,000 x $100 x 2
But, additional inventory = $5M.
@gibsonbiddle
Perfect NR test

21. NEW on DVD
Launch NR testor
No launch
@gibsonbiddle

22. NEW on DVD
Launch NR testor
No launch
@gibsonbiddle

23. Hard to copy
@gibsonbiddle

24. Hard to copy
@gibsonbiddle

25. @gibsonbiddle
“Movie enjoyment made easy”
Personalization
Device ecosystem
Original content
/ ExamplesHard to copy
Brand
Unique technology
Network effect
Economies of scale

26. @gibsonbiddle
Personalized Instant (DVD, streaming)
Easy Exclusives
Open APIs Entertainment
Friends (social) Movie-finding tools

27. Entertainment
Instant (DVD, streaming)
Easy Exclusives
Open APIs
Friends (social) Movie-finding tools
@gibsonbiddle
Personalized

28. Entertainment
Instant (DVD, streaming)
Easy Exclusives
Open APIs
Friends (social) Movie-finding tools
Originals
TV devices
@gibsonbiddle
Personalized

29. Product strategy
Personalization
Instant
Easy
Device ecosystem
Margin-enhancing
/ Tactics
Ratings Wizard, new algorithms
From 20 to 100 hubs; streaming
Day One experience
API launch; Xbox, Playstation
Price & plan testing
@gibsonbiddle

30. @gibsonbiddle
Product strategy
Personalization
Instant
Easy
Device ecosystem
Margin-enhancing
/ Metrics
% members: 50 + ratings at M2
% members watch > 15 min./mo.
% with > 3 titles in queue, day 1
% of demand weighted TVs
Lifetime value, gross margin

31. Consumer
Science
@gibsonbiddle

32. Consumer
Science
Consumer insight through:
Qualitative
Survey
Existing data
A/B tests
Hypotheses
Procedures
(Experiments)
Data
(Results)
SCIENTIFIC
METHOD
Findings
(Conclusions)
@gibsonbiddle

33. 2005
@gibsonbiddle

34. @gibsonbiddle
P2 > 50 ratings
% members add > 6 dvds/mo.
RMSE
Average rating
/ MetricProduct strategy
Explicit data
Implicit data
Matching algorithms
High-quality movies

35. @gibsonbiddle
Ratings Wizard
Queue-add data
Collaborative filtering
Acquire better movies
/ TacticProduct strategy
Explicit data
Implicit data
Matching algorithms
High-quality movies

36. Pop Quiz #1
@gibsonbiddle
What % of members rate > 50 movies in first 6 weeks?

37. % of members who rate > 50 movies
@gibsonbiddle
0
0.075
0.15
0.225
0.3
2004 2005 2006 2007 2008 2009 2010 2011
30%
15%
0%

38. Ratings Wizard
@gibsonbiddle
The more you rate, the better we’ll
do at finding movies you’ll love.

39. @gibsonbiddle
Pop Quiz #2
Will demographics help predict movie taste?

40. Movie predictions improve over time
@gibsonbiddle
0.6
0.7
0.8
0.9
1
2004 2005 2006 2007 2008 2009 2010 2011

41. Collaborative filtering
@gibsonbiddle

42. Improve RMSE by 10%, win $1M!
@gibsonbiddle

43. @gibsonbiddle
Pop Quiz #3
Will the new algorithm improve retention?

44. Category interest algorithm

45. @gibsonbiddle
0
20
40
60
80
2004 2005 2006 2007 2008 2009 2010 2011
% Members who add more than 6 DVDs/month to their queue
DVDs by mail

46. @gibsonbiddle
% Members who add more than 6 DVDs/month to their queue
DVDs by mail
0
20
40
60
80
2004 2005 2006 2007 2008 2009 2010 2011
% Members who
watch > 15
minutes/month
Streaming

47. Pop Quiz #4
Is it helpful to know you’re French?

48. Thumbs v. Stars?

49. @gibsonbiddle
0
0.95
1.9
2.85
3.8
2004 2005 2006 2007 2008 2009 2010 2011
Average rating improves over time

50. @gibsonbiddle
Movie rating does not equal movie enjoyment

52. Casino = 98% match
@gibsonbiddle

53. @gibsonbiddle

54. @gibsonbiddle

55. Free Trial
Reminder
@gibsonbiddle

56. @gibsonbiddle

57. @gibsonbiddle

58. (2018: All data are guesstimates)
@gibsonbiddle
2%
90%
$10M
85%
Consumer insight
“Cancel anytime”
Customer service:
Trials/visits conversion:
Month one retention:
A/B test results
Month one retention:
DHM?
High or low stakes?
Who makes the decision?
Free trial reminder

59. Send to allor
Don’t send
@gibsonbiddle

60. Send to allor
Don’t send
@gibsonbiddle

61. Low stakes (reversible)
DHM: Double down on delight
Cost of ethics & brand
Debate, decide, do.
Free trial
learnings
@gibsonbiddle

62. Threemore things required
@gibsonbiddle

63. Courage
@gibsonbiddle

64. No Late Fees
@gibsonbiddle
?
Patience

65. Humility
@gibsonbiddle

66. @gibsonbiddle
Existing data
Data focus
Customer & Bus.
Qualitative
1-1, usability,
focus groups
Survey data
Demo, cancel,
quality, NPS
A/B testing
Measure change
Delight v. margin
Strategy
Hypotheses
DHM model
Are You
Obsessed?

67. Customer
Obsession
Putting the customer at
the center of everything
you do, to delight
customers in hard-to-copy,
margin-enhancing ways,
to invent the future.”

68. @gibsonbiddle

70. @gibsonbiddle

72. www.gibsonbiddle.com
Complete Net Promoter Score survey
Download PDF of presentation
“Customer Obsession” article on Medium
@gibsonbiddle

73. Questions?
www.gibsonbiddle.com
Download PDF of presentation
“Customer Obsession” article on Medium

74. www.gibsonbiddle.com
Complete Net Promoter Score survey
Download PDF of presentation
“Customer Obsession” article on Medium