Test & Learn: Building Habits That Matter - How WSJ Models Subscriber Behavior to Improve Retention & Engagement
•
2 likes•621 views
In creating experiments to increase product sales or customer acquisition, the end goal is clear: the more orders placed, the more dollars back to the business. But how should we think about prioritizing tests to increase product engagement? In a world where we can encourage users to take a wide variety of actions with unclear business benefits, it seems almost impossible to determine what is best. How can we promote behaviors that build habits and promote retention? The answer is simple: we analyze what habits matter most. In this session Olivia will share: How she created the Wall Street Journal’s first habit model How the team identified which actions influence member tenure How WSJ uses habit data to inform their testing and engagement strategies
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Test & Learn: Building Habits That Matter - How WSJ Models Subscriber Behavior to Improve Retention & Engagement
Similar to Test & Learn: Building Habits That Matter - How WSJ Models Subscriber Behavior to Improve Retention & Engagement (20)
More from Optimizely
More from Optimizely (20)
Recently uploaded
Recently uploaded (20)
Test & Learn: Building Habits That Matter - How WSJ Models Subscriber Behavior to Improve Retention & Engagement
- 1. Olivia Simon WSJ Building Habits That Matter: How WSJ Models Subscriber Behavior to Improve Retention and Engagement Tune in on Twitter: #testandlearn19
- 2. Building Habits That Matter at WSJ
- 3. DOW JONES OPTIMIZATION What Does Experimentation at WSJ Look Like?
- 4. DOW JONES OPTIMIZATION Where Do We Test? Acquisition: Homepage placements Article roadblocks Shop Checkout Engagement: Onboarding Newsletter Center Platforms Center Customer Center
- 5. DOW JONES OPTIMIZATION Where Do We Test? Product: Home page Article pages Videos / Video Center Retention: Online Cancel Winback shop Winback emails Credit Card Decline
- 6. WSJ & BARRON’S iOS TESTING - 2016 DOW JONES OPTIMIZATION How Do We Quantify the Value of Product Testing?
- 7. DOW JONES OPTIMIZATION Acquisition Tests Make Us $$$$ 100 People Shop Test 10% Uplift! 10 More orders! 10 orders x $2 per order = $20!
- 8. DOW JONES OPTIMIZATION But Do Product Tests Make Us $$$? 100 People Article Test: Email Shares 10% Uplift! 10 More email shares 10 more email shares? So what?
- 9. DOW JONES OPTIMIZATION Determining A Fair Metric For Product Testing Did they ride it again? Did they order it again? Test question: What does WSJ have that keeps members around longer?
- 10. DOW JONES OPTIMIZATION Turning Product Actions Into Value The Habit Project: A model to determine what onsite habits make members stay longer.
- 11. DOW JONES OPTIMIZATION Habit in a Nutshell Over 50 product actions ● Content types (section, column) ● Authors ● Video/Audio ● Sharing ● Product benefits (app, newsletter, WS+) Resulting 1-year retention rate + Likelihood Yes! :) No! >:( In a member’s first 100 days, did they do the action?
- 12. DOW JONES OPTIMIZATION First 100 Days = Habit Magic Opportunity Window First 100 Days
- 13. DOW JONES OPTIMIZATION Correlation vs Causation? What if people who share via email are just more engaged members than those who watch video content?” 1. Model focuses on non-habituated readers (ie, first 100 days). 2. Trends held true regardless of tenure and subscription package selected.
- 14. DOW JONES OPTIMIZATION Our Framework for Mapping Habits Habit’s Impact on 1 Year Retention Rate CurrentAdoptionRateoftheHabit LOW HIGH HIGH LOW Unlikely and Not Sticky! Highly Likely and Very Sticky! Very Sticky, but Unlikely Very Likely, but not Sticky
- 15. DOW JONES OPTIMIZATION LOW HIGH 2 122 11 10 4 8HIGH LOW 1 Our Framework for Mapping Habits Habit’s Impact on 1 Year Retention Rate CurrentAdoptionRateoftheHabit
- 16. DOW JONES OPTIMIZATION A Peek At The Results WSJ App Newsletters Our Content!
- 17. WSJ & BARRON’S iOS TESTING - 2016 DOW JONES OPTIMIZATION Actioning Sticky Habits via Testing
- 18. DOW JONES OPTIMIZATION Testing Our Onboarding Flow Mobile Apps Card 1 Newsletter Card 2 WSJ+ Card 4 Social Card 5 PI Card Card 3 Control
- 19. DOW JONES OPTIMIZATION Testing Our Onboarding Flow Variation App Card 1 Newsletter Card 2 Puzzle Card 4 Watchlist Card 5 Video Card 6 Podcast Card 7 WSJ+ Card 8 Social Card 9 PI Card 3 Books and Sports Newsletters Added +51% increase in average number of engagement actions being taken
- 20. DOW JONES OPTIMIZATION Get Them to The App! Variation +4.3% app download rate Control
- 21. DOW JONES OPTIMIZATION Get Them to The App! Part 2 Variation +7194% app download rate Control
- 22. DOW JONES OPTIMIZATION Get Them to The App! Part 3 Variation +13% app download rate Control
- 23. WSJ & BARRON’S iOS TESTING - 2016 DOW JONES OPTIMIZATION Evaluating Success
- 24. DOW JONES OPTIMIZATION How Much We’ve Changed Engagement With Sticky Actions Habit Change Play Podcast on WSJ.com +371% Read Content A +118% Play Puzzle +54% Read Author A +51% Read Author B +51% Subscribe to New Newsletter +44% Post Comment +14% Read Content B +10% Visit On Weekend +7% Login App +6% Add to Watchlist -2% Read Content C -7% Read Content D -16%
- 25. DOW JONES OPTIMIZATION Building Habits That Matter at WSJ
- 26. Next Session How to Leverage Design to Learn & Deliver Results Quickly Angel Steger Director of Product Design, Dropbox Tune in on Twitter: #testandlearn19