Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Devtribe a/ b testing on multiple platforms with recurring and paying users

164 views

Published on

Talk at DevTribe gathering 3. 10 minutes about how feature toggling enable A/B testing.

Published in: Technology
  • DOWNLOAD THAT BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book that can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer that is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story That Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money That the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths that Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Devtribe a/ b testing on multiple platforms with recurring and paying users

  1. 1. A/B testing(on multiple platforms with recurring and paying users) at Spotify DevTribe Gathering 3 2016-12-06. Oscar Carlsson, lad@spotify.com A/B testing at Spotify(on multiple platforms with recurring and paying users)
  2. 2. Oscar Carlsson Data Scientist
  3. 3. Feature toggling + A/B testing
  4. 4. picture from https://www.elegantthemes.com/blog/tips-tricks/easy-ab-testing-with-divi-leads The classic example of A/B testing Winner if CTR/Conversion is up.
  5. 5. Oftenly no requirement to understanding the why
  6. 6. New users Existing users Multiple platforms Premium & Free
  7. 7. New users Existing users Platforms
  8. 8. New users Newly registered users with no habits or routines in the app. They have never seen the UI and any impression or interaction is guaranteed to be the first. Relatively low traffic. Activation: How many registered users use the app on first day? Engagement: How much are they using the product? Retention: How many registered users use the app second week/month? Conversion: Do they turn premium?
  9. 9. New users Newly registered users with no habits or routines in the app. They have never seen the UI and any impression or interaction is guaranteed to be the first. Relatively low traffic. Activation: How many registered users use the app on first day? Retention: How many registered users use the app second week/month? Engagement: How much are they using the product? Conversion: Do they turn premium?
  10. 10. Existing users Users who are using your product already. They probably have habits and routines and they might have seen the surface before. Relatively high traffic. Engagement: How much are they using the product? Conversion: Do they turn premium?
  11. 11. Existing users Users who are using your product already. They probably have habits and routines and they might have seen the surface before. Relatively high traffic. Engagement: How much are they using the product? Conversion: Do they turn premium?
  12. 12. Platforms (Often) test separately Not fully aligned - We’re testing in different phase on all platforms Users can use multiple platforms and usage can shift Android iPhone Desktop Web player and more.
  13. 13. Every team should be testing! .. could there be challenges with that?
  14. 14. A/B testing A/B testing A/B testing A/B testing
  15. 15. I want long term effect! I want your change and I want long term effect! I also want those changes and I want long term effect! Me too and I want long term effect!
  16. 16. I want long term effect! I want your change and I want long term effect! I also want those changes and I want long term effect! Me too and I want long term effect! Dependencies! & Complicated code?
  17. 17. New bold font Center header Remove card background Center follower count
  18. 18. Fully scientific? Quick product development?
  19. 19. Fully scientific? Quick product development?
  20. 20. Fully scientific? Quick product development? A mix of both?
  21. 21. How many tests can a user live through?
  22. 22. How do you track, and do you track what happens to the users when the test is over?
  23. 23. picture from http://codevoyagers.com/2016/03/16/design-like-youre-right-test-like-youre-wrong/
  24. 24. Thanks for listening Oscar Carlsson lad@spotify.com

×