Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A/B Testing with Yammer's Product Manager

924 views

Published on

This presentation by Anna Marie Clifton, Product Manager at Yammer, covers the important topics of when to use A/B testing, how to implement it and most importantly, how to measure the results.

The content is directed for software engineers who want to transition to product management, MBA's with finance/consulting background who wish to work high-tech companies as product managers and Project Managers, Marketers, and Designers who are seeking opportunities in product management.

Published in: Business
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1lite.top/XxgWl ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

A/B Testing with Yammer's Product Manager

  1. 1. A/B Testing with Yammer’s Product Manager www.productschool.com
  2. 2. FREE INVITE Join 10,000+ Product Managers on productschool.com/slack-community
  3. 3. Product Management Courses
  4. 4. Coding for Managers Courses
  5. 5. Data Analytics for Managers Courses
  6. 6. Include @productschool and #prodmgmt at the end of your tweet Tweet to get a free ticket for our next Event!
  7. 7. Anna Marie Clifton Tonight’s Speaker
  8. 8. Anna Marie Clifton Product Manager at Yammer Co-host of Clearly Product Podcast All the ways that A/B testing sucks
  9. 9. The Usual Story
  10. 10. Anna Marie Clifton Product Manager, Yammer
  11. 11. ● @TweetAnnaMarie ○ Medium & Twitter ● Clearly Product ○ iTunes & Stitcher Anna Marie Clifton Product Manager, Yammer
  12. 12. 1. The rosy side of A/B testing Our roadmap
  13. 13. 1. The rosy side of A/B testing 2. The grey side of A/B testing Our roadmap
  14. 14. Our roadmap 1. The rosy side of A/B testing 2. The grey side of A/B testing 3. Somewhere in the middle
  15. 15. Our roadmap 1. The rosy side of A/B testing 2. The grey side of A/B testing 3. Somewhere in the middle
  16. 16. What is it? 1. The rosy side of A/B testing
  17. 17. New Signup — sequence test 1. The rosy side of A/B testing
  18. 18. ● 80% of users complete the sign up New Signup — sequence test 1. The rosy side of A/B testing
  19. 19. ● 80% of users complete the sign up ● Release a new flow to all your users New Signup — sequence test 1. The rosy side of A/B testing
  20. 20. ● 80% of users complete the sign up ● Release a new flow to all your users ● Measure again New Signup — sequence test 1. The rosy side of A/B testing
  21. 21. ● 80% of users complete the sign up ● Release a new flow to all your users ● Measure again ● Now only 70% of users complete. New Signup — sequence test 1. The rosy side of A/B testing
  22. 22. ● 80% of users complete the sign up ● Release a new flow to all your users ● Measure again ● Now only 70% of users complete. New Signup — sequence test 1. The rosy side of A/B testing
  23. 23. New Signup — A/B test 1. The rosy side of A/B testing
  24. 24. ● 80% of users complete the sign up New Signup — A/B test 1. The rosy side of A/B testing
  25. 25. ● 80% of users complete the sign up ● Release a new flow to 50% of your users — “Treatment” New Signup — A/B test 1. The rosy side of A/B testing
  26. 26. ● 80% of users complete the sign up ● Release a new flow to 50% of your users — “Treatment” ● 70% of users in the treatment complete New Signup — A/B test 1. The rosy side of A/B testing
  27. 27. ● 80% of users complete the sign up ● Release a new flow to 50% of your users — “Treatment” ● 70% of users in the treatment complete ● Meanwhile, only 60% of control users completed New Signup — A/B test 1. The rosy side of A/B testing
  28. 28. ● 80% of users complete the sign up ● Release a new flow to 50% of your users — “Treatment” ● 70% of users in the treatment complete ● Meanwhile, only 60% of control users completed New Signup — A/B test 1. The rosy side of A/B testing
  29. 29. Our roadmap 1. The rosy side of A/B testing 2. The grey side of A/B testing 3. Somewhere in the middle
  30. 30. Engineering 2. The grey side of A/B testing
  31. 31. Engineering issues: 2. The grey side of A/B testing
  32. 32. 1. Takes longer to build Engineering issues: 2. The grey side of A/B testing
  33. 33. 1. Takes longer to build 2. Can’t make changes in flight Engineering issues: 2. The grey side of A/B testing
  34. 34. Engineering issues: 2. The grey side of A/B testing 1. Takes longer to build 2. Can’t make changes in flight 3. Confusion about expected experience
  35. 35. Engineering issues: 2. The grey side of A/B testing 1. Takes longer to build 2. Can’t make changes in flight 3. Confusion about expected experience 4. Code cleanup isn’t always perfect
  36. 36. Product 2. The grey side of A/B testing
  37. 37. Product issues: 2. The grey side of A/B testing
  38. 38. 1. It can take Forever™ Product issues: 2. The grey side of A/B testing
  39. 39. 1. It can take Forever™ 2. Need tons of users Product issues: 2. The grey side of A/B testing
  40. 40. 1. It can take Forever™ 2. Need tons of users 3. Lots of up front work Product issues: 2. The grey side of A/B testing
  41. 41. 1. It can take Forever™ 2. Need tons of users 3. Lots of up front work 4. Lots of after-the-fact work Product issues: 2. The grey side of A/B testing
  42. 42. 1. It can take Forever™ 2. Need tons of users 3. Lots of up front work 4. Lots of after-the-fact work 5. Geared toward small optimizations Product issues: 2. The grey side of A/B testing
  43. 43. 1. It can take Forever™ 2. Need tons of users 3. Lots of up front work 4. Lots of after-the-fact work 5. Geared toward small optimizations 6. Only useful in single-player mode Product issues: 2. The grey side of A/B testing
  44. 44. 1. It can take Forever™ 2. Need tons of users 3. Lots of up front work 4. Lots of after-the-fact work 5. Geared toward small optimizations 6. Only useful in single-player mode 7. Can’t be scientific Product issues: 2. The grey side of A/B testing
  45. 45. 1. It can take Forever™ 2. Need tons of users 3. Lots of up front work 4. Lots of after-the-fact work 5. Geared toward small optimizations 6. Only useful in single-player mode 7. Can’t be scientific Product issues: 2. The grey side of A/B testing
  46. 46. Our roadmap 1. The rosy side of A/B testing 2. The grey side of A/B testing 3. Somewhere in the middle
  47. 47. Totally sucks 3. Somewhere in the middle
  48. 48. Still worth it 3. Somewhere in the middle
  49. 49. Questions? @TweetAnnaMarie
  50. 50. Part-time Product Management Courses in San Francisco www.productschool.com

×