Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The 11 Biggest A/B Testing Mistakes and How To Avoid Them

646 views

Published on

With a variety of easy-to-use tools available, A/B testing is something that every business can and should be doing. However, there’s more to A/B testing than just setting up the test. So many companies waste time and money by making simple and easy to avoid mistakes.

In this presentation, conversion rate expert and guest presenter, Peep Laja, discusses 11 of the biggest testing mistakes he frequently sees companies make. Leave with actionable tips and strategies on how to avoid them. The webinar was moderated by Hanapin’s CRO Manager, Roopa Carpenter.

You’ll get expert-level CRO tips like:

*Why testing random ideas comes at a huge expense
*How to avoid calling a test too early
*Utilizing small wins to make a big impact

Published in: Marketing
  • Be the first to comment

The 11 Biggest A/B Testing Mistakes and How To Avoid Them

  1. 1. #thinkppc HOSTED BY: The 11 13 Biggest A/B Testing Mistakes and How To Avoid Them Featuring Peep Laja
  2. 2. #thinkppc Presenters • Peep Laja – Guest Presenter – Founder of Conversion XL – @peeplaja • Roopa Carpenter – Moderator – CRO Manager at Hanapin Marketing – @roopacarpenter
  3. 3. #thinkppc Join the conversation • Include the hashtag #thinkppc in your Twitter tweets. Or use the webinar question box to send us questions.
  4. 4. #thinkppc Live Poll Question #1 How long have you been doing CRO? #thinkppc A. Less than 1 year B. 1-3 years C. 3-5 years D. 5+ years
  5. 5. #thinkppc Live Poll Question #2 How do you manage your account(s)? #thinkppc a) I’m part of an in-house PPC team. b) I do all the marketing myself. c) I work with an agency. d) I’m a consultant.
  6. 6. #thinkppc Peep Laja, ConversionXL @peeplaja Typical scenario: A company runs 100 tests / year. Lots of wins. But no difference on the bank account.
  7. 7. #thinkppc Precious time wasted on stupid tests Testing mistake #1:
  8. 8. #thinkppc You don’t have a data- driven, learning- oriented hypothesis Testing mistake #2:
  9. 9. #thinkppc Learn more: http://conversionxl.com/how-to-come-up-with-more- winning-tests-using-data/
  10. 10. #thinkppc You think you know what will work Testing mistake #3:
  11. 11. #thinkppc Your sample size is way too small Testing mistake #4:
  12. 12. #thinkppc
  13. 13. #thinkppc Calculate needed sample size in advance:
  14. 14. #thinkppc I won’t even look at results until there’s at least 250-350 conversions PER variation. Way more if you want to segment the data
  15. 15. #thinkppc Your tests don’t run long enough Testing mistake #5:
  16. 16. #thinkppc
  17. 17. #thinkppc You don’t test full weeks at a time Testing mistake #6:
  18. 18. #thinkppc
  19. 19. #thinkppc You run tests when you have no traffic Testing mistake #7:
  20. 20. #thinkppc •Learn more: http://conversionxl.com/how-to-do-conversion- optimization-with-very-little-traffic/
  21. 21. #thinkppc Test data is not sent to third-party analytics Testing mistake #8:
  22. 22. #thinkppc
  23. 23. #thinkppc (function(){ var experimentID = 2809164105; window.optimizely = window.optimizely || []; if (typeof window.optimizely.data.experiments[experimentID] !== 'undefined') { var experimentName = window.optimizely.data.experiments[experimentID].name; var variationName = window.optimizely.data.state.variationNamesMap[experimentID]; window.ga = window.ga || function() {(window.ga.q = window.ga.q || []).push(arguments);}; window.ga.l =+ new Date(); window.ga('send', 'event', 'Optimizely', experimentID + ' ' + experimentName, variationName, {'nonInteraction': 1}); } })();
  24. 24. #thinkppc Peep Laja, ConversionXL @peeplaja
  25. 25. #thinkppc You give up after your first test for a hypothesis fails. Testing mistake #9:
  26. 26. #thinkppc
  27. 27. #thinkppc Full case study: http://conversionxl.com/case-study-how-we-improved-landing-page-conversion/ 79.3% uplift in conversions
  28. 28. #thinkppc
  29. 29. #thinkppc
  30. 30. #thinkppc
  31. 31. #thinkppc
  32. 32. #thinkppc
  33. 33. #thinkppc
  34. 34. #thinkppc You’re not aware of validity threats Testing mistake #10:
  35. 35. #thinkppc History effect Instrumentation effect Selection effect
  36. 36. #thinkppc You’re ignoring small gains Testing mistake #11:
  37. 37. #thinkppc 5% monthly increase in your conversion rate will result in 80% uplift for the year. That’s how math works.
  38. 38. #thinkppc You copy competitors / leaders / other people’s tests Testing mistake #12:
  39. 39. #thinkppc Which Test Won?
  40. 40. #thinkppc WhichTestWon? WhichTestWasNot RunProperly?
  41. 41. #thinkppc Somebody else’s test results are useless
  42. 42. #thinkppc “Site shows stats from various A/B tests - Finally I've got evidence to show clients on a load of design decisions!”
  43. 43. #thinkppc
  44. 44. #thinkppc The process, the method, the analysis
  45. 45. #thinkppc You’re not running tests at all times Testing mistake #13:
  46. 46. #thinkppc PPC Retainers Need some assistance for your PPC? We offer on-going and one-time retainers. Learn More: http://www.hanapinmarketing.com/ppc-retainers/
  47. 47. #thinkppc Live Q&A Time!
  48. 48. #thinkppc Have more questions? Thank you for attending our webinar! #thinkppc • Contact us Directly: » Hanapin Feedback: marketing@hanapinmarketing.com

×