Your SlideShare is downloading. ×

How to Run AB Tests that Get REAL Results

7,006

Published on

• 3. We’ll cover...1 The wrong way to A/B test2 5 places to get ideas for testing3 Get the most from your A/B testing tools …

• 3. We’ll cover...1 The wrong way to A/B test2 5 places to get ideas for testing3 Get the most from your A/B testing tools
• 4. HOW MOST PEOPLEA/B Test
• 5. A/B testing is totally rad! We should do it!
• 6. Everyone loves button colors VS
• 7. Next up? Layouts!
• 8. What about photos? VS
• 9. And conversions went up by 4%! Party time!
• 10. There’s a better way... Testing random elements is very inefficient. You will not find the big wins this way.
• 11. You’re also not learning Red vs green buttons doesn’t help you understand your customer.
• 12. GETQualitative
• 13. Qualitative data definition Data that isn’t a number.
• 14. Start with qualitative insights Use other sources of data to get a basic understanding of what your customers want.
• 15. Understand first, then do the A/B test You’ll focus on the right tests and increase your conversions a LOT faster.
• 16. The A/B Testing Process 1 Get qualitative insights 2 Predict how to improve 3 Confirm prediction with an A/B test
• 17. Why is this process better? Helps you find the 40% increase in conversions instead of only the 4%.
• 18. WHERE IS THE QUALITATIVE Data?
• 19. Location #1 Surveys
• 20. Use one-question surveys on your site
• 21. Also try long surveys
• 22. Survey best practices1 Keep it short2 Only ask for answers that you’ll use3 Start with open-ended questions
• 23. Location #2 Feedback
• 24. Do you have a feedback email?
• 25. Talk to your support team They know exactly what problems keep coming up.
• 26. Also talk to your sales team They talk to customers every day.
• 27. Location #3 Reach Out
• 28. Nothing beats talking to customers personally Get them on the phone and deeply understand how they use your product.
• 29. If local, take them out to lunch or coffee.
• 30. This is not a pitch Focus on asking good questions and listening.
• 31. Location #4 User Activity
• 32. Look at activity of individual people You’ll need customer analytics for this.
• 33. KISSmetrics People report
• 34. When you have an insight... Confirm by looking at other people.
• 35. Location #5Usability Tests
• 36. What’s a usability test? Have someone use your site or product while you watch.
• 37. Grab someone off the street
• 38. Amazon’s Mechanical Turk - mturk.com
• 39. UserTesting.com
• 40. Places to Find Qualitative Insights
• 41. The A/B Testing Process
• 42. When should you test the small stuff? After you’ve found the big wins and you have a full-time growth team.
• 43. A/B TESTINGTools
• 44. A/B testing tools only measure the next step
• 45. Tie A/B tests back to the customer life-cycle
• 46. How do we actually do this?
• 47. First, make sure you’re using customer analytics
• 48. Then use Visual Website Optimizer or Optimizely OR
• 49. Now all your A/B testing data is in KISSmetrics

Published in: Technology, Education
0 Comments
29 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
7,006
On Slideshare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
199
Comments
0
Likes
29
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • \n
  • \n
  • Let’s take an Apple product for example.\nIt’s obvious to use Apple as a case study for user experience. Their whole business model is built on UX and delighting their customers.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Let’s take an Apple product for example.\nIt’s obvious to use Apple as a case study for user experience. Their whole business model is built on UX and delighting their customers.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Let’s take an Apple product for example.\nIt’s obvious to use Apple as a case study for user experience. Their whole business model is built on UX and delighting their customers.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Let’s take an Apple product for example.\nIt’s obvious to use Apple as a case study for user experience. Their whole business model is built on UX and delighting their customers.\n
  • Let’s take an Apple product for example.\nIt’s obvious to use Apple as a case study for user experience. Their whole business model is built on UX and delighting their customers.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript

    • 1. How to Run A/B Tests that Get REAL Results Lars Lofgren Marketing Analyst - July 2012 info@kissmetrics.com - Confidential - Do not distribute
    • 2. @larslofgren #KISSwebinar
    • 3. We’ll cover...1 The wrong way to A/B test2 5 places to get ideas for testing3 Get the most from your A/B testing tools
    • 4. HOW MOST PEOPLEA/B Test
    • 5. A/B testing is totally rad! We should do it!
    • 6. Everyone loves button colors VS
    • 7. Next up? Layouts!
    • 8. What about photos? VS
    • 9. And conversions went up by 4%! Party time!
    • 10. There’s a better way... Testing random elements is very inefficient. You will not find the big wins this way.
    • 11. You’re also not learning Red vs green buttons doesn’t help you understand your customer.
    • 12. GETQualitative
    • 13. Qualitative data definition Data that isn’t a number.
    • 14. Start with qualitative insights Use other sources of data to get a basic understanding of what your customers want.
    • 15. Understand first, then do the A/B test You’ll focus on the right tests and increase your conversions a LOT faster.
    • 16. The A/B Testing Process 1 Get qualitative insights 2 Predict how to improve 3 Confirm prediction with an A/B test
    • 17. Why is this process better? Helps you find the 40% increase in conversions instead of only the 4%.
    • 18. WHERE IS THE QUALITATIVE Data?
    • 19. Location #1 Surveys
    • 20. Use one-question surveys on your site
    • 21. Also try long surveys
    • 22. Survey best practices1 Keep it short2 Only ask for answers that you’ll use3 Start with open-ended questions
    • 23. Location #2 Feedback
    • 24. Do you have a feedback email?
    • 25. Talk to your support team They know exactly what problems keep coming up.
    • 26. Also talk to your sales team They talk to customers every day.
    • 27. Location #3 Reach Out
    • 28. Nothing beats talking to customers personally Get them on the phone and deeply understand how they use your product.
    • 29. If local, take them out to lunch or coffee.
    • 30. This is not a pitch Focus on asking good questions and listening.
    • 31. Location #4 User Activity
    • 32. Look at activity of individual people You’ll need customer analytics for this.
    • 33. KISSmetrics People report
    • 34. When you have an insight... Confirm by looking at other people.
    • 35. Location #5Usability Tests
    • 36. What’s a usability test? Have someone use your site or product while you watch.
    • 37. Grab someone off the street
    • 38. Amazon’s Mechanical Turk - mturk.com
    • 39. UserTesting.com
    • 40. Places to Find Qualitative Insights 1 Surveys 2 Feedback 3 Reach Out 4 User Activity 5 Usability Tests
    • 41. The A/B Testing Process 1 Get qualitative insights 2 Predict how to improve 3 Confirm prediction with an A/B test
    • 42. When should you test the small stuff? After you’ve found the big wins and you have a full-time growth team.
    • 43. A/B TESTINGTools
    • 44. A/B testing tools only measure the next step Great for testing copy, photos, and layouts. Terrible for bigger tests like pricing.
    • 45. Tie A/B tests back to the customer life-cycle Critical for measuring how your changes impact customers over the long term.
    • 46. How do we actually do this?
    • 47. First, make sure you’re using customer analytics
    • 48. Then use Visual Website Optimizer or Optimizely OR
    • 49. Now all your A/B testing data is in KISSmetrics
    • 50. For a demo of KISSmetrics or to learnhow KISSmetircs can help grow your business, contact Ben Sardella at: bsardella@kissmetrics.com

    ×