How to Run AB Tests that Get REAL Results
 

How to Run AB Tests that Get REAL Results

on

  • 6,247 views

• 3. We’ll cover...1 The wrong way to A/B test2 5 places to get ideas for testing3 Get the most from your A/B testing tools ...

• 3. We’ll cover...1 The wrong way to A/B test2 5 places to get ideas for testing3 Get the most from your A/B testing tools
• 4. HOW MOST PEOPLEA/B Test
• 5. A/B testing is totally rad! We should do it!
• 6. Everyone loves button colors VS
• 7. Next up? Layouts!
• 8. What about photos? VS
• 9. And conversions went up by 4%! Party time!
• 10. There’s a better way... Testing random elements is very inefficient. You will not find the big wins this way.
• 11. You’re also not learning Red vs green buttons doesn’t help you understand your customer.
• 12. GETQualitative
• 13. Qualitative data definition Data that isn’t a number.
• 14. Start with qualitative insights Use other sources of data to get a basic understanding of what your customers want.
• 15. Understand first, then do the A/B test You’ll focus on the right tests and increase your conversions a LOT faster.
• 16. The A/B Testing Process 1 Get qualitative insights 2 Predict how to improve 3 Confirm prediction with an A/B test
• 17. Why is this process better? Helps you find the 40% increase in conversions instead of only the 4%.
• 18. WHERE IS THE QUALITATIVE Data?
• 19. Location #1 Surveys
• 20. Use one-question surveys on your site
• 21. Also try long surveys
• 22. Survey best practices1 Keep it short2 Only ask for answers that you’ll use3 Start with open-ended questions
• 23. Location #2 Feedback
• 24. Do you have a feedback email?
• 25. Talk to your support team They know exactly what problems keep coming up.
• 26. Also talk to your sales team They talk to customers every day.
• 27. Location #3 Reach Out
• 28. Nothing beats talking to customers personally Get them on the phone and deeply understand how they use your product.
• 29. If local, take them out to lunch or coffee.
• 30. This is not a pitch Focus on asking good questions and listening.
• 31. Location #4 User Activity
• 32. Look at activity of individual people You’ll need customer analytics for this.
• 33. KISSmetrics People report
• 34. When you have an insight... Confirm by looking at other people.
• 35. Location #5Usability Tests
• 36. What’s a usability test? Have someone use your site or product while you watch.
• 37. Grab someone off the street
• 38. Amazon’s Mechanical Turk - mturk.com
• 39. UserTesting.com
• 40. Places to Find Qualitative Insights
• 41. The A/B Testing Process
• 42. When should you test the small stuff? After you’ve found the big wins and you have a full-time growth team.
• 43. A/B TESTINGTools
• 44. A/B testing tools only measure the next step
• 45. Tie A/B tests back to the customer life-cycle
• 46. How do we actually do this?
• 47. First, make sure you’re using customer analytics
• 48. Then use Visual Website Optimizer or Optimizely OR
• 49. Now all your A/B testing data is in KISSmetrics

Statistics

Views

Total Views
6,247
Views on SlideShare
5,188
Embed Views
1,059

Actions

Likes
23
Downloads
183
Comments
0

12 Embeds 1,059

http://blog.getvero.com 656
http://www.zoommetrix.com 240
https://twitter.com 106
http://feedly.com 28
http://getvero.staging.wpengine.com 13
http://www.copyscape.com 9
http://translate.googleusercontent.com 2
http://tweetedtimes.com 1
https://www.inoreader.com 1
http://www.newsblur.com 1
http://digg.com 1
http://apps.activemailservice.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • Let’s take an Apple product for example.\nIt’s obvious to use Apple as a case study for user experience. Their whole business model is built on UX and delighting their customers.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Let’s take an Apple product for example.\nIt’s obvious to use Apple as a case study for user experience. Their whole business model is built on UX and delighting their customers.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Let’s take an Apple product for example.\nIt’s obvious to use Apple as a case study for user experience. Their whole business model is built on UX and delighting their customers.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Let’s take an Apple product for example.\nIt’s obvious to use Apple as a case study for user experience. Their whole business model is built on UX and delighting their customers.\n
  • Let’s take an Apple product for example.\nIt’s obvious to use Apple as a case study for user experience. Their whole business model is built on UX and delighting their customers.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

How to Run AB Tests that Get REAL Results How to Run AB Tests that Get REAL Results Presentation Transcript

  • How to Run A/B Tests that Get REAL Results Lars Lofgren Marketing Analyst - July 2012 info@kissmetrics.com - Confidential - Do not distribute
  • @larslofgren #KISSwebinar
  • We’ll cover...1 The wrong way to A/B test2 5 places to get ideas for testing3 Get the most from your A/B testing tools
  • HOW MOST PEOPLEA/B Test
  • A/B testing is totally rad! We should do it!
  • Everyone loves button colors VS
  • Next up? Layouts!
  • What about photos? VS
  • And conversions went up by 4%! Party time!
  • There’s a better way... Testing random elements is very inefficient. You will not find the big wins this way.
  • You’re also not learning Red vs green buttons doesn’t help you understand your customer.
  • GETQualitative
  • Qualitative data definition Data that isn’t a number.
  • Start with qualitative insights Use other sources of data to get a basic understanding of what your customers want.
  • Understand first, then do the A/B test You’ll focus on the right tests and increase your conversions a LOT faster.
  • The A/B Testing Process 1 Get qualitative insights 2 Predict how to improve 3 Confirm prediction with an A/B test
  • Why is this process better? Helps you find the 40% increase in conversions instead of only the 4%.
  • WHERE IS THE QUALITATIVE Data?
  • Location #1 Surveys
  • Use one-question surveys on your site
  • Also try long surveys
  • Survey best practices1 Keep it short2 Only ask for answers that you’ll use3 Start with open-ended questions
  • Location #2 Feedback
  • Do you have a feedback email?
  • Talk to your support team They know exactly what problems keep coming up.
  • Also talk to your sales team They talk to customers every day.
  • Location #3 Reach Out
  • Nothing beats talking to customers personally Get them on the phone and deeply understand how they use your product.
  • If local, take them out to lunch or coffee.
  • This is not a pitch Focus on asking good questions and listening.
  • Location #4 User Activity
  • Look at activity of individual people You’ll need customer analytics for this.
  • KISSmetrics People report
  • When you have an insight... Confirm by looking at other people.
  • Location #5Usability Tests
  • What’s a usability test? Have someone use your site or product while you watch.
  • Grab someone off the street
  • Amazon’s Mechanical Turk - mturk.com
  • UserTesting.com
  • Places to Find Qualitative Insights 1 Surveys 2 Feedback 3 Reach Out 4 User Activity 5 Usability Tests
  • The A/B Testing Process 1 Get qualitative insights 2 Predict how to improve 3 Confirm prediction with an A/B test
  • When should you test the small stuff? After you’ve found the big wins and you have a full-time growth team.
  • A/B TESTINGTools
  • A/B testing tools only measure the next step Great for testing copy, photos, and layouts. Terrible for bigger tests like pricing.
  • Tie A/B tests back to the customer life-cycle Critical for measuring how your changes impact customers over the long term.
  • How do we actually do this?
  • First, make sure you’re using customer analytics
  • Then use Visual Website Optimizer or Optimizely OR
  • Now all your A/B testing data is in KISSmetrics
  • For a demo of KISSmetrics or to learnhow KISSmetircs can help grow your business, contact Ben Sardella at: bsardella@kissmetrics.com