Successfully reported this slideshow.
Your SlideShare is downloading. ×

Implementing an A/B & Usability testing process

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 57 Ad

Implementing an A/B & Usability testing process

Download to read offline

Presented by me at Emerce Conversion 2015, Meet Magento Germany and Belgento. THis talk is about 5 internal/process tips and 5 external/ on-site tips for applying A/B testing and usability improvements to your website or webshop.

Presented by me at Emerce Conversion 2015, Meet Magento Germany and Belgento. THis talk is about 5 internal/process tips and 5 external/ on-site tips for applying A/B testing and usability improvements to your website or webshop.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to Implementing an A/B & Usability testing process (20)

Advertisement

More from Guido X Jansen (20)

Recently uploaded (20)

Advertisement

Implementing an A/B & Usability testing process

  1. 1. Implementing an A/B & Usability testing process @guido | userlegion.com
  2. 2. @guido userlegion.com Guido? 2
  3. 3. @guido userlegion.com3
  4. 4. Netherlands: May 27 & 28, Utrecht
  5. 5. @guido userlegion.com Usability? 5
  6. 6. @guido userlegion.com Usability? 6 Learnability Efficiency Memorability Errors Satisfaction
  7. 7. @guido userlegion.com Usability versus Persuasion 7
  8. 8. @guido userlegion.com A/B Testing? 8 A B C D E A B C D E A B C D E Multivariate Testing (MvT)
  9. 9. @guido userlegion.com So why even bother…? 9
  10. 10. @guido userlegion.com Selling online is tough! 10
  11. 11. @guido userlegion.com Selling online is tough! 11 Average conversion rates Offline: 20 - 25% Online: 5 - 15%
  12. 12. @guido userlegion.com Selling online is tough! 12 Average conversion rates by product category Source:  MarketingSherpa  Benchmark  2014 0% 5% 10% 15% 20% 25%
  13. 13. @guido userlegion.com Selling online is tough! 13 Some shop examples Source:  MarketingSherpa  Benchmark  2014 8% - 11% Average order value: €430,-
  14. 14. @guido userlegion.com Selling online is tough! 14 Some shop examples Source:  Marketingcharts.com 15.8%
  15. 15. @guido userlegion.com Selling online is tough! 15 Some shop examples Source:  Marketingcharts.com http://www.google.nl/imgres?imgurl=http://cdn.bargainez.com/womanwithin/store/600/woman logo&imgrefurl=http://bargainez.com/ womanwithin&h=168&w=600&tbnid=Md9NfBl9JPFALM:&zoom=1&docid=uCe3emUjdu4dkM&ei= PY6cgbAE&tbm=isch&ved=0CDYQMygCMAI 25.3%
  16. 16. @guido userlegion.com Selling online is tough! 16 Some shop examples Source:  Marketingcharts.com 40.6%
  17. 17. But Still…
  18. 18. What are users missing out on?
  19. 19. @guido userlegion.com What are online users missing out on? 19 Smell Feedback Commitment Personalization
  20. 20. @guido userlegion.com 2 x 5 tips 20 My experience with testing process Direct optimization tips
  21. 21. 5 Testing Tips Learnings from the process behind optimization
  22. 22. @guido userlegion.com Learnings first 22 Proper hypotheses Time & resources are limited Brute force vs smart testing 1
  23. 23. @guido userlegion.com Get yourself a team 23 2
  24. 24. @guido userlegion.com Optimization Team 24 Create a well balanced team CM Web/ Business Analist Functional Tester (Frontend) Developer Content (UX/UI) Designer Psychologist
  25. 25. @guido userlegion.com Supporting roles 25 Your team will need support Team Marketing Customer support (Backend) Development Traffic (SEO/SEA) Business Sponsor (shit umbrella)
  26. 26. @guido userlegion.com Optimize for the journey 26 Not just single elements 3 Experience Ordering ShippingInspiration Detailed research Service
  27. 27. @guido userlegion.com Optimize for the journey 27 Not just single elements Communicate consistently Segmentation Offsite behaviour
  28. 28. @guido userlegion.com Optimization Workflow 28 4
  29. 29. @guido userlegion.com Optimization Workflow 29
  30. 30. @guido userlegion.com Optimization Workflow 30 Input Hypotheses Analysis TestingPriority Results
  31. 31. @guido userlegion.com Optimization Workflow 31 Don’t forget: Share results Get employees and management involved
  32. 32. @guido userlegion.com Involve your users 32 5
  33. 33. @guido userlegion.com When to test? 33 Competitors Old/ New Design Sketch Test Preparation Prototype Live site
  34. 34. @guido userlegion.com What to test? 34 Setting priorities with the PIE Framework Potential Heuristics Voice of Customer P I Importance Business Traffic E Ease Development Cost
  35. 35. @guido userlegion.com What to test? 35 Setting priorities with the PIE Framework PIE Scoring Example Idea  A Idea  B Idea  C Potential .50 .20 .80 Importance .20 1 .50 Ease .40 .80 .20 PIE  Score   x  100 4 16 8
  36. 36. @guido userlegion.com How many users to include? 36 Test Preparation Source:  nngroup.com
  37. 37. @guido userlegion.com What kind of users to include? 37 Test Preparation Source:  nngroup.com Everyone
  38. 38. @guido userlegion.com Where to test? 38 Test Preparation Remote On-site
  39. 39. Test Day
  40. 40. @guido userlegion.com Let the user do the talking 40 Help getting comfortable Help explaining the case Help thinking aloud Other than that: shut up :)
  41. 41. @guido userlegion.com Review day 41 Lots of ideas > A/B testing Lots of bugs > fix it Better estimate A/B test potentials
  42. 42. @guido userlegion.com Sidenote: Toolbox 42 Project management Web analytics BI A/B testingDevelopment Raw data Visualisation Feedback UX Testing
  43. 43. 5 Direct Optimization Tips Learnings about users from watching Usability and A/B testing
  44. 44. @guido userlegion.com Get customers to return 44 Small next steps to the next micro conversion 1 Almost 50 % of users do not come back if they found it hard to find relevant information on the web site (Forrester research)
  45. 45. @guido userlegion.com Negative search / filter results 45 2
  46. 46. @guido userlegion.com Discontinued products 46 3
  47. 47. @guido userlegion.com Sidestep: context matters! 47 Perceptual contrast / framing Web only Print only Web & print $59 $125 $125 16% 0% 84% 68% 32% Source:  Dan  Ariely
  48. 48. @guido userlegion.com Direct checkout 48 + 4,77% CTR 4 Source:  Kieskeurig
  49. 49. @guido userlegion.com Mark visited products 49 or even make it a checklist 5 Source:  Kieskeurig
  50. 50. @guido userlegion.com Mark visited products 50 or even make it a checklist Checkmark in button + 3,97% CTR Source:  Kieskeurig
  51. 51. @guido userlegion.com That’s it! 51
  52. 52. @guido userlegion.com So… will it make you money? 52 Conclusion
  53. 53. @guido userlegion.com Case: UX sprint 53
  54. 54. @guido userlegion.com Conversion is not a stable factor 54
  55. 55. @guido userlegion.com Important off-site factors 55 Weather Calendar Stock/ Interest rates (National) Holidays Sport events Competitors
  56. 56. Thanks For Listening Guido Jansen guido@userlegion.com +31 6 42093976 @guido
  57. 57. @guido userlegion.com Simple, Affordable Usability Testing in your Local Language 57 Business card = Free 10 user trial :) Agency accounts available

×