Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Build a Winning Conversion Optimization Strategy


Published on

By Srikant Kotapalli – VWO

AB Testing has traditionally been one of the most commonly used methods to boost conversion rates. Many marketers have since used it tactically, that gets them some early wins but are unable to convert it into repeatable success. Building an effective optimization strategy needs an in-depth understanding of user behaviour, adopting a structured framework and constantly measuring and improving upon the results. This session will walk you through everything you need to know to transform your AB testing into a winning optimization engine.

Published in: Marketing
  • Be the first to comment

  • Be the first to like this

Build a Winning Conversion Optimization Strategy

  1. 1. Build a winning optimization strategy Srikant Kotapalli Head of Product @ VWO
  2. 2. Most commonly used methods for optimizing conversion rates
  3. 3. What is A/B testing In the most simplest form, A/B Testing is comparing two versions of a web page to see which version performs better for a defined conversion goal
  4. 4. Adding ‘Free Shipping’ threshold leads to increase in AOV by 7.3% A N E X A M P L E Variation Control
  5. 5. Is A/B Testing really working? Only Only More than 1in 8 Tests create a major impact on conversions 25 % Companies are quite satisfied with their conversion rates 7in 10 Companies do not take a structured approach to testing Source: Conversionxl and econsultancy 2016 report High frequency, low returns testing
  6. 6. What does it take to be successful at Testing?
  7. 7. A successful Optimization Engine People Process Techniques
  8. 8. Building a Team
  9. 9. Different models for a Team In-house • Completely managed by employees • Typical skills required to kick off a team - Marketing + Strategy + UX + Coding Agency/Vendor • Entire Optimization program managed by an Agency or Full service Vendor Hybrid • In-house team with certain activities outsourced to Freelancers or Agencies
  10. 10. r W h a t i s t h e t e a m s i z e f o r C R O ? Average team size >4 FTE with a strong correlation between the team size and how old the optimization program is 10% 8% 52% 31% 1 1,5 2,5 7,5 0 1 2 3 4 5 6 7 8 0% 10% 20% 30% 40% 50% 60% <6 months 6 months to an Year 1-3 years >3 years
  11. 11. r W h a t a r e a s o f o p t i m i z a t i o n a r e y o u m o s t l i k e l y t o o u t s o u r c e ? Development, Designand QAof the experiment are the top skills that companies outsource 0% 5% 10% 15% 20% 25% 30% 35% Entire Optimization Program Strategy Project Management Idea Generation User Research UI/Design of the Test Copywriting Development of the Test QA of the Test None
  12. 12. Adopting the right process
  13. 13. A Simplified Conversion Optimization Process Multiple frameworks and processes prescribed by expert practitioners all follow the same flow for optimization Research Hypothesize Prioritize Test & Learn
  14. 14. Tools & Techniques
  15. 15. r H o w m a n y t o o l s d o y o u u s e f o r c o n v e r s i o n o p t i m i z a t i o n ? Conversion optimization requires considerable expertise with multiple tools throughout the process 0% 10% 20% 30% 40% 50% Less than 3 3 to 5 5 to 10 10 to 15
  16. 16. Techniques for Research Research Hypothesize Prioritize Test & Learn
  17. 17. r M o s t c o m m o n l y u s e d m e t h o d s t o g e n e r a t e U s e r i n s i g h t s Heatmaps Session Replays Customer interviews Surveys Web analytics User Testing Expert Reviews Click testingFive second test Card sorting/Tree testing Form Analytics Difficulty in generating insightsDifficult Easy Valueofinsights Low High *Size of the bubble indicates the popularity or usage of the method
  18. 18. Heuristic Analysis Evaluation of a website by a usability expert against a set heuristics to identify potential conversion barriers
  19. 19. Heatmap Navigation bar distracting users from the primary goal of this page which is to download the app
  20. 20. Scrollmap A sharp drop-off of 40 %age points happens here and content below this is going to be ineffective. Also acts as a in page funnel representation 98% (103) views 58% (61) views
  21. 21. Session Replays A huge drop-off of 40 %age points happens here and content below this is going to be ineffective. Also acts as a in page funnel representation 98% (103) views 58% (61) views
  22. 22. Surveys Understanding Word of mouth effect on your direct traffic 98% (103) views 58% (61) views
  23. 23. Surveys Understanding why visitors abandon your site 98% (103) views 58% (61) views
  24. 24. Form Analytics Go beyond tracking form submission rates. Form analytics help you understand how visitors interact with every form field. It gives you insights on where visitors hesitate and drop off 98% (103) views 58% (61) views 27% 34% of the visitors submitted the form 20.5 seconds Total time spent on completing this field 10.3 seconds Time spent by visitors hesitating to give this data 20% of the visitors drop off on this field of the visitors refill this field
  25. 25. Usability Testing Task Analysis or usability testing helps you evaluate how visitors would use the website to achieve a desired goal. 98% (103) views 58% (61) views
  26. 26. Card Sorting Helps you understand how your visitors would logically group information into categories. 98% (103) views 58% (61) views Clear winners Ambiguous classification, might need follow up tests Tests run using Optimal workshop
  27. 27. Five Second test Helps you understand how visitors perceive your messaging 98% (103) views 58% (61) views What solution are they selling? Tests run using Usabilityhub
  28. 28. Techniques for better Hypothesis Research Hypothesize Prioritize Test & Learn
  29. 29. Build a structured Hypothesis Based on Observations I believe Solution will address Problem for Audience and impact Goal by uplift % Evidence or Research that identifies a pattern or a problem that needs solving If you are just starting to test, then Audience is probably all visitors. But as you mature in optimization your audience is a subset Brings in accountability on what your team was trying to achieve
  30. 30. Techniques for better Hypothesis Research Hypothesize Prioritize Test & Learn
  31. 31. r W h a t % a g e o f i d e a s d o y o u t e s t f r o m y o u r b a c k l o g ? There are always more ideas than you can actually test. Prioritizationof which ideas to test is essential 0% 5% 10% 15% 20% 25% 30% 35% <10% 10% - 25% 25% - 50% 50% - 75% >75%
  32. 32. Frameworks to prioritize Hypothesis Potential Importance Ease Time Impact Resources Bryan Eisenberg Impact Confidence Ease Sean Ellis PXL model by by by by
  33. 33. What we use Confidence Importance Ease How confident are you of achieving the uplift in conversion rates from this test? How valuable are the audience you are testing for? How Easy is it implement this test? Generally higher confidence can be achieved with high level of research Not just effort design and dev, but also take into account how easy is it to get all stakeholders to agree to doing this For example a test conducted on bottom of the funnel could be more important than a test on top of the funnel
  34. 34. Techniques for Test & Learn Research Hypothesize Prioritize Test & Learn
  35. 35. Measuring the right Goals Measuring the impact on Macro vs Micro conversions When creating AB tests choosing a goal to focus is about tradeoff between certainty and cost Case 1: Pricing change in SaaS Signups vs Average deal size Case 2: Changes on home page in ecommerce Product views vs AOV Case 3: Improving product recommendations Engagement vs Revenue
  36. 36. r N u m b e r o f G o a l s m e a s u r e d i n a n e x p e r i m e n t 36% of the experiments have a secondary goal that is being tracked 64% 28% 5% 3% 1 2 to 4 5 to 7 8 or more
  37. 37. Uncover learnings Say you are running a test to see if adding a price store to your category or home page increases your conversions. And find it wins for the goal add-to- carts. What do you do next? Are there segments that performed better or worse for the price store? Impact on profitability of adding a price store? Were there any segments who converted better but with a lower AOV?
  38. 38. r H o w m a n y t i m e s d o e s a f a i l e d t e s t l e a d t o a n o t h e r i d e a ? Every test, even a failedone can result in better understanding of visitor behaviorand generate follow up ideas 0% 5% 10% 15% 20% 25% 30% 35% 40% 1 in 10 1 in 5 1 in 3 1 in 2 No follow up ideas
  39. 39. Thank You! Q&A Come talk to us at Booth 4 to discuss how VWO can help with your Optimization Program A/B Testing & Conversion Optimization Platform powered by SmartStats 2.5bn+ Users served per month Goals FunnelsTrack Heatmaps Recordings Surveys FormsAnalyze Test Plan HypothesisObservation