Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

2017 06-test withintelligence-conversionelite

855 views

Published on

Full deck from Conversion Elite 2017 with additional slides and further on slide notes added. Covers the concept of understanding where you are in the user relationship and which additional metrics to consider when planning tests as well as those which you are trying to optimise in an MVT or AB test

Published in: Data & Analytics
  • Hello! High Quality And Affordable Essays For You. Starting at $4.99 per page - Check our website! https://vk.cc/82gJD2
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

2017 06-test withintelligence-conversionelite

  1. 1. Test with Intelligence Using Data for Decisive Testing
  2. 2. Convert Intent
  3. 3. Into Action
  4. 4. Who is this intended for? The Conversion Elite
  5. 5. Not General Marketers
  6. 6. Conversion Specialists like me
  7. 7. Conversion Specialists like me
  8. 8. Not Data Scientists
  9. 9. Know Analytics & KPIs
  10. 10. But this is not a “How To Set Up Analytics”
  11. 11. This will not teach you Query Folding and Data Modelling
  12. 12. Credit you with the intelligence to investigate or delegate
  13. 13. If that is not you
  14. 14. What is in this presentation? Thinking about the data & detail needed to plan for better results
  15. 15. What is in this presentation? Not just measuring Traffic & Sales Considering WHAT is a positive outcome
  16. 16. What is in this presentation? Leveraging micro conversion data Understand different parts of the research and buy user journeys
  17. 17. What will you get as a result?
  18. 18. Understand how to turn user behaviour data into value
  19. 19. Think through factors that may impact test results
  20. 20. Understand causes of bottlenecks on key templates
  21. 21. Why is that important? Improved understanding of data aids performance Test uplift can still fail to meet business KPIs Understand business data Investigate User data
  22. 22. Makes on-target delivery easier
  23. 23. Growth is limited If Optimisation strategy doesn’t fit
  24. 24. Why is that YOUR problem?
  25. 25. If business growth targets are missed If Revenue targets fall short Optimisation costs come under more scrutiny
  26. 26. The business problem IS your problem
  27. 27. It quickly becomes everyone’s problem
  28. 28. You can do something about it
  29. 29. You can do something about it Because you solve the customers problems
  30. 30. How will you feel at the end?
  31. 31. Inspired
  32. 32. Excited
  33. 33. Exhausted
  34. 34. Data can be sexy
  35. 35. Disclaimers The stunts you are about to see have been performed by a trained professional using the appropriate safety equipment It may contain • Bad puns, jokes and visual gags • Deliberately silly and inappropriate stock imagery • Some NSFW references and images • Tortured analogies and stretched metaphors
  36. 36. You must be this Tall to join the ride
  37. 37. Do not try this at home
  38. 38. Do not try this at home But do try this at work
  39. 39. Tools & Stacks
  40. 40. Google Analytics API Script or paid plugins like
  41. 41. Google Analytics API to Excel, csv or database Also API for Facebook, Twitter, AdWords, Search Console & more Allows scheduled and batch data pulls
  42. 42. Google Analytics API to Excel, csv or database Also API for Facebook, Twitter, AdWords, Search Console & more Allows scheduled and batch data pulls Also has save to Drive or Cloud database options
  43. 43. Combine Business & Web Data. Deeper Analysis and visualisation PowerBI also has API link to Google Analytics API etc. PowerQuery and PowerBI have JSON API (make your own API query) Also have csv folder extract for batch processing
  44. 44. Bigger datasets, more powerful statistical analysis PowerBI cloud has real language querying “What is the best performing product for revenue this Quarter?”
  45. 45. Geo lookup or CRM for customer addresses (or combine both) Visualise on drill down map or overlay ArcGIS data e.g. Conversions > Avg from Areas with Household income > average
  46. 46. Tableau, R or PowerBI to look at and query distribution patterns Average alone can mask opportunity & risks Skewed distribution or Revenue to consider for Test “winners” Best for your lowest value, least loyal customers, worst for your big spenders Often not a business win, even if it reports as a Test win
  47. 47. User & Session Recording
  48. 48. Also, you know. Maybe talk to them Ask Customer Services Look at Feedback form data Jump on a Live Chat
  49. 49. Hit level – Measurement Protocol
  50. 50. Gets a little spendy at the top end but Free Option is good to try out
  51. 51. Traffic is vanity
  52. 52. Do you Agree?
  53. 53. Conversion Rates are vanity
  54. 54. Do you Agree?
  55. 55. Profits are Sanity
  56. 56. Do you Agree?
  57. 57. Relationships develop Value
  58. 58. Do you Agree?
  59. 59. Shopping Analogy Google Analytics in Real Life Playlist http://bit.ly/GA-irl
  60. 60. Shopping Friction – UX choices Mechanical – Basic Functionality and Accessibility Look for those, minimise Friction Yes, make sure the Shop actually works (for the user)
  61. 61. Dating not Shopping If talking profit and business growth Its about customer relationships Particularly if we are talking lifetime value from ecommerce For Lead Gen, Media Owner, SaaS … Thinking Relationship works better to understand what to measure What Measurement matters, depends on what Relationship
  62. 62. What type of relationship are we trying to model? What are the metrics that will show it developing? What sort of customers do we attract? What sort of customers do we want to attract? What Type of Conversion?
  63. 63. Tantric Lifestyle Identity Affirming
  64. 64. Relaxed Casual FWB
  65. 65. Er.. Fun?
  66. 66. Seeking Longer Commitments
  67. 67. Finding Longer Commitments
  68. 68. Quick Fling
  69. 69. Time to Convert?
  70. 70. Forgetting something?
  71. 71. Times have changed
  72. 72. Foreplay
  73. 73. Metrics and Measurement
  74. 74. Micro Conversions Next page Clicked to View Larger Clicked Reviews Tab Checked Stock Showed interest by playing with your drop downs If they aren’t Adding or Checking out What are they doing? • Which of them relate to positive outcomes? • Which indicate they are unsure on the relationship on offer?
  75. 75. Yes? Little confirmations build up How do those first touches feel? Do they still seem a little unsure? Did your change not increase Add to Basket But increased use of Shipping estimator? Tempted them but not convinced yet Measuring other outcomes provides insight
  76. 76. Yes! You don’t sell the final decision first You sell the next page, the not-Bounce, the not-Exit They start to warm up to your offers
  77. 77. Yes!! Encourage them to use their fingers a little more To spend a Click to explore deeper Deliver on what you promised from each Click until…
  78. 78. YES!!! You build lots of little endorphin hits… into one moment of ….completion
  79. 79. Don’t disappoint with an over eager CTA Oh they didn’t bounce? They must love my offer Buy me NOW. BUY… oh…Bye Oops, sorry, this never usually happens…
  80. 80. Don’t blindly “Test CTA Copy” Think about what action you are calling to happen • Will you really deliver Buy next? • Are they Ready to Buy? • Does the rest of the page fit that message? • Has their journey primed them to want to take that Action? When is the right time? When is right for this type of relationship? What extra data will help to understand this? Add that.
  81. 81. Keep momentum Measure the appropriate metric
  82. 82. Quantify Value of Changes IF fixed THEN [this] is the range of Potential Value or Risk Avoided Not a “Quick Fix” or uplift on a “Test Metric” Identifying optimal relationship value is a longer term process You get a first date, you get the next date You update your Facebook status to Its complicated Maybe you get lucky first time; it can and does happen But to get lucky again you need to put in the work
  83. 83. Flirt
  84. 84. Intent and Attraction Where they are from is not just a Marketing problem Traffic patterns and motivation can impact test planning and reporting
  85. 85. Venue & Context Where you met How you interacted What you promised All factor into the journey to conversion Will change what you measure to progress further
  86. 86. First Impressions Do Count What can you measure as the baseline? Flirt (Marketing) or First Date (Landing) General Rules:
  87. 87. 1. Be Attractive
  88. 88. 2. Don’t be Unattractive
  89. 89. Attraction is Subjective Are you attractive to YOUR target users? Don’t feel it should work because You like it Measure if THEY like it Users have varied ideas of attractive And their reaction counts most
  90. 90. Sometimes very er… Specific
  91. 91. Long Tail Traffic?
  92. 92. Landing Pages The S of your Advertising AIDAS Attention, Interest, Desire, Action … on click do they land on Satisfaction • Does this deliver what you promised with your Flirt? • Do they start the journey satisfied? It is easier if they do • Have they had their first little Yes? Every page is a Landing Page (almost) Think of it as the Venue for your First Date Context and Audience appropriate
  93. 93. Bounce Traffic “I came, I puked, I left” Avinash Kaushik Stood up for your date Expectations mismatched Profile pic vs Reality Total Loss? Maybe not, but that’s another area to investigate Because the user, usually has an alternative
  94. 94. Plenty more fish
  95. 95. Non Bounce Traffic Actual interest? Yes this looks like something I want? For now this is: Failed to Lose They did something you can actually measure But in the early stages that’s all you can say for sure They didn’t bounce. Or your analytics setup is faulty If it was a positive reaction – a positive forward action - then non-Bounce Non-Bounce, interpreted as positive forward Action = Turned up to your date.
  96. 96. Your initial offer got a bite You have them hooked (for now)
  97. 97. Homepage
  98. 98. Homepages are Noisy The absolute worst page to optimise Unless you have a single clear offering Homepage is trying to pull in a nightclub All flashing lights, noise and moving images
  99. 99. Desperate Much? Desperate is never a good look What is a positive step here? Clicking ANYTHING? Only interested if you are easy? More valuable if they click • Sale Banner? • Category? • Brand? Didn’t Bounce - will not tell you much
  100. 100. Mobile Homepage Eek! Half Price sale: Best at Forward Clicks? Or because it’s the only one they see?
  101. 101. Trying to please all the people • Returning Regulars • New to Vaping Hardware, Juices, Coils, Info Some want a quick restock Some are completely lost Who does this page serve best? How do Clicks Forward split? New | Returning? Desired Outcome?
  102. 102. Mobile Less Noise But not Better Worse Especially as Lost New Vaper Is mostly on Mobile
  103. 103. Homepage 3 x3 grid Every decision is friction Each distraction slows the first Yes Simplify the concept: 3 x 3 equal grid Most important things on site? • For Business • For User (and which ones) At a Functional level: What is NEEDED by the USER on homepage Set aside design & promotion bias – 9 equal slots. What helps the user most? What is currently less obvious but clicked lots?
  104. 104. Weighted Grid Not all priorities are equal Now 9 must-have items: but weight them Top products may still work in not-Top positions Lagging Products may benefit from emphasis Negative space can draw attention to key areas Signpost the areas most users need first Net effect of more areas appealing = more clicks Lose on some, gain on others. Net Gain on page Test to improve your Not-Bounce Positive Forward total
  105. 105. Business Focus Every business priority above fold “Which Device” as key focus Hardware = Most Profit 3 panels plus the main Navigation Subscriptions = New Potential Does this help those Lost New Vapers? Or assume knowledge of where to go?
  106. 106. User Intent Focus Who am I? Which path for me? Left vs Right split Starters | Pro Users Eliquids | Hardware • Equally valid for both Eliquid click & purchase frequency MUCH higher • Middle of page • Whitespace for clarity • Supporting Text “Help to choose?” Hardware demoted and again split Starter | Latest Measure clicks per section by Intended audience
  107. 107. What won? Indecision Impatience
  108. 108. I told you Homepages were shit to optimise What won?
  109. 109. Lesson? Let the Wookiee win
  110. 110. First Date
  111. 111. Product List Page Weird There’s a sale on They should have mentioned that PLP is the Get to Know You stage Some people spend a lot time here How much time? Why? What tempts them to take it further?
  112. 112. Venue & Context It depends on the relationship they plan to have What your proposition is Do they need to browse? Is that part of the appeal and courtship? Or are they trying to get to your goodies ASAP?
  113. 113. Intent By Type Again consider how you first met New vs Returning Channel Devices Product List Page Behaviour varies a LOT Segment out interaction metrics to look for patterns On a varied or non-Normal distribution catalogue Split out by Content Groups or Categories
  114. 114. Things escalate
  115. 115. Interest rises
  116. 116. Interest rises! Are they really showing they are ready for … Brown chicken, brown cow (say it fast)
  117. 117. Or…Considering Options Maybe Lots of PLP activity is not escalation • Browsing • Comparing • Sizing up your proposition Is there lots of filter use? • Which Filters? Is there a Compare function? • Do they use it?
  118. 118. Or… are they lost? Too much choice Poor signposting means they are hunting not finding Too little help for where they are in the relationship
  119. 119. Can’t see what they want?
  120. 120. Is it in an odd place? Taxonomy IA issues
  121. 121. Weak Signal Not seeing Product Only seeing part of Proposition Not reaching first Decision point
  122. 122. Your User Journey Not theirs Does this show issues with Information Architecture? You know your way from A to B – it’s your site Is poor signposting making the User work too hard? Are you looking to optimise a path they simply don’t take
  123. 123. Are you measuring Lost? Drop off points – particular Categories or # PLP loads Lots of Pathfinding • Frequent ‘Reset’ to Homepage or Category Level • High Filter/Sort Clicks but Low Product Clicks
  124. 124. Product List Page • Behaviour shows Intent • Audience provides Context Lots of Category & Top Nav resets Very few New users Add from PLP Return Users more likely to Add here Mobile users almost never (either)
  125. 125. Second Date but no kiss
  126. 126. Common to New and Returning Common to both – Drop off to Product Detail View • Basket Add at similar levels • But they Add in different places • Are you tracking WHERE your Adds to Cart happen or just Total?
  127. 127. How much effort for that first kiss? That is some funky bad breath And impossible on Mobile – our New & Lost Vapers [animated version shows three mandatory drop downs before Add enables] [CTA buttons only show on mouse over]
  128. 128. Multi Adds are rarer but do happen Multi Adds most common on Return • Some Outliers • 2 users > 130 items • These 2 users would be in 2nd place for Items Added “Average” Adds per user is misleading Do you know your distribution? Largest Bin by volume/value? Mix of: • New Browsers/Lost: 1 or 2 adds each • Return Restockers: 2 to 130 (3-5 being the Median)
  129. 129. Product List Page User Survey Feedback Lots of extra Detail Clicks just to see what flavours in Liquid because it is not clear on PLP “Pro” Restock-Users switch to List view where Add to Cart is visible without hover: Click, Click, Click, Checkout Users often reset Categories hunting for similar flavours that are in different Categories e.g. Lemon in Fruits, Lemon Cheesecake in Bakery Text to explain flavours on PLP? Or Flavour Note Icons on PLP • Can we make them filters? Can we get rid of stupid mouseover?
  130. 130. Product List Page Wireframe New Layout • Bring Results to top of page • Move filter nav to left • Compact listings • More above fold Quick Add or Read more visible on load • No mouse over needed Flavour Notes text • Click and it will filter to similar • Brings results from all Categories
  131. 131. PLP New Version Most ideas tested well and implemented • PLP total clicks reduced – less hunting • Category Resets reduced • PLP Adds increased +34% Users get what they want, more quickly Items Added Per User increased • New Users closer to 2 • Return Restockers 4 – 6 AOV and Revenue impact… big
  132. 132. Product Detail Consider this heavy flirting More than just superficial stuff They don’t want to just sort by Most Popular They want to know your Full Specifications baby What previous romantic interests thought about your goods & delivery
  133. 133. Product Detail Page We’ve barely started Literally 5 seconds after load But you want feedback?! Feedback: • Don’t stop me doing what I want • Now is not appropriate If you do this 1. Track how fast people close it 2. Track how many people leave 3. Don’t do this (so early)
  134. 134. Product Detail Page Major Drop out point – some was users from PLP looking for flavour detail The Rest? If they don’t click Add, what ARE they clicking?
  135. 135. Same painful 3 drop down selection as was on PLP Minimum 7 clicks to Add to Cart was the big heat concentration But Check Available Stock was nearly as highly used Why? Because site logic meant user only told Out of Stock after they confirmed options Still want to Add? Then you need to Check Stock to see what combinations ARE in stock
  136. 136. Little investigation into when this happened most, to which type of users and how it impacted Revenue.
  137. 137. This required being able to split out by Product SKU and Category to work out where it happened most This also meant it could be filtered In or Out of tests where Add to Cart metrics were being used But this required a little more work than just using the Analytics UI – example Data Model to split this type of detail out
  138. 138. This can then be crunched – in this case in Tableau – and Category & Product Performance compared through the steps Some obviously Popular products get lots of Views, lots of Clicks, much lower Purchases Most often out of Stock – revenue (and conversion rate) impact can be quantified. Plus: Most Popular with New First Daters Eek
  139. 139. Track Failure not just Success
  140. 140. New Stock Less No, No, Damn More Yes Yes YES Sometimes fixing conversions is about fixing the business processes not the website Measure errors, test to reduce frustration & understand why users don’t get to Complete
  141. 141. PDP - New • Thumb friendly No Drop Downs • Stock Key shown on load • Flavour Notes Plus change in production process • Data means Popularity better predicted • Range & Options drastically simplified • Stock levels more consistent • Out of Stock now only end of line or sell-out promotions (i.e. deliberate)
  142. 142. Time to Convert
  143. 143. Practice Safe Conversion
  144. 144. Reassure, don’t frighten
  145. 145. Login | Register | Guest A common debate “Guest is best Practice” Depends on many factors There is no one Right answer
  146. 146. But there are some horrifically bad standard templates
  147. 147. Approach 1: Default to Guest Make other options clear Minimise CRM data loss Still ask for basic opt-in
  148. 148. I’d Like to Register Even though there is NO incentive or benefit explained
  149. 149. Welcome Back? Nice to see you again? Your loyalty discount is ready?
  150. 150. Approach 2: Default to Guest Defer Account Create until later (because the ONLY difference is password in terms of the effort required) Or Login (but we won’t say Hi or show Guest users what they are missing)
  151. 151. Approach 3: Default to Login/Returning Brave move? It depends How many of each user type do you have? What is the acquisition & growth strategy? New Customer OR Guest is treated as one (because again, only 1 field of extra effort) And again No benefits shown (Login to see your discount points?) No Welcome to Return Users In relationship terms: If your FWB blanked you next time Or you saw them blanking a previous conquest How attracted does that make you feel? They can still walk away at this point. Many do.
  152. 152. Approach 4: Guest Only as Default Straight to business Give me all that field filling action right now (and I won’t touch your dirty PO Box) Slight hint of Benefit of an Account If you had an account, this isn’t needed Not that compelling though, is it? 4 different approaches 4 different “best practices” But which of these looked at User Types? Which of these thought about feel of the message When it is time to convert, don’t stop the flirt
  153. 153. Which is the best approach? What wins tests? Usually whichever users you have most of New or Returning? Quick one-off Guests, occasional FWB, Loyal Lovers? Do you plan sample volumes, design, metrics & segment for that? What benefit is there in having an account? Do you tell people? Don’t Assume. Tell. Show. Make it look GOOD Make Guest users want to be in Team Return next time? Are you One Night Stand or Lifetime Value focus? Are they? At the door to the bedroom… willing to Convert ….And the message you send is … this > > One Night Stands only I may remember you next time But I won’t treat you any differently In fact I might put you behind the next Guest “Best” practice? SRSLY?
  154. 154. Conversion
  155. 155. Mutually fulfilling
  156. 156. The deed is done You got your Transaction You generated that Lead hard Next meeting you can brag about how you Returned that Investment Well done you. Slow clap One more question: How does your partner in this dance feel? Also happy? Transaction Regret? Buyers Remorse? Already adding your email to their spam folder?
  157. 157. Transaction Satisfaction Your Next Flirt. Your Next opportunity to Convert Starts NOW. It will depend on the memory of a low-friction good time. Delivered on time and in nice packaging The Hug, the warm afterglow You got what you want, you made sure they got what they wanted too. Didn’t you? You’ve promised to follow up in in 3 – 5 working days “I’ll call you, real soon babes” (24 hours if you convert before 5pm and pay for next day service, except in the Outer Hebrides and the Channel Islands. Terms & Conditions may apply) Did you? Your next Cost Per Acquisition will be lower if you have delivered If you did, you won’t have to handle as many objections If you are a known quantity for satisfaction, even if they see better offers, CPA will be lower
  158. 158. Or do they feel a little railroaded? Worried about what happens next?
  159. 159. Mutually fulfilling One final YES • Your Order is Safe • Thank You • Now Social Me • Call me if you need me • Email Thanks • Reassure on delivery
  160. 160. Retention Avoid complacency Keep looking sharp Treat them better than your new side piece Become that reliable partner Even if they do stray, they’ll be thinking of you And you can win them back
  161. 161. Revenue is not the only Goal You can also measure and test User Guides, Customer Service & FAQ This is quite often more for your benefit than theirs This is how you weed out the unsuitable suitors Reduce customer service costs Guide your potential partners to know how to handle you the way you need to be for best results This saves you time, customer service costs, returns costs

×