Tracking Errors in Google Analytics to Improve Lifetime Value: Superweek 2014


Published on

Every error a visitor sees on your site is a small point of pain which will reduce the chance of them converting and also of them visiting your site again. And each error message is another point of friction which will wear away at your brand's reputation.

If you track these errors in GA you can estimate the money you're throwing away in this visit. Configure 'user' advanced segments and you may even get an idea of the impact on likelihood to return.

Read more here:

Published in: Business, Technology, Design

Tracking Errors in Google Analytics to Improve Lifetime Value: Superweek 2014

  1. 1. #spwk @timlb Tracking Errors in GA
  2. 2. Started working on web sites in 1993 In 2014 I find myself asking: “After 20 years of the web, why are so many sites still so hard to use?”
  3. 3. For 20 years we have been ● Building web sites ● Using web sites ● Measuring and testing web sites But sites are still… ● Hard to navigate ● Difficult to understand ● Counter-intuitive ● Sometimes broken And full of… Errors!
  4. 4. Question: what is ‘bad’? Web analysts start by asking the business: “what is ‘good’?” Answer defined as desired commercial outcome: ● ● ● ● orders / revenue leads sign-ups content-views Web analyst then defines GA goals to match We then measure ‘good’ in terms of percentage of visits from various channels which complete goals Easy to then think ‘bad’ = ‘not converting’
  5. 5. The analytics trap Tempting to measure ‘good’ in ways which encourage us to ignore the site itself Our numbers seldom record the things that people using the site would complain about (Or praise!) The conversion rates in GA can say that a site is ‘good’ in terms of macro-goal completion People using it might say it was ‘bad’ meaning that they found it difficult to do what they wanted
  6. 6. So, why track errors? Reduce errors = make sites ‘less bad’ Errors can be useful indication of usability issues ● ‘Usability’ normally = lab tests or remote usability testing = sampling & not real visitors ● ● Error messages = real visitors, no sampling Error messages = measurable
  7. 7. Errors are good (for insights) Consistent, measurable, indicator of usability problems So a valuable source of insight and prioritisation for usability improvements Benefits ● Short term measurable improvements (online, this session) ● May well have more important long-term benefits ○ Lifetime value ○ Brand perception off-line and on-line
  8. 8. Each error is a small cry of pain from your visitor Each error is a point of friction which damages your brand So, how many errors do your visitors see? In most cases you won’t find the answer in GA (or other WA tools)
  9. 9. How to track errors in GA What can you already track? How to track the others
  10. 10. Universal Example of Error: 404 page not found You can’t stop people posting corrupt links, so every site will have 404 pages ● Ugly truth: in real life you may even have bad internal links ● Ugly truth: your redirects to mobile site may break
  11. 11. Can you count 404s in GA? Early webmasters used to check for 404 errors and fix links Now some sites make it hard to see 404 errors Examples: ● ● ● Wordpress & Magento serve 404 page using the requested URI But Title field usually includes ‘not found’ Tip: use a view filter to add 404 detail to URI
  12. 12. Quick 404 test 404 is so common that people compile galleries of examples and write blog posts on ‘best practice’ But are these pages tracked? Helpful example of a good 404 gallery: Test: visit sample sites requesting and see what is sent to GA
  13. 13. Of 11 sites, 2 could not even see 404 in GA 6 could only see 404 in Title Would need view filter for goals
  14. 14. How does tracking 404 help Fix any internal broken links (bonus hint: look for redirects to mobile site causing 404s) Contact external sources and suggest fix Experiment with more helpful 404 page to improve: ● ● ● bounce rate exit rate conversion rate for visits including a 404
  15. 15. Moving on from 404s Checkout forms are another common source of errors What’s your most common error? What’s your most annoying error?
  16. 16. And the big checkout errors are… Not what you’d expect?
  17. 17. What if it really is “user error” How do you deal with ‘email already exists?
  18. 18. Examples of errors which can be fixed ● Very valuable: Confusing promotional codes ● Mandatory fields ○ Is it really needed ○ If yes, can it be made more obvious ● Formats for post codes and phone numbers ○ Use better validation and intelligent templates ● ● ● ● Requiring selection of options when there is only one Requiring double-selection of option (e.g address lookup) No default quantity on product pages Site search 0 results ○ nb search ‘enter search term here’ prompt
  19. 19. So, where do we start with GA? First: check if any other errors visible in GA Few errors visible in GA since they are dynamic elements within the page, or even modal dialog boxes But check in case page reloads in a way which can be identified in GA Example: Magento onepage checkout login fail reloads with /index appended to URI (and note unique pageviews…)
  20. 20. If not in GA: what we need to know? Need to add custom GA tracking in most cases to record ● What the error was ● Where the visitor experienced it Prioritise errors if you’re adding to an existing site and need to limit budget or are very worried about hit quota ● ● Start with pages with high pageviews Check usability tests to see which errors are common
  21. 21. Big decision: Virtual Pageviews or Events VPV For ● ● ● ● ● For Can be used as goal Can be seen in funnel visualization Has an exit rate Can be seen in flow reports Can be used in content grouping Against ● ● ● Events Inflates pageviews Need to track page explicitly Breaks convention of using pageviews for units of content which correspond with visitor seeing a new chunk of material ● ● ● ● ● ● Can be used as goal (now) Doesn’t inflate pageviews Built in reporting of event page Can be seen in behavior flow Category / Action / Label structure Event value can be used in goal Against ● ● ● Not visible in funnel visualisation No exit rate Cannot be used for Content Grouping (unless you explicitly code them)
  22. 22. How to track “What the error was” Simple version: Record verbatim text of message + Easy to specify and treat as code library: “if this error text is displayed, then send it to GA” + Standard rule means cheapest to implement throughout site on a new build - Reports can be difficult to read: too many words - If same text is used multiple times on same form (‘invalid value’) then report is less valuable Last two points = lesson on quality of error message Real text means reports are constant reminder of how bad micro-copy is! Advice on good error messages here: Real-life tip: it may be worth adding a unique reference code to each error if there are multiple versions of same error on same page
  23. 23. Where the error happened: VPV directory structure thoughts Choice: Think about: Start with path or error? Content Drilldown Report Content Grouping /original/path/errors/’the message’ (like page’s own URI) or /errors/original/path/’the message’
  24. 24. Reporting on errors in GA Goals and Intelligence Alerts Dashboards Custom Reports (Segmentation) (Content Grouping)
  25. 25. Why use Goals? Powerful standard reports ● ● Conversion rate everywhere in GA Reverse Goal Path Report ○ ○ ● What were people looking at in steps before error Very useful for internal broken links! Goal URLs Report ○ Great if Goal has been configured to match more than one error
  26. 26. Bonus: Intelligence Alerts ● Configure an Intelligence Alert for key ‘error’ goals ● Set up an automatic daily email alert ○ ○ ● Will arrive far too late in this timezone But still useful in case you did not check or notice Special reason: the Intelligence Alert system will attempt to identify the source of the change for you ○ Not always successful but sometimes works very well
  27. 27. Choice: which view for goals Should you use a separate view for ‘error goals’? ● For: if you normally use the GA ‘All goals’ rolled-up metrics then having goals for ‘bad’ things will mess up your reports ● Against: if you keep ‘error goals’ outside your normal working views, how often will you check (e.g. you can’t include in standard dashboards) I don’t use ‘all goals’ so I prefer to have ‘read and react’ goals in working views
  28. 28. Examples of Read and React Goals Characteristics: ● ● ● You can take action in hours Can change rapidly from day to day Can be caused by off-site or user-generated content 404 page ● ● ● ● If conversion rate jumps Find source of broken link Fix if on-site Contact source if off-site (or post good link if it’s user-generated content and use chance to engage with potential visitors!) Rejected promo code ● ● ● If ‘bad’ code is common because it’s confusing (0/o/O, i/I/l) configure an extra matching code Learn lesson and don’t use ambiguous codes If ‘bad’ code is one-off on user-generated site visit the site, apologise and post a ‘good’ code (engage with visitors)
  29. 29. Dashboards: really powerful Great example of when to use a Real Time Widget: ● ● ● ● ● ● ● ● ● ● RT: active visitors with errors RT: the active 404 error pages RT: the other active error pages External source/medium of errors External referring sites (only) Adwords target errors Error conversion rate Error completions Source of internal bad links (Goal previous step!) Bad link requested (Goal completion URL) Include just conversion in other dashboards
  30. 30. Custom Reports: Vital Filter to focus on what counts
  31. 31. Pick only the metrics you need
  32. 32. Those checkout errors
  33. 33. Comparison view?
  34. 34. Actual numbers = actual money
  35. 35. Different people - (source/medium drill down)
  36. 36. Segment ideas ‘User’ scope segments: Potential to compare effect of error messages over different visits ‘Sequence’ segment rules: Possibility for looking at impact of errors at different stages of visit Original segments: Great for looking at impact of with/without error(s) on macro-conversion
  37. 37. Content Grouping Thoughts Same question as with VPV path Group errors with original page or Group errors as dedicated group
  38. 38. Summary Errors are a proxy for usability problems Each error message is a small cry of pain from your visitor Each error message is a tiny dent in your brand’s reputation Tracking errors in GA lets you see where the points of friction are Measuring errors allows you to prioritise improvements Improvements likely to bring enough short-term income to pay for work Real big pay-off will be in customer lifetime value in multi-channel world
  39. 39. Resources Bundle of all my Superweek2014 Links: Great article on form validation: Includes link to Luke Wroblewski post “When compared to our control version, the inline validation form with the best performance showed compelling improvements across all the data we measured. Specifically, we saw: ● ● ● ● ● a 22% increase in success rates, a 22% decrease in errors made, a 31% increase in satisfaction rating, a 42% decrease in completion times, and a 47% decrease in the number of eye fixations.” Great advice on error messages: My first blog post on the subject: Includes very useful discussion thread