1.
#spwk
@timlb
Tracking Errors in GA

2.
http://www.bryaneisenberg.com/hidden-secrets-of-amazon/
Started working on web
sites in 1993
In 2014 I find myself asking:
“After 20 years of the
web, why are so many
sites still so hard to
use?”

3.
For 20 years we have been
●
Building web sites
●
Using web sites
●
Measuring and testing web sites
But sites are still…
●
Hard to navigate
●
Difficult to understand
●
Counter-intuitive
●
Sometimes broken
And full of…
Errors!

4.
Question: what is ‘bad’?
Web analysts start by asking the business:
“what is ‘good’?”
Answer defined as desired commercial outcome:
●
●
●
●
orders / revenue
leads
sign-ups
content-views
Web analyst then defines GA goals to match
We then measure ‘good’ in terms of percentage of visits from various channels
which complete goals
Easy to then think ‘bad’ = ‘not converting’

5.
The analytics trap
Tempting to measure ‘good’ in ways which encourage us to
ignore the site itself
Our numbers seldom record the things that people using
the site would complain about
(Or praise!)
The conversion rates in GA can say that a site is ‘good’ in
terms of macro-goal completion
People using it might say it was ‘bad’ meaning that they
found it difficult to do what they wanted

6.
So, why track errors?
Reduce errors = make sites ‘less bad’
Errors can be useful indication of usability issues
●
‘Usability’ normally = lab tests or remote usability testing
= sampling & not real visitors
●
●
Error messages = real visitors, no sampling
Error messages = measurable

7.
Errors are good (for insights)
Consistent, measurable, indicator of usability problems
So a valuable source of insight and prioritisation for usability improvements
Benefits
●
Short term measurable improvements (online, this session)
●
May well have more important long-term benefits
○ Lifetime value
○ Brand perception off-line and on-line

8.
Each error is a small cry of pain from your visitor
Each error is a point of friction which damages your brand
So, how many errors do your visitors see?
In most cases you won’t find the answer in GA
(or other WA tools)

9.
How to track errors in GA
What can you already track?
How to track the others

10.
Universal Example of Error:
404 page not found
You can’t stop people posting corrupt links, so every site
will have 404 pages
● Ugly truth: in real life you may even have bad internal
links
● Ugly truth: your redirects to mobile site may break

11.
Can you count 404s in GA?
Early webmasters used to check for 404 errors and fix links
Now some sites make it hard to see 404 errors
Examples:
●
●
●
Wordpress & Magento serve 404 page using the requested URI
But Title field usually includes ‘not found’
Tip: use a view filter to add 404 detail to URI

12.
Quick 404 test
404 is so common that people compile galleries of
examples and write blog posts on ‘best practice’
But are these pages tracked?
Helpful example of a good 404 gallery:
https://creativemarket.com/blog/2013/07/22/the-best-404-pages-on-the-internet
Test: visit sample sites requesting
http://thesite.com/404pageTrackingTest
and see what is sent to GA

13.
Of 11 sites, 2 could not even see 404 in GA
6 could only see 404 in Title
Would need view filter for goals

14.
How does tracking 404 help
Fix any internal broken links
(bonus hint: look for redirects to mobile site
causing 404s)
Contact external sources and suggest fix
Experiment with more helpful 404 page to
improve:
●
●
●
bounce rate
exit rate
conversion rate for visits including a 404

15.
Moving on from 404s
Checkout forms are another common source of
errors
What’s your most common error?
What’s your most annoying error?

16.
And the big checkout errors are…
Not what you’d expect?

17.
What if it really is “user error”
How do you deal with ‘email already exists?

18.
Examples of errors which can be
fixed
●
Very valuable: Confusing promotional codes
●
Mandatory fields
○ Is it really needed
○ If yes, can it be made more obvious
●
Formats for post codes and phone numbers
○ Use better validation and intelligent templates
●
●
●
●
Requiring selection of options when there is only one
Requiring double-selection of option (e.g address lookup)
No default quantity on product pages
Site search 0 results
○ nb search ‘enter search term here’ prompt

19.
So, where do we start with GA?
First: check if any other errors visible in GA
Few errors visible in GA since they are dynamic elements within the page, or
even modal dialog boxes
But check in case page reloads in a way which can be identified in GA
Example: Magento onepage checkout login fail reloads with /index
appended to URI (and note unique pageviews…)

20.
If not in GA: what we need to know?
Need to add custom GA tracking in most cases to record
● What the error was
● Where the visitor experienced it
Prioritise errors if you’re adding to an existing site and need to limit budget or
are very worried about hit quota
●
●
Start with pages with high pageviews
Check usability tests to see which errors are common

21.
Big decision:
Virtual Pageviews or Events
VPV
For
●
●
●
●
●
For
Can be used as goal
Can be seen in funnel visualization
Has an exit rate
Can be seen in flow reports
Can be used in content grouping
Against
●
●
●
Events
Inflates pageviews
Need to track page explicitly
Breaks convention of using pageviews for
units of content which correspond with
visitor seeing a new chunk of material
●
●
●
●
●
●
Can be used as goal (now)
Doesn’t inflate pageviews
Built in reporting of event page
Can be seen in behavior flow
Category / Action / Label structure
Event value can be used in goal
Against
●
●
●
Not visible in funnel visualisation
No exit rate
Cannot be used for Content Grouping
(unless you explicitly code them)

22.
How to track “What the error was”
Simple version:
Record verbatim text of message
+ Easy to specify and treat as code library: “if this error text is displayed, then send it to GA”
+ Standard rule means cheapest to implement throughout site on a new build
- Reports can be difficult to read: too many words
- If same text is used multiple times on same form (‘invalid value’) then report is less valuable
Last two points = lesson on quality of error message
Real text means reports are constant reminder of how bad micro-copy is!
Advice on good error messages here: http://bit.ly/SuperWeek2014ErrorIdeas
Real-life tip: it may be worth adding a unique reference code to each error if there are multiple
versions of same error on same page

23.
Where the error happened:
VPV directory structure thoughts
Choice:
Think about:
Start with path or error?
Content Drilldown Report
Content Grouping
/original/path/errors/’the message’
(like page’s own URI)
or
/errors/original/path/’the message’

24.
Reporting on errors in GA
Goals and Intelligence Alerts
Dashboards
Custom Reports
(Segmentation)
(Content Grouping)

25.
Why use Goals?
Powerful standard reports
●
●
Conversion rate everywhere in GA
Reverse Goal Path Report
○
○
●
What were people looking at in steps before error
Very useful for internal broken links!
Goal URLs Report
○
Great if Goal has been configured to match more than one error

26.
Bonus: Intelligence Alerts
●
Configure an Intelligence Alert for key ‘error’ goals
●
Set up an automatic daily email alert
○
○
●
Will arrive far too late in this timezone
But still useful in case you did not check or notice
Special reason: the Intelligence Alert system will attempt to identify the
source of the change for you
○
Not always successful but sometimes works very well

27.
Choice: which view for goals
Should you use a separate view for ‘error goals’?
●
For: if you normally use the GA ‘All goals’ rolled-up metrics then having
goals for ‘bad’ things will mess up your reports
●
Against: if you keep ‘error goals’ outside your normal working views, how
often will you check (e.g. you can’t include in standard dashboards)
I don’t use ‘all goals’ so I prefer to have ‘read and react’ goals in working views

28.
Examples of Read and React Goals
Characteristics:
●
●
●
You can take action in hours
Can change rapidly from day to day
Can be caused by off-site or user-generated content
404 page
●
●
●
●
If conversion rate jumps
Find source of broken link
Fix if on-site
Contact source if off-site (or post good link if it’s user-generated content and use chance to
engage with potential visitors!)
Rejected promo code
●
●
●
If ‘bad’ code is common because it’s confusing (0/o/O, i/I/l) configure an extra matching code
Learn lesson and don’t use ambiguous codes
If ‘bad’ code is one-off on user-generated site visit the site, apologise and post a ‘good’ code
(engage with visitors)

29.
Dashboards: really powerful
Great example of when to use a Real
Time Widget:
●
●
●
●
●
●
●
●
●
●
RT: active visitors with errors
RT: the active 404 error pages
RT: the other active error pages
External source/medium of
errors
External referring sites (only)
Adwords target errors
Error conversion rate
Error completions
Source of internal bad links
(Goal previous step!)
Bad link requested (Goal
completion URL)
Include just conversion in other
dashboards

30.
Custom Reports: Vital
Filter to focus on what counts

31.
Pick only the metrics you need

32.
Those checkout errors

33.
Comparison view?

34.
Actual numbers = actual money

35.
Different people - (source/medium drill down)

36.
Segment ideas
‘User’ scope segments:
Potential to compare effect of error messages over different visits
‘Sequence’ segment rules:
Possibility for looking at impact of errors at different stages of visit
Original segments:
Great for looking at impact of with/without error(s) on macro-conversion

37.
Content Grouping Thoughts
Same question as with VPV path
Group errors with original page
or
Group errors as dedicated group

38.
Summary
Errors are a proxy for usability problems
Each error message is a small cry of pain from your visitor
Each error message is a tiny dent in your brand’s reputation
Tracking errors in GA lets you see where the points of friction are
Measuring errors allows you to prioritise improvements
Improvements likely to bring enough short-term income to pay for work
Real big pay-off will be in customer lifetime value in multi-channel world

39.
Resources
Bundle of all my Superweek2014 Links:
http://bit.ly/Superweek2014TLB
Great article on form validation: http://bit.ly/Superweek2014Forms
Includes link to Luke Wroblewski post
“When compared to our control version, the inline validation form with the best performance showed compelling improvements
across all the data we measured. Specifically, we saw:
●
●
●
●
●
a 22% increase in success rates,
a 22% decrease in errors made,
a 31% increase in satisfaction rating,
a 42% decrease in completion times, and
a 47% decrease in the number of eye fixations.”
Great advice on error messages: http://bit.ly/SuperWeek2014ErrorIdeas
My first blog post on the subject: http://bit.ly/Superweek2014Errors
Includes very useful discussion thread