3. About this presentation
Part 2: Using analytics to influence change
How analytics can be used to influence change in a
business process or web service.
◇ KPI’s and Conversion Rates.
◇ Multivariate Testing.
◇ Custom Reporting.
◇ User Experience.
This template is free to use under Creative Commons Attribution license. If you use the graphic assets (photos,
icons and typographies) provided with this presentation you must keep the Credits slide.
Part 1: Analysis of an IPO service
Show the steps taken to set up and analyse completion
rates and drop out points and identify areas of change.
Within Google Analytics, we will look at:
◇ Conversions: How to set up goals.
◇ Goal Flow - where are we losing them?
◇ Time on page and bounce rates.
◇ How can we improve?
4. Analysis of an IPO service
Let’s start by looking at the new Register a Design service
1
5. Conversions:
How to track completion rates
❖ In Google Analytics, we can set up Goals that allow
us to track how often users complete a specific
action.
❖ This is useful for monitoring key performance
indicators and revenue generation.
❖ By creating a funnel, we are able to see where the
drop out points are and determine how to improve.
Google has made this really easy by creating templates
for the most common actions...
6. ❖ You can select from a goal
template or select Custom for
more complicated tasks.
❖ For the purposes of tracking how
many people have registered a
design, the Acquisition template
‘Create an account’ is perfect.
7. ❖ This goal template can track
various actions, such as:
❖ The length of time on page, if
you’re worried some steps may
take too long.
❖ The number of pages per
session, can be useful for seeing
how engaged users are with your
service.
❖ An event or destination page
such as the final summary of this
prototype.
8. Here’s one I made earlier!
❖ This is the goal I set-up to track this
service.
❖ To correctly monitor the drop off points,
I’ve tried to be concise when including the
steps in the goal funnel.
❖ Due to how the service works, there will
be loopbacks to previous steps. Google
Analytics will show where this occurs.
9. “
A funnel shows the path your traffic traveled
through towards a Goal conversion.
We can view the Goal Flow in Analytics to help us see
if users are navigating the content as expected, or if
there are problems, such as high drop-off rates or
unexpected loops.
10. Here’s what the
Goal Flow looks like:
Whilst this is a top level overview, we can see how users
progress along the funnel.
The red bits are the drop off points that we’ll need to look at
in more detail.
11. “
Where are we losing them?
From the 46 users that entered the funnel, 20
users completed the action to giving a conversion
rate of 43%.
This is a good percentage in the real world but
given the small dataset and the context in which
the data was gathered, we can expect it to be
higher.
However, were test subjects instructed to leave
the test at a certain point to give us data for
expected drop off points?
12. Single Owner drop off
❖ Intellectual Property Office have
indicated they suspect there is a
usability issue where a single owner is
applying.
❖ We can see here a drop off at these two
steps to reinforce that notion.
❖ We see a 2 user drop off when
prompted to select the owner of the
design and a further 5 exit when asked
for the company contact details.
13. Why do we lose them at Defer?
❖ Another noticeable drop off point is
when we ask the user if they want to
defer registering the design.
❖ Having run through the process several
times, I can’t see a usability reason for
this drop off.
❖ Whilst unfamiliar with the product,
perhaps the user is unsure of what to
choose and elects to come back at a
later time when they can say for
definite.
14. “
No back button?
In a 2010 study, Firefox creator Mozilla found that
the back button is the most used browser function:
“By a landslide the 'Back' button was the most clicked of all navigation buttons
which include the Back, Forward, Reload, Stop, and Home buttons. Across
Windows, Mac and Linux 93.1 percent of users clicked the button at least once over
the course of a five-day period. In total the study reported that users clicked on the
back button 66 times over the course of five days.”
Whilst this service does allow the user to change details when
they reach summary milestones, there is no option to go back to
change details. Physical use of the back button appears to break
the service, necessitating clearing the browser cache.
Could this to be to blame for some of the drop offs?
16. An effective web analysis
process
Data collection
Data
becomes
information
Develop
KPI’s
Strategy
Basic counts:
❖ Visitors
❖ Referers
❖ Key words
Ratios lead metrics:
❖ Unique
visitors
❖ Bounce rate
❖ Time on page
Counts/Ratios meet
strategy:
❖ Conversion
rate
❖ Abandon rate
❖ Revenue
Goals/objective:
❖ Save money
❖ Make money
❖ Increase
conversions
17. Web analytics is all about
human behaviour
Behaviours are...
Something that we can
detect and record.
Blue
Actions or goal driven
events.
Red
Reactive responses to
contextual stimuli.
Google Analytics can record all of the above behaviours but it can’t record
affective, cognitive or situational aspects. For that, we need to look deeper.
18. We know the drop off points
We can see we lose people at
the Defer point and then later
when they have to select the
Owner of the Design.
Based on the
figures so far...
We know the completion rate
We can see how many people
reach the end of the task.
We can draw basic conclusions based on this data and what we
suspect going on past experience. However, can we say for
certain how to change this service for the better?
19. Multivariate
TestingIt’s easy to come to a simple
conclusion based on one set
of data. But to come to the
right conclusion, we need
more numbers.
20. “
Google made $200m by testing 42
shades of blue
“In the world of the hippo, you ask the chief designer or the
marketing director to pick a blue and that’s the solution. In the
world of data you can run experiments to find the right
answer…We ran ‘1%’ experiments, showing 1% of users one blue,
and another experiment showing 1% another blue. And actually, to
make sure we covered all our bases, we ran forty other
experiments showing all the shades of blue you could possibly
imagine and we saw which shades of blue people liked the most,
demonstrated by how much they clicked on them. As a result we
learned that a slightly purpler shade of blue was more conducive
to clicking than a slightly greener shade of blue, and gee whizz,
we made a decision…But the implications of that for us, given the
scale of our business, was that we made an extra $200m a year in
ad revenue.”
Dan Cobley; Marketing Director, Google UK
21. Every MVT is unique
but it’s most often
used for...
Call To Actions
Should we change the wording?
Are the colours working? Should it
be placed elsewhere on page?
Headings or Descriptions
Are we using the right font? Is the
tone correct for our audience?
Forms
Is it too long? Are we using the
right type of fields? Should we
paginate?
Page Layout and Style
Are our column ratios correct?
Should we use different fonts or
colours?
Images
Are the ones we using
inspirational or on brand?
And, of course: £££
Copy
Too much or too little text? How
does it affect white space? Does
the tone match the message?
22. How to set up MVT split tests
❖ Develop at least two versions of a page.
❖ Randomly divide users into groups for each version.
❖ Show each group a different version.
❖ Track how each group performs.
❖ Evaluate each version based on results.
❖ Repeat as needed.
❖ Go with the winning result.
23. “
What if we can’t implement MVT or
A/B testing?
If you really want to influence change, MVT is the
best way to go because you can back up your
decisions with data. It’s no longer “We should do
this because I think…” it’s “We should do this
because I know…”
However, we don’t always have that luxury.
A lot of changes based on web analytics without
using split testing is based on gutfeelings drawn
from experience. We can still look at counts and
ratios and draw simple conclusions on how to
affect the outcomes.
We just can’t back it up with quantifiable data.
25. Credits
Special thanks to all the people who made and released
these awesome resources for free:
◇ Presentation template by SlidesCarnival
◇ Portrait photograph by Simon Ayre
