1. Be able to apply a framework for balancing quantitative
and qualitative research methods.
2. Have a grasp of several key Google Analytics tools that
are most relevant to UX practices.
3. Learn through labs, workshops and case studies how web
analytics is applied to actual UX projects.
Business Goals /
Websites Run on Goals
A “Goal” is a measurable outcome resulting in a user
completing some desired activity on your website. Typical
– Confirmation page at the end of a sales transaction.
– Thank-you page after filling out a contact or quote request form.
– Application or content downloads.
– Playing a game or watching a video on a site.
The Best Goals are Existentially Critical
– Is it a “Holiday Bonus” question?
– If this goal stopped happening, would your organization (or your
department) still exist?
– Most companies should have a few goals filtered through many
BUT...Goals Can’t Say Why or How
– Goals describe behavior badly.
– Goals can’t describe intent at all.
A hypothesis suggests a functional change based on a
theory of action that has a measurable outcome.
Goals and Hypotheses
– Goals link clear up or down numbers to the outcome of
a specific site behavior.
– Hypotheses provide the testable narrative about how
the user’s experience on the site affects those goals.
– The testable narrative does not have to BE a goal, but
should specifically be IN SERVICE OF a goal.
What is Statistically Significant?
Determining whether the differences seen in data is more
than random chance.
Why Use It?
- Addresses the HiPPO problem.
- Saves time by getting to outcomes faster.
- Uncovers subtle effects.
- Confronts our own biases about aesthetic and design.
Quantifying a “measurable outcome”
If goals have been set up properly, outcomes can be
measured using simple statistics. And simple is all we need!
The recommended statistical method for UX professionals is
the A/B test.
Appropriate A/B Tests Should:
- Be immediately apparent to anyone looking comparing
- Be defined in a functional UX way.
- Represent a set of coherent conceptual changes against
a single hypothesis.
Typical A/B Test Candidates
Question Testing For Best Testing Tool
Is the navigation layout
affecting conversion rate?
Google Analytics Experiements (Not out of
the box but you can hack it)
Which of two landing pages
by page version
Google Analytics Experiments
Which User segment
Advanced segments, Confidence Interval
What Statistics Don’t Tell You
- Why a test failed. This can be just as critical as a
- Why it succeeded.
- How to thoughtfully create testable hypotheses.
Abstracted UX Process
● Establish clearly the “Why” and
“Who” for the site.
● Organizational goals are
articulated and prioritized.
● The audience is clearly identified.
● The ultimate measures of success
are agreed upon.
Research and Analysis
● Research how your users
approach your current site.
● Evaluate the website’s design and
● Synthesize the details into high-
level models that represent both
user needs and a high-level
Design and Testing
● Specify the site structure.
● Determine the ways in which the
goals will be achieved through site
Qualitative and Quantitative Research