Web Analytics and Usability Testing: Triangulate your way to better design recommendations by Eva Kaniasty

What’s your favorite method?

Web Analytics Explosion

Who wins the battle?

Usability Testing Sucks

Usability Testing Moderator Required Moderator Bias Recruiting Bias Unrepresentative Participants Artificial Situation Task Design Bias Small Sample Results not Generalizable

Usability Testing 3 users experienced X… Consider changing X to Y.

Usability Testing is Disconnected

Web Analytics are Awesome!

Web Analytics No Moderator Automatic Data Collection/Analysis Real Users “ In the Wild” Real Goals Real Tasks Representative/ Large Sample Size

Numbers and graphs: universally beloved

Analytics rundown

Visitor/Session Tracking

Visitor/Session Tracking Confirm Usability Test findings Inform usability testing tasks Some visitor characteristics: locations, browsers, OS Visit & page view trends Engagement measures: time on page Search data: keyword referrals Custom tracking: detailed information on paths/goal conversion Uses Data Gathered

Confirm Usability Test findings

Inform usability testing tasks

Some visitor characteristics: locations, browsers, OS

Visit & page view trends

Engagement measures: time on page

Search data: keyword referrals

Custom tracking: detailed information on paths/goal conversion

Clickmaps/Heatmaps

Clickmaps/Heatmaps IA Visual hierarchy Interface refinement Info not on just which pages are being accessed, but from where Full picture of users’ interactions with the page Measure of user interest/engagement Some segmentation by browser/screen size/time to click Uses Data Gathered

IA

Visual hierarchy

Interface refinement

Info not on just which pages are being accessed, but from where

Full picture of users’ interactions with the page

Measure of user interest/engagement

Some segmentation by browser/screen size/time to click

Email Data Mining

Email Data Mining Utest Recruiting Profiles Personas Demographics Geography User Profiles Site usage data: accounts on amazon, facebook, linkedin, etc. Uses Data Gathered

Utest Recruiting Profiles

Personas

Demographics

Geography

User Profiles

Site usage data: accounts on amazon, facebook, linkedin, etc.

Session Recording

Session Recording Identify potential usability problems QA/Browser Testing View all interactions with page, including mouse movements Scroll maps Form analytics Uses Data Gathered

Identify potential usability problems

QA/Browser Testing

View all interactions with page, including mouse movements

Scroll maps

Form analytics

A/B or Multivariate Testing

A/B or Multivariate Testing IA Visual hierarchy Interface refinement Relative success of design alternatives: impact on conversion and other engagement measures Uses Data Gathered

IA

Visual hierarchy

Interface refinement

Relative success of design alternatives: impact on conversion and other engagement measures

Unmoderated Usability Testing

Unmoderated Usability Testing Alternative to usability testing, when large samples are required Compensate on weaknesses of survey methods Immediate feedback from real users as they engage with the website Uses Data Gathered

Alternative to usability testing, when large samples are required

Compensate on weaknesses of survey methods

Immediate feedback from real users as they engage with the website

Custom Metrics

Custom Metrics Sky is the limit Functionality usage Form errors Patterns of use Conversion measurement Uses Data Gathered

Sky is the limit

Functionality usage

Form errors

Patterns of use

Conversion measurement

The what. But why?

Analytics Suck

Web Analytics Data on current state only Overwhelming amounts of data to analyze One-dimensional view (i.e. Page-level data only from Clickmaps) Limited information about context No info about user intent

Usability Testing is Awesome

Usability Testing Ability to test new interfaces/ prototypes Moderator can probe to clarify intent Info about goals Rich data/ holistic view

Triangulate!

Triangulate Focus Deep Data Goals and Intent Confirm Problems Extent Severity Usability Testing Analytics

Focus

Deep Data

Goals and Intent

Confirm Problems

Extent

Severity

Case Study Form errors No abandonment Confusion about messaging Detail on types/extent of errors True abandonment rates Visual hierarchy problems Usability Testing Analytics

Form errors

No abandonment

Confusion about messaging

Detail on types/extent of errors

True abandonment rates

Visual hierarchy problems

Triangulate Surveys Analytics Usability Testing

Questions first

CSI or Law & Order?

Questions? Twitter: @kaniasty [email_address] kaniasty.com

Resources Google Analytics Crazyegg Clicktales Keynote Systems

Google Analytics

Crazyegg

Clicktales

Keynote Systems