0
Web Analytics
and Usability Testing:
Triangulate your way
to better design
recommendations
by Eva Kaniasty
What’s your favorite
method?
Web Analytics Explosion
Who wins the battle?
Usability Testing Sucks
Usability Testing
Small Sample
Results not
Generalizable
Recruiting Bias
Unrepresentative
Participants

Artificial Situati...
Usability Testing

3 users experienced X…
Consider changing X to Y.
Usability Testing is
Disconnected
Web Analytics are
Awesome!
Web Analytics
Representative/
Large Sample Size

Real Goals
Real Tasks

Real Users
“In the Wild”

No Moderator
Automatic D...
Numbers and graphs:
universally beloved
Analytics rundown
Visitor/Session Tracking
Visitor/Session Tracking
Data Gathered

Uses

•Some visitor characteristics:
locations, browsers, OS
•Visit & page view tr...
Clickmaps/Heatmaps
Clickmaps/Heatmaps
Data Gathered

Uses

•Info not on just which
pages are being accessed,
but from where
•Full picture of ...
Email Data Mining
Email Data Mining
Data Gathered

Uses

•Demographics
•Geography
•User Profiles
•Site usage data:
accounts on amazon,
faceb...
Session Recording
Session Recording
Data Gathered
•View all interactions
with page, including
mouse movements
•Scroll maps
•Form analytics

...
A/B or Multivariate Testing
A/B or Multivariate Testing
Data Gathered

Uses

•Relative success of design
alternatives: impact on
conversion and other
...
Unmoderated Usability Testing
Unmoderated Usability Testing
Data Gathered

Uses

•Immediate feedback from
real users as they engage
with the website

•A...
Custom Metrics
Custom Metrics
Data Gathered

Uses

•Functionality usage
•Form errors
•Patterns of use
•Conversion measurement

•Sky is th...
The what.

But why?
Analytics Suck
Web Analytics
Limited information
about context
No info about user
intent

One-dimensional
view (i.e. Page-level
data only...
Usability Testing is Awesome
Usability Testing
Rich data/ holistic
view
Moderator can
probe to clarify
intent

Info about goals
Ability to test new
int...
Triangulate!
Triangulate

Usability Testing

• Confirm Problems
• Extent
• Severity

• Focus
• Deep Data
• Goals and Intent

Analytics
Case Study

Usability Testing

• Detail on types/extent of
errors
• True abandonment
rates
• Visual hierarchy
problems

• ...
Triangulate

Usability Testing

Analytics

Surveys
Questions first
CSI or Law & Order?
Questions?

Twitter: @kaniasty
kaniasty@gmail.com
kaniasty.com
Resources
• Google Analytics
• Crazyegg
• Clicktales
• Keynote Systems
Upcoming SlideShare
Loading in...5
×

Analyticspresentation 090613183722-phpapp02

147

Published on

Published in: Technology, Design
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
147
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Web analytics
    -have come a long way
    Not just log analysis
    Sophisticated
    See where someone is clicking
    Replay user sessions
    survey people as they are interacting with the site.
  • What does this mean for us?
    It means we have to understand these methods, and understand when to use them
  • What does this mean for us?
    It means we have to understand these methods, and understand when to use them
  • Goal tracking
    Custom graphs
  • Goal tracking
    Custom graphs
  • Full picture of users’ interactions with the page
    Info not on just which pages are being accessed, but from where
    Tests design or link text alternatives
    Measure of user interest/engagement
    Poor man’s eye tracking
  • Goal tracking
    Custom graphs
  • Also known as cyber-stalking on a grand scale
    -
  • Goal tracking
    Custom graphs
  • Goal tracking
    Custom graphs
  • Goal tracking
    Custom graphs
  • Conversion
    Drop-off
    Feature Usage
    Error rates
  • Goal tracking
    Custom graphs
  • Goal tracking
    Custom graphs
  • Goal tracking
    Custom graphs
  • Do it yourself
    Make friends with the data guy
    Plug into
  • Be method agnostic
  • Be method agnostic
  • Transcript of "Analyticspresentation 090613183722-phpapp02"

    1. 1. Web Analytics and Usability Testing: Triangulate your way to better design recommendations by Eva Kaniasty
    2. 2. What’s your favorite method?
    3. 3. Web Analytics Explosion
    4. 4. Who wins the battle?
    5. 5. Usability Testing Sucks
    6. 6. Usability Testing Small Sample Results not Generalizable Recruiting Bias Unrepresentative Participants Artificial Situation Task Design Bias Moderator Required Moderator Bias
    7. 7. Usability Testing 3 users experienced X… Consider changing X to Y.
    8. 8. Usability Testing is Disconnected
    9. 9. Web Analytics are Awesome!
    10. 10. Web Analytics Representative/ Large Sample Size Real Goals Real Tasks Real Users “In the Wild” No Moderator Automatic Data Collection/Analysis
    11. 11. Numbers and graphs: universally beloved
    12. 12. Analytics rundown
    13. 13. Visitor/Session Tracking
    14. 14. Visitor/Session Tracking Data Gathered Uses •Some visitor characteristics: locations, browsers, OS •Visit & page view trends •Engagement measures: time on page •Search data: keyword referrals •Custom tracking: detailed information on paths/goal conversion •Confirm Usability Test findings •Inform usability testing tasks
    15. 15. Clickmaps/Heatmaps
    16. 16. Clickmaps/Heatmaps Data Gathered Uses •Info not on just which pages are being accessed, but from where •Full picture of users’ interactions with the page •Measure of user interest/engagement •Some segmentation by browser/screen size/time to click •IA •Visual hierarchy •Interface refinement
    17. 17. Email Data Mining
    18. 18. Email Data Mining Data Gathered Uses •Demographics •Geography •User Profiles •Site usage data: accounts on amazon, facebook, linkedin, etc. •Utest Recruiting Profiles •Personas
    19. 19. Session Recording
    20. 20. Session Recording Data Gathered •View all interactions with page, including mouse movements •Scroll maps •Form analytics Uses •Identify potential usability problems •QA/Browser Testing
    21. 21. A/B or Multivariate Testing
    22. 22. A/B or Multivariate Testing Data Gathered Uses •Relative success of design alternatives: impact on conversion and other engagement measures •IA •Visual hierarchy •Interface refinement
    23. 23. Unmoderated Usability Testing
    24. 24. Unmoderated Usability Testing Data Gathered Uses •Immediate feedback from real users as they engage with the website •Alternative to usability testing, when large samples are required •Compensate on weaknesses of survey methods
    25. 25. Custom Metrics
    26. 26. Custom Metrics Data Gathered Uses •Functionality usage •Form errors •Patterns of use •Conversion measurement •Sky is the limit
    27. 27. The what. But why?
    28. 28. Analytics Suck
    29. 29. Web Analytics Limited information about context No info about user intent One-dimensional view (i.e. Page-level data only from Clickmaps) Overwhelming amounts of data to analyze Data on current state only
    30. 30. Usability Testing is Awesome
    31. 31. Usability Testing Rich data/ holistic view Moderator can probe to clarify intent Info about goals Ability to test new interfaces/ prototypes
    32. 32. Triangulate!
    33. 33. Triangulate Usability Testing • Confirm Problems • Extent • Severity • Focus • Deep Data • Goals and Intent Analytics
    34. 34. Case Study Usability Testing • Detail on types/extent of errors • True abandonment rates • Visual hierarchy problems • Form errors • No abandonment • Confusion about messaging Analytics
    35. 35. Triangulate Usability Testing Analytics Surveys
    36. 36. Questions first
    37. 37. CSI or Law & Order?
    38. 38. Questions? Twitter: @kaniasty kaniasty@gmail.com kaniasty.com
    39. 39. Resources • Google Analytics • Crazyegg • Clicktales • Keynote Systems
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×