Combining Methods: Web Analytics and User Testing

Combining Methods Web Analytics and User Testing Martijn Klompenhouwer & Adam Cox UPA - May 27 th , 2010

Combining Methods Web Analytics and User Testing Martijn Klompenhouwer & Adam Cox UPA - May 27 th , 2010 Who are we..? Web Analytics User Research Why you should combine methods Practical examples Why you should try it too

About User Intelligence

Martijn Klompenhouwer UX researcher 11 years UX experience 8 years at User Intelligence

Adam Cox Web Analyst 7 years UX / Analyst experience 3 years at User Intelligence

User Intelligence
It’s our goal to design interactive products that provide great User Experiences, based on knowledge of both the users and the business.
Services we offer:
Research
Design
Optimize
www.userintelligence.com

How to conduct Global User Research written by partners of the UXalliance and other experts
Available since November 2009 published by Morgan Kaufman
Handbook of Global User Research

About Web Analytics

"Web Analytics is the measurement , collection , analysis and reporting of Internet data for the purposes of understanding and optimizing Web usage.“ - The Official WAA Definition of Web Analytics
Web Analytics Definition Measurement Data Collection Analysis Reporting

What does the definition mean? Where are your users coming from? What are they doing? Where and when are they leaving?

Some examples of Web Analytic measurements Referrals Visitors, visits and page views Popular pages Bounce rates Funnel analysis Path Analysis

Issues with Web Analytics  Data often perceived as boring statistics  Implementation of WA tool not always correct  Reports: just the facts, no insights, no actions

Web Analytics is quantitative in nature To get insights, interpretation of the numbers is needed

About User Research

Some examples of User Research methods Field observations Personas Diary studies Card sorting Expert Review Interviews Focus groups User Testing

Issues with User Research methods  Data usually from small numbers  Most methods take a snapshot in time  Difficult to capture some behavior  Setting sometimes artificial (e.g. lab test)

User Research is qualitative in nature You get the ‘why’, but not the ‘big numbers’…

So, why should you combine these two methods?  User Research findings can help interpret web data  Web data can help focus the User Research  More certainty of findings (based on two sources)

 The methods complement each other Quantitative vs. Qualitative

Let’s clarify with some examples…

Example 1: Recruiting the right participants Visitor Analysis Analyzing current traffic on website gives clues about audience Target audience Who to invite to participate in a test? Better insights More knowledge on users helped in defining recruitment profiles

Example 2: Test scenarios User journeys Homepage was not the main entry point Writing test script Creating realistic test scenarios: how is site used now..? Google scenario Testing outcomes of a common scenario

Example 3: Explaining abandonment rates Funnel analysis Sales funnel analysis revealed in which steps users left the flow Target test tasks Knowledge enabled us to concentrate usability test on those steps More focus Finding reasons for abandonment and looking for solutions

Example 4: Use of Advanced features Not measured Use of feature was known, but no data on the use of settings Popular settings? Which options are used and are there any patterns or issues? Collect data Decided not to use too much test time: wait for proper measurements

Example 5: Validating findings Impact analysis Quantified issue using data of thousands of visitors Big issue? Only 2 out of 10 test participants had this problem End of discussion Discussion changed from: “Is that an issue?” to: “Let’s solve it!”

Some additional examples… Effectiveness of segmentation Verifying user feedback Unintended user-flows Interpreting bounce rate data

Why you should try it too  Combining methods works! You can tell a story backed up with data  One report: Clear actions and no conflicting recommendations  Useful in different stages of projects (Research, Design, Optimize)  The combination works both ways! Web Analyst and User Researcher benefit from each other  Web Analytics can be used with many User Research methods

Just remember…  Fruitful results can already be gained from basic analysis You don’t need to be a Web Analytics expert, but it helps ;-)  Web Analytics tools do not magically provide the insights...  Measure the impact of changes (Optimize)  The UX team should make more use of Web Analytics! It shouldn’t just belong to the IT or Marketing departments

What we want you to take away…  Find out what web data is available  Use it!..... Just try it! No more excuses ;-)  Web data will help you in different stages of a project  If important data is missing, try to get it measured in the future  Integrate Web Analytics into your process and methods

user intelligence Amsterdam office www.userintelligence.com Thank you! Martijn Klompenhouwer [email_address] Adam Cox [email_address]

http://www.userintelligence.com	270
http://www.slideshare.net	88
http://www.usabilityblog.de	25
http://www.onetooneinteractive.com	9
http://bitpakkit.blogspot.com	5
http://www.userintelligence.nl	4
http://netvantagemedia.com	2
http://static.slidesharecdn.com	1
http://feeds2.feedburner.com	1
http://userintelligence.nl	1

Combining Methods: Web Analytics and User Testing

by User Intelligence on May 27, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

10 Embeds 406

Statistics

Combining Methods: Web Analytics and User Testing — Presentation Transcript